“Twitter’s link service at http://t.co is used to better protect users from malicious sites that engage in spreading malware, phishing attacks, and other harmful activity. A link converted by Twitter’s link service is checked against a list of potentially dangerous sites. When there’s a match, users can be warned before they continue.”

Declan McCullagh reports that Twitter announced in an email message that when someone click “on these links from Twitter.com or a Twitter application, Twitter will log that click.” Such information is extremely valuable. Give Twitter’s tens of millions of active users, just knowing how often certain URLs are clicked by people indicates what entities and topics are of interest at the moment.

“Our link service will also be used to measure information like how many times a link has been clicked. Eventually, this information will become an important quality signal for our Resonance algorithm—the way we determine if a Tweet is relevant and interesting.”

Associating the clicks with a user, IP address, location or device can yield even more information — like what you are interested in right now. Moreover, Twitter now has a way to associate arbitrary annotation metadata with each tweet. Analyzing all of this data can identify, for example, communities of users with common interests and the influential members within them.

Note that Twitter has not said it will do this or even that it will record and keep any user-identifiable information along with the clicks. They might just log the aggregate number of clicks in a window of time. But going the next step and capturing the additional information would be, in my mind, irresistible, even if there was no immediate plan to use it.

Search engines like Google already link clicks to users and IP addresses and use the information to improve their ranking algorithms and probably in many other ways. But what is troubling is the seemingly inexorable erosion of our online privacy. There will be no way to opt out of having your link wrapped by the t.co service and no announced way to opt out of having your clicks logged.

Providers of first generation semantic technologies calculate rather basic “semantic networks” by co-occurency analysis which results sometimes in  disappointing results. Bearing in mind that Google just bought a company (“Google buys Metaweb“) which has been working on one of the largest knowledge bases in the world, we could assume that some of the last miles towards a semantic search engine can be achieved by applying thesauri or other structured knowledge bases.

A demo application was recently developed by PoolParty team where one can find out how thesauri will improve search results on top of second generation semantic technologies. With PoolParty SKOS based controlled vocabularies can be managed and also can be enriched with linked data. PoolParty Tag & Content Recommender analyzes virtually any text or website to recommend corresponding tags, concepts from (in this case) STW (Standard Thesaurus für Wirtschaft), DBpedia and respective articles from Wikipedia.

STW which was developed by the German National Library of Economics (ZBW) provides vocabulary on any economic subject: about 6,000 standardized subject headings and about 18,000 entry terms to support individual keywords.

This background knowledge is used in this demo app to improve the search for similar documents dramatically:

Similarity between two documents can be calculated not only on a key-phrase basis but also on a rather conceptual basis. Even if two documents do not have one single word or phrase in common they can be identified as “similar documents”.

This can be achieved because thousands of important relations between economic subjects are represented in the domain specific thesaurus. Thus, in this special case best results are achieved with documents from economics (for instance from Econstor) but of course for other recommender systems thesauri from other domains can be used instead of STW.

Nevertheless, also this approach can be improved and this development is underway: SKOS thesauri enriched with Linked Data do an even better job. This kind of third generation semantic technologies are currently developed by LASSO project and LOD2 project, two innovative projects in the area of linked data and the semantic web.

To get to the ECS building atrium, walk from ITE to the ECS building from the second floor of ITE and you will enter the atrium. Please come out on the day before classes and enjoy some food while catching up.

Everyone is also encouraged to also attend Convocation 2010, the formal opening of the academic year at UMBC, from 3:30 to 4:30 pm in the Retriever Activities Center. President Hrabowski will address the gathering and Wendy Salkind, Presidential Teaching Professor for 2010-13, will make brief remarks.

After Convocation, all faculty, staff and students are invited to yet another free Community Picnic on the UMBC Quad from 4:30 to 7:00pm. The rain location will be the Residence Life Dining Hall.

I’ve been at Yahoo! nearly 5 years so it is both a happy and sad time for me, and I wish all the excellent people I worked with the best of luck in future.

Here is a summary of the main changes:

• Silicon Valley -> San Francisco
• 15,000 staff -> 100 staff
• Architect -> Software engineer
• strategizing, meeting -> coding
• Powerpoint, OmniGraffle, twiki -> emacs, eclipse, …?
• (No coding!) -> Python, Java, Hadoop, Cassandra, …?
• Sunny days -> Foggy days
• 15 min commute -> 2.5hr commute (until I move to SF)
• Public company -> private company

Exciting!

I was luckily enough to meet David Huynh as a workshop recently and chatted to him briefly about another aspect of the Gridworks: its ability to match field values in a dataset to entities in Freebase, e.g. identifying a place based on just it’s name. Within Gridworks this process is known as “reconciliation”.

Reconciliation is an important step for generating good Linked Data as you’ll often need to correlate values in a dataset with URIs in existing datasets in order to generate links. E.g. matching company names to their URIs. While it is possible to generate identifiers algorithmically during a conversion this typically just defers the reconciliation work until a later stage, when you carry out cross-linking to introduce equivalence links.

Recognising that the ability to introduce new reconciliation services would be a powerful extension to Gridworks, David Huynh has been creating a draft specification that will allow third-parties to create and deploy their own reconciliation services. He’s been documenting his progress on implementing the client side of this protocol and has published a testing service.

It occurred to me that the reconciliation API is essentially a structured search over a dataset and thus could be implemented against the search interface exposed by Talis Platform stores. The RSS 1.0 feeds that the Platform returns includes enough information to rank and filter results as required by the API.

I’ve created a simple Ruby application, using the Sinatra web framework, that implements the reconciliation API for any Talis Platform store. You can find the code on github if you want to have a play with it. As I note in the README there are some areas where customisation is useful to get the most from the service. So while in principle it can be used against any existing Platform store you can create a simple JSON config to tweak it for particular datasets.

There’s a live version of the code running one my server here: http://ldodds.com/gridworks/.

That page has a simple API console for carrying out queries, but consult the draft specification for more details. I think I’ve covered all of the basic features (but bug reports welcome!). Consult the README for notes on configuration options and implementation decisions.

As a simple illustration, lets say that I have the value “Bath” in a dataset and want to match that to some area in the UK administrative geography. This information is available from the Linked Data exposed by statistics.data.gov.uk and this happens to be hosted in this platform store. The reconciliation API we need can therefore be found at: http://ldodds.com/gridworks/govuk-statistics/reconcile. An HTTP GET on that location retrieves the service metadata.

If we use the API explorer we can use a simple HTML form to try out examples. Select govuk-statistics from the Store drop-down and then type Bath into the search box. You’ll get this result. This is not very readable by default, so if you’re using Firefox I recommend you install the JSONView extension which provides a nicely formatted display.

Our initial search returns a number of results. The highest ranked of these being the Westminster Constituency for Bath. That seems like a pretty good initial result to me. As it is the most relevant result in the search it’s marked as an exact match, so once integrated with Gridworks it will capture and store the reconciled identifier for you.

However, we may know that in the imaginary dataset we’re working with, that a particular field doesn’t contain names of constituencies. It may instead refer to a Local Education Authority. We can refine our search by adding the URI that defines that type of resource into the type field in the API explorer.

Try pasting in http://statistics.data.gov.uk/def/geography/LocalEducationAuthority into the post and running the search again. You’ll find that this time you get a single result, which is Bath and North East Somerset. Job done.

Of course, to get the most from this you need to know what URIs you can use for filtering by types (and properties). But this is something that the Gridworks UI will help with. It can integrate with “suggestion services” that can be used to help map values to a properties and types within a schema. I’ll be looking at how to expose those as my next piece of work.

Hopefully you can see how the overall system works. Feel free to have a play with the API to try it out for yourself. If you have comments on the implementation then I’d love to hear them, but I’d suggest that comments on the specification are best addressed to the gridworks mailing list.

I also suspect the Reconciliation API has uses outside of just Gridworks. For example, I wonder how easy it would be to introduce reconciliation into Google Spreadsheets using Google Apps Script? It’s also another nice demonstration of how easy it is to map simple RESTful APIs onto RDF datasets, this implementation works for any data in the Platform, no matter what schema it confirms with. Neat.

The San Jose Mercury News reports:

“Tuesday, nearly 13 months after Yahoo and Microsoft announced plans to collaborate on Internet search in hopes of challenging Google’s market dominance, the two companies announced that the results of all Yahoo English language searches made in the United States and Canada are coming from Microsoft’s Bing search engine. The two companies are still racing to complete the transition of paid search, the text advertising links that run beside and above the standard search results, before the make-or-break holiday period — a much more difficult task.”

Combining the traffic from Microsoft and Yahoo will give the Bing a more significant share of the Web search market. That should help them by providing both companies with a larger stream of search related data that can be exploited to improve search relevance, ad placement and trend spotting. It will also help to foster competition with Google focused on developing better search technology.

Hopefully, Bing will be able to benefit from the good work done at Yahoo! on adding more semantics to Web search.

Security researchers J. Alex Haldeman and Ariel Feldman demonstrated PAC-MAC running on a Sequoia voting machine last week at the EVT/WOTE Workshop held at the USENIX Security conference in DC.

Amazingly, they were able to install the game on a Sequoia AVC Edge touch-screen DRE (direct-recording electronic) voting machine without breaking the original tamper-evident seals.

Here’s how they describe what they did on Haldeman’s web site:

“What is the Sequoia AVC Edge?

It’s a touch-screen DRE (direct-recording electronic) voting machine. Like all DREs, it stores votes in a computer memory. In 2008, the AVC Edge was used in 161 jurisdictions with almost 9 million registered voters, including large parts of Louisiana, Missouri, Nevada, and Virginia, according to Verified Voting.

What’s inside the AVC Edge?

It has a 486 SLE processor and 32 MB of RAM—similar specs to a 20-year-old PC. The election software is stored on an internal CompactFlash memory card. Modifying it is as simple as removing the card and inserting it into a PC.

Wouldn’t seals expose any tampering?

We received the machine with the original tamper-evident seals intact. The software can be replaced without breaking any of these seals, simply by removing screws and opening the case.

How did you reprogram the machine?

The original election software used the psOS+ embedded operating system. We reformatted the memory card to boot DOS instead. (Update: Yes, it can also run Linux.) Challenges included remembering how to write a config.sys file and getting software to run without logical block addressing or a math coprocessor. The entire process took three afternoons.”

You can find out more from the presentation slides from the EVT workshop, Practical AVC-Edge CompactFlash Modifications can Amuse Nerds. They sum up their study with the following conclusion.

“In conclusion, we feel our work represents the future of DREs. Now that we know how bad their security is, thousands of DREs will be decommissioned and sold by states over the next several years. Filling our landfills with these machines would be a terrible waste. Fortunately, they can be recycled as arcade machines, providing countless hours of amusement in the basements of the nations’ nerds.”

1. Support more of the new W3C SPARQL working drafts of 1 June 2010 for SPARQL 1.1 Query and SPARQL 1.1 Update.
2. Support building with Raptor V2 API as well as Raptor V1 API..

The main change is to start to add to Rasqal’s APIs and query engine changes for the new SPARQL 1.1 working drafts. This release adds support the syntax for all the changes for Query and Update. The new draft syntax is available via the ‘laqrs’ query language name, until the SPARQL 1.1 syntax is finalized. The ‘sparql’ query language provides SPARQL 1.0 support.

On Query 1.1, the addition is primarily syntax and API support for the new syntax. There is expression execution for the new functions IF(), URI(), STRLANG(), STRDT(), BNODE(), IN() and NOT IN() which are noew usable as part of the normal expression grammar. The existing aggregate function support was extended to add the new SAMPLE() and GROUP_CONCAT() but remains syntax-only. Finally the new GROUP BY with HAVING conditions were added to the syntax and had consequent API updates but no query engine execution of them.

For Update 1.1 the full set of update operations syntax were added and they create API structures. Note, however there seem to be some ambiguities in the draft syntax especially around multiple optional tokens in a row near WITH which are particularly hard to implement in flex and bison (aka “lex and yacc”).

The main non-SPARQL 1.1 related change is to allow building Rasqal with Raptor V2 APIs rather than V1. Raptor V2 is in beta so this is not a final API and is thus not the default build, it has to be enabled with --enable-raptor2 with configure. When raptor V2 is stable (2.0.0), Rasqal will require it.

The changes to Rasqal in this release, in summary, are:

• Updated to handle more of the new syntax defined by the SPARQL 1.1 Query and SPARQL 1.1 Update W3C working drafts of 1 June 2010
• Added execution support for new SPARQL 1.1 query built-in expressions IF(), URI(), STRLANG(), STRDT(), BNODE(), IN() and NOT IN().
• Added an ‘html’ query result table format from patch by Nicholas J Humfrey
• Added API support for group by HAVING expressions.
• Added XSD Date comparison support.
• Support building with Raptor V2 API if configured with --with-raptor2.
• Many other bug fixes and improvements were made.
• Fixed Issues: #0000352, #0000353, #0000354, #0000360, #0000374, #0000377 and #0000378

See the Rasqal 0.9.20 Release Notes for the full details of the changes.

Get it at http://download.librdf.org/source/rasqal-0.9.20.tar.gz.

PS The source code control has also moved to GIT and hosted at GitHub.

The index is based on queries tracked via Google Insights for Search that are tuned to different countries and you can also focus on particular regions or metropolitan areas and compare the index in several locations. Here’s an example comparing Florida (blue) and Maryland (red).

About six seminars are planned for Fall 2010 to be held (mostly) on Wednesdays from 4-6pm and UMBC faculty, staff and students are encouraged to participate. They will include a ~45 minute presentation followed by a lively discussion and opportunity to socialize and enjoy light refreshments.

The first speaker, Peter Kelly-Detwiler leads a group at Constellation Energy that is developing new methods for data analysis and presentation. He is an “entrepreneur” within Constellation with 20 years of experience in the energy field and he has a perspective on the Smart Grid like few others.

Many people have heard of the term “smart grid” and there are many varying interpretations of what it means. But everybody can agree on three things:

• It involves increased and timely access to information
• There’s money in it
• It will create new and unforeseen technologies and entrepreneurial opportunities

The discussion will center around why smart grid is needed, how an energy provider views the challenges and opportunities, the forces we see gathering on the horizon, and how Constellation Energy is responding. Issues related to power grid economics, volatility, risk management, and customer elasticities and perspectives will be addressed.

Peter Kelly-Detwiler is Senior Vice President of Energy Technology Services for Constellation NewEnergy, Inc., a subsidiary of Constellation Energy Group. He and his company-wide team oversee the integration of efficiency technologies and applications that help customers better manage their total energy bills and create optimal energy solutions. Peter has 20 years of experience in the energy industry. His accomplishments include managing the development of energy efficiency projects and reviewing economic impact of energy products.

Please RSVP to Bjorn Frogner (bjorn.frogner@umbc.edu), the CETI Entrepreneur in Residence, if you plan to attend.

Technology review has a short article, A New Kind of Microchip, on computer chip being developed by Lyric Semiconductor that process signals representing probabilities rather than digital bits.

“A computer chip that performs calculations using probabilities, instead of binary logic, could accelerate everything from online banking systems to the flash memory in smart phones and other gadgets. … And because that kind of math is at the core of many products, there are many potential applications. “To take one example, Amazon’s recommendations to you are based on probability,” says Vigoda. “Any time you buy [from] them, the fraud check on your credit card is also probability [based], and when they e-mail your confirmation, it passes through a spam filter that also uses probability.”

All those examples involve comparing different data to find the most likely fit. Implementing the math needed to do this is simpler with a chip that works with probabilities, says Vigoda, allowing smaller chips to do the same job at a faster rate. A processor that dramatically speeds up such probability-based calculations could find all kinds of uses.”

Lyric’s chip is called LEC and was developed with support from DARPA. It is 30 times smaller in size than current digital error correction technology according to Wired. Although small it yields “a Pentium’s worth of computation,” according to Lyric CEO Vigoda. His 2003 dissertation at MIT was on a related topic, Analog Logic: Continuous-Time Analog Circuits for Statistical Signal Processing.

You can also read about the LEC chip in a story in yesterday’s NYT, A Chip That Digests Data and Calculates the Odds.

Last week I had a chance to look at the latest information from the Department of Education’s National Center for Education Statistics, which is available from NSF’s WebCASPAR site. Data from the IPEDS Completions Survey shows that UMBC is fourth among U.S. research universities in the production of IT degrees and certificates.

In this analysis, I averaged the numbers from the two most recent years available — 2007 and 2008. Here are the top ten in terms of total production in the Carnegie classification categories RU/VH and RU/H.

In this group, UMBC also ranks #2, #21 and #31 for undergraduate, MS and PhD degree production, respectively. Here’s a graph of the top 50 — click through for a larger version.

Looking at all schools shows the University of Phoenix generates the most IT grads, with an average of 3318 students over 2007 and 2008! Here are the top 15 schools of any type.

The Workshop has listed a number of work items that might be of interest for such a Working Group, and has also conducted an informal poll as for the relative priority of those items (with links to the detailed description of the items themselves). As a next step, a public questionnaire has been created listing, essentially, those items (although some of them have been regrouped for a better readability). The goal of the questionnaire is to poll the Web community at large so that the upcoming charter would reflect the real needs for the years to come.

So… if you are interested in the evolution of RDF, here is the chance to make your opinion heard. All the results of the questionnaire will be public. The questionnaire will stay open until the 13th of September.

The reasoning behind doing this is multi-fold, but basically the code had a lot of cruft and bad design choices that couldn’t be removed without breaking the APIs in lots of ways, and at some point it’s easier to just do it all at once, and that’s where we are now.

Cruft meant removing stuff deprecated for a long time but also renaming all the functions to follow the same “objects in C” style used throughout Redland’s libraries which has standard naming forms:

• raptor_class_method()
• Constructors: raptor_new_class() (core constructor or 1 arg constructor) and raptor_new_class_from_extras()
• Copy constructor: raptor_class_copy()
• Destructors: raptor_free_class()

The major addition was a raptor_world object that is used as a single object to hold on to all shared resources and configuration. This was a design pattern I put in librdf and Rasqal but for some reason, never considered it for raptor. This turned out to be a mistake since I had to then pass around a lot of parameters and configuration to individual object instances, more than was really needed. Examples of this include the error handling which added two parameters to several constructors. The error handling, now expanded to a general log mechanism after librdf’s handles multiple structured log record types and the logging policy is once-per-world.

The addition of the world object meant that each constructor for an object in raptor now takes that object, so it can get access to the shared configuration and resources. That itself meant the change was extensive, broad in scope. The single place to manage resources means it’s easier to ensure proper cleanup and deal with library-wide issues.

One other pain point was Raptor’s simplistic (but functional!) URI class. It manipulated URIs as plain old C strings (char*). I knew from building librdf, that this could be more efficient by interning the strings so a URI for a particular string is held only once, and reference counted. I used the already built raptor AVL-Tree to implement it, and as a bonus, moved that AVL Tree to the public API, so it can be reused (Rasqal has a copy of the implementation). The resulting reference-counted URIs mean that after URI construction, comparison and copying are very cheap – and given that this is RDF, those are done a lot. The old URI code also had a swappable implementation which added a lot of complexity to the code and that has gone now, since the new implementation is more sophisticated. There is probably more work that can be done here to make this URI work better, such as caching the URI structure so that it’s quicker to generate relative URIs. Also one day I should really validate that all the URIs built are legal to the syntax.

Another long term problem was the triple itself, which I had called ‘statement’ way back when I was creating it. Unfortunately a raptor_statement had hard-coded the RDF specifics – the subject can only be URI or blank node, predicate can only be a URI etc. That meant the code was twisty. That has been replaced by an array of 3 or 4 raptor terms (URI or blank node or literal) so it can handle both triples, quads and any possible extension beyond RDF (2004), although today none of the current parsers or serializers expect non-RDF statements. That change also made a lot of the internal code simpler to understand and quicker. The RDF terms were also introduced in a reference count manner, along with adding reference counting to the statements, it meant that passing triples around which used to involve a lot of copying, is now a simple integer increment of the reference. More speed!

That sorted out the fundamentals of statements, terms and URIs and changed pretty much every piece of code that touched them in all the parsers and serializers and core code.

There were a few pieces of new work added – two new serializers and one new parser. Two of those were written by Nicholas J Humfrey who is now a core committer.

I’d also like to call out thanks to Lauri Aalto for keeping raptor, rasqal and librdf relatively buildable while I was refactoring and breaking things. He wrote the code to make Rasqal and librdf build and work with raptor V1 and V2 at the same time.

Other work included updating all the reference documentation, tutorials, examples and sundry documentation for the new APIs including admin code to automate some of the documentation so it always included accurate details about formats.

There is lots more that changed in detail, listed in the Raptor 1.9.0 Release Notes, help on upgrading and there’s even a perl script docs/upgrade-script.pl thrown in (generated by another perl script!) that may help with applying the changes. The reference manual contains a full reference on changes between raptor 1.4.21 and 1.9.0 in the form of old / new mappings with explanations.

I know that Raptor 2 is not going to place Raptor 1 for applications for some time, so this is a separately installed library with a new location for the header file and a new shared library base. However, once this hits 2.0.0 it’ll be a dependency of Rasqal and librdf.

Summary of release:

• Removed all deprecated functions and typedefs.
• Renamed all functions to the standard raptor_class_method() form.
• All constructors take a raptor_world argument.
• URIs are interned and there is no longer a swappable implementation.
• Statement is now an array of 3-4 RDF Terms to support triples and quads.
• World object owns logging, blank node ID generation and describing syntaxes.
• Features are now called options and have typed values.
• GRDDL parser now saves and restores shared libxslt state.
• Added serializers for HTML ‘html’ and N-Quads ‘nquads’.
• Added parser ‘json’ for JSON-Resource centric and JSON-Triples.
• Switched to GIT version control hosted by GitHub.
• Added memory-based AVL-Tree to the public API.
• Fixed reported issues:

  0000357, 0000361, 0000369, 0000370, 0000373 and 0000379

It turns out that after all that, the resulting libraries for raptor 2 are actually 4% smaller than raptor 1 when installed (Debian, i386):

 -rw-r--r-- 1 root root 379780 Mar 10 06:59 /usr/lib/libraptor.so
 -rw-r--r-- 1 root root 364448 Aug 16 17:30 /usr/lib/libraptor2.so

The gzipped tarball itself is as small as raptor 1.4.17 from 2008!

Get it at http://download.librdf.org/source/raptor2-1.9.0.tar.gz

PS The source code control has also moved to GIT and hosted at GitHub.

I must admit I was a little skeptical of the timing when I accepted the invitation to provide the keynote for a Linked Data session – on the last day of IFLA 2010 – at 8:30 in the morning – in August – on a Sunday.  Who was going to want to get up at that time, on the day they were probably going to leave beautiful Gothenburg, to hear me witter on about the Semantic Web and the obvious benefits of Linked Data for libraries? A few minutes before the start, I was beginning to think my skepticism was well founded, viewing the acres of empty seats laid out in their menacing ranks in front of me. But then almost as if from nowhere, the room rapidly filled and by the time I took the stage we had something approaching a full house.  As you can see from my iPhone snap below, we ended up with a significant group (I lost count at about 250) of interested librarians.

So was it worth them turning up at such an unsociable time?  I obviously can’t speak for my session, but I believe it was well worth turning up.  We had a series talks which varied from the in-depth technical/ontological spectrum to the rousing plea to open up your data now – and don’t hamper it with too much licensing.

First on after my session was Gordon Dunsire from the University of Strathclyde who gave us some in depth reasoning as to why we needed complex detailed ontologies based upon standards like RDA, FRBR, FRAD, and RDA to describe library resources in RDF for the Semantic Web.   To represent the full detail that catalogers have, and want to, provide for resource description I agree with him.  I also believe that we need to temper that detailed view by including more generic ontologies in addition. People from outside of the library world, dipping into library data [with more ways to describe a title than there are flavors of ice cream], will back off and not link to it unless the can find a nice friendly dc:title or foaf:name that they understand.

Some of the other speakers that I caught included Patrick Danowski’s entertaining presentation entitled “Step 1: Blow up the silo!”. He took us through the possible licenses to use for sharing data, only to conclude that the best approach was totally open public domain.  He then went on to recommend CC0 and/or PDDL as the best way to indicate that your data is open for anyone to do anything with.

Jan Hanneman from the German National Library delivered an interesting description [pdf]of the way they have been publishing their authority data as Linked Data, and the challenges they met on the way.  These included legal and licensing issues, around what and under what terms they could publish.  Scalability of their service, being another key issue once they move beyond authority data.

All in all it was an excellent Sunday morning in Gothenburg.  I presume the organizers of IFLA 2011 will take note of the interest and build a larger, more convenient, slot in the programme for Linked Data.

Note: My presentation slides can be viewed on Slideshare and downloaded in pdf form

A recent paper by two researchers from Microsoft concludes that an organization’s usability requirements is the main factor that determines the complexity of its password policy.

Dinei Florencio and Cormac Herley, Where Do Security Policies Come From?, Symposium on Usable Privacy and Security (SOUPS), 14–16 July 2010, Redmond.

We examine the password policies of 75 different websites. Our goal is understand the enormous diversity of requirements: some will accept simple six-character passwords, while others impose rules of great complexity on their users. We compare different features of the sites to find which characteristics are correlated with stronger policies. Our results are surprising: greater security demands do not appear to be a factor. The size of the site, the number of users, the value of the assets protected and the frequency of attacks show no correlation with strength. In fact we find the reverse: some of the largest, most attacked sites with greatest assets allow relatively weak passwords. Instead, we find that those sites that accept advertising, purchase sponsored links and where the user has a choice show strong inverse correlation with strength.

We conclude that the sites with the most restrictive password policies do not have greater security concerns, they are simply better insulated from the consequences of poor usability. Online retailers and sites that sell advertising must compete vigorously for users and traffic. In contrast to government and university sites, poor usability is a luxury they cannot afford. This in turn suggests that much of the extra strength demanded by the more restrictive policies is superfluous: it causes considerable inconvenience for negligible security improvement.

h/t Bruce Schneier

Bruce Schneier offered a taxonomy of social networking data in a short article in the July/August issue of the IEEE Security & Privacy. A version of the article, A Taxonomy of Social Networking Data, is available on his site.

“Below is my taxonomy of social networking data, which I first presented at the Internet Governance Forum meeting last November, and again — revised — at an OECD workshop on the role of Internet intermediaries in June.

• Service data is the data you give to a social networking site in order to use it. Such data might include your legal name, your age, and your credit-card number.
• Disclosed data is what you post on your own pages: blog entries, photographs, messages, comments, and so on.
• Entrusted data is what you post on other people’s pages. It’s basically the same stuff as disclosed data, but the difference is that you don’t have control over the data once you post it — another user does.
• Incidental data is what other people post about you: a paragraph about you that someone else writes, a picture of you that someone else takes and posts. Again, it’s basically the same stuff as disclosed data, but the difference is that you don’t have control over it, and you didn’t create it in the first place.
• Behavioral data is data the site collects about your habits by recording what you do and who you do it with. It might include games you play, topics you write about, news articles you access (and what that says about your political leanings), and so on.
• Derived data is data about you that is derived from all the other data. For example, if 80 percent of your friends self-identify as gay, you’re likely gay yourself.”

I think most of us understand the first two categories and can easily choose or specify a privacy policy to control access to information in them. The rest however, are more difficult to think about and can lead to a lot of confusion when people are setting up their privacy preferences.

As an example, I saw some nice work at the 2010 IEEE International Symposium on Policies for Distributed Systems and Networks on “Collaborative Privacy Policy Authoring in a Social Networking Context” by Ryan Wishart et al. from Imperial college that addressed the problem of incidental data in Facebook. For example, if I post a picture and tag others in it, each of the tagged people can contribute additional policy constraints that can narrow access to it.

Lorrie Cranor gave an invited talk at the workshop on Building a Better Privacy Policy and made the point that even P3P privacy policies are difficult for people to comprehend.

Having a simple ontology for social media data could help us move forward toward better privacy controls for online social media systems. I like Schneier’s broad categories and wonder what a more complete treatment defined using Semantic Web languages might be like.

Researchers naturally are interested in knowing how to attract more citations to their papers. Publishing the results of good work helps of course, but everyone knows there are many other factors. Nature news reports on research by Gregory Webster that analyzed the 53,894 articles and review articles published in Science between 1901 and 2000.

The advice the study supports is “cite and you shall be cited”.

A long reference list at the end of a research paper may be the key to ensuring that it is well cited, according to an analysis of 100 years’ worth of papers published in the journal Science.
     The research suggests that scientists who reference the work of their peers are more likely to find their own work referenced in turn, and the effect is on the rise, with a single extra reference in an article now producing, on average, a whole additional citation for the referencing paper.
     ’There is a ridiculously strong relationship between the number of citations a paper receives and its number of references,” Gregory Webster, the psychologist at the University of Florida in Gainesville who conducted the research, told Nature. “If you want to get more cited, the answer could be to cite more people.’
…
A plot of the number of references listed in each article against the number of citations it eventually received reveal that almost half of the variation in citation rates among the Science papers can be attributed to the number of references that they include. And — contrary to what people might predict — the relationship is not driven by review articles, which could be expected, on average, to be heavier on references and to garner more citations than standard papers.

This is an amazing result and a testament to more than 30 years of work on the problem. The Cube was invented in 1974 and almost immediately the subject for programs to solve it. In 1981, Morwen Thistlethwaite proved that any configuration could be solved in no more than 52 moves. Periodically, tighter upper bounds for the maximum solution length were found. This result ends the quest — there are some configurations (about 300M) that require 20 moves to solve and there are none that require more than 20 moves.

In their own words, here’s how the group solved all 43,252,003,274,489,856,000 Cube positions:

• We partitioned the positions into 2,217,093,120 sets of 19,508,428,800 positions each.
• We reduced the count of sets we needed to solve to 55,882,296 using symmetry and set covering.
• We did not find optimal solutions to each position, but instead only solutions of length 20 or less.
• We wrote a program that solved a single set in about 20 seconds.
• We used about 35 CPU years to find solutions to all of the positions in each of the 55,882,296 sets.

This reminds me of the first program I wrote for my own enjoyment, which used brute force to find all solutions to Piet Hein’s Soma Cube. In 1969 I had a summer job as the night operator for an IBM 360 and I would turn off the clock to run my program so that the management wouldn’t know how much computer time I was consuming.

See this BBC story more more information on this amazing result.

Libraries and the Big Society

Towards the end of July in Liverpool, the UK’s new Prime Minister, David Cameron, finally set out his plans for his vision of the Big Society. The initiative is based on the devolution of power from Westminster to local communities, and the empowerment of citizens to run local public services (including, of course, public libraries).

Cutting costs or empowering communities

In announcing the Big Society, David Cameron focused on the pivotal role of people and communities saying it is a “big advance for people power” and that his “great passion is building the Big Society”. Much of the discourse around the Big Society is underpinned by the need for the government to reduce the budget deficit. The Opposition has certainly made this point, and certain individuals, notably Tessa Jowell, are also enraged about the lack of originality.

What does this mean to our public library service?

Public libraries are going through tough times, and this announcement raises broad questions about the public library mission. The DCMS review earlier in the year attempted to answer these questions and set out a framework, but the new government has tried to distance itself from this review, only acknowledging the relevance of a small number of points (library membership from birth, free internet access, co-location). Ed Vaizey, Culture Minister, has scrapped his election promise of setting up a Library Development Agency, and has instead set up a “support programme”. This will go ahead in conjunction with the Museums, Libraries and Archives Council (MLA) and the Local Government Association (LGA), despite the DCMS announcement that the MLA is to be abolished within two years. So, this announcement only raises more questions about how important the government really feels libraries are.

Library advocate Tim Coates has set out some basic points as to what libraries should do to address these problems, including increasing opening hours, improving stock and making sure the library meets local needs. Meeting local needs is something that perhaps the Big Society will address.

So, will the Big Society benefit libraries?

It has been widely reported that the Big Society is essentially about having volunteers running public services. The main benefit of this will be reduced costs. According to the government, freeing up budget and cutting costs will allow libraries “to focus efforts on frontline, essential services and ensure greater value for money”.

The use of volunteers in libraries throws the professional status of librarianship in doubt. Many professional librarians argue that considerable formal education is needed and that this new initiative implies that someone with no experience can come in and effectively do the same job. Since these volunteers will be unpaid, how can we ensure reliability and quality? Who actually has time to volunteer for a day a week at their local library? We may end up with libraries opening less and not meeting local needs, all of which contradict the Big Society.

Volunteers in libraries are not uncommon though. The Summer Reading Challenge utilises many volunteers, this year more than ever before, and libraries find them useful. Young adults who have volunteered, after participating in the Challenge when they were younger, can encourage reader development in the current wave of children taking part.

Even before the Big Society was announced, many libraries across the country were closing, or were under review. The Big Society proposals set out plans for local citizens who oppose the closures to get involved, work in libraries and fill the gaps in local authority budgets. Perhaps a mixed economy of reliable, professional librarians and enthusiastic volunteers will reinvigorate public libraries. Let’s hope so.

Many travelled from the far corners of the UK and Europe, but from the several emails I received it was clear that many others could not make it.  To that end we took along the technology to capture as much of the event as possible.

The videos have now been edited and published on our sister blog, Nodalities, where you will also find links to the associated presentation slides.  I can highly recommend these as an introduction to the topic and an overview of the thinking and activities in this area from such as the British Library and Europeana.

Question: Why did you launch yet another journal on Semantic Web?

Hitzler: Because the community is growing and the need for publication outlets grows with it. I heard the objection that there weren’t enough quality papers for all the journals, but I don’t think so. It’s just that most of the quality papers still end up in journals which are not dedicated to the Semantic Web as such.

Personally, my desire to start a new journal began when I wanted to do a special issue on Semantic Web reasoning in some other, established, journal, and the Editors-in-Chief basically replied with a lapidary “Is there anything to report?” I didn’t push the case back then (though I probably should have). But this and similar experiences made me think about scientific publishing from a different angle, a normative one: What should scientific publishing in our field look like? The journal gives me a possibility to realize some of my answers – or at least to go a few steps into the right direction. So when the opportunity arose to set up this journal with a well-known publishing house (IOS Press) and with a co-Editor-in-Chief (Krzysztof Janowicz, a strong proponent of open and transparent reviewing) who I knew would also put a maximum of energy into the venture, it was simply too good an opportunity to let it pass. However I also realize that the reality of scientific publishing can change only slowly, and that it needs time and gradual improvements. We can’t do it all at once.

Question: Your journal uses an open review process. What is that and why?

Hitzler: Open reviewing, in the sense we use it for the Semantic Web journal, is all about transparency. Submitted papers are made publicly available. Solicited reviews are made publicly available. Anybody else can additionally contribute a public review. Reviewers are publicly known by name. Discussions between reviewers and authors can (and should) happen in public. Reviewers and editors are acknowledged by name in the published versions of the papers.

The obvious reason for setting up an open review process is to improve the quality of the decision-making process. We have to realize that some persisting habits about reviewing have their origin in times when scientific publishing was made for a small expert audience, and had to be conducted by sending manuscripts and letters by conventional mail. Today, however, reviewing and publishing is inflationary, which substantially reduces the quality of the typical paper – and of the typical review. While we cannot simply reverse this trend, we can take advantage of the World Wide Web to counteract these developments and improve quality by bringing the review process out into the public space. Reviewers will put more effort into providing constructive reviews if they publicly sign their reviews. Open and public discussions on controversial submissions minimize errors in the decision making.

Personally, I also hope that the ensuing discussions will help to bring back a scientific tradition which has long been on the decline in our field: controversial but constructive discussion. Regretfully, these days we somehow tend to mainly present incremental results, bash opposing opinions, and sugarcoat our own …

Question: Past attempts to set up open reviewing for journals have failed …

Hitzler: Yes, I remember seeing some of these early attempts many years ago when I was a PhD student. Even back then I was doubtful if the sometimes rather radical setups had a chance. In the meantime, there is growing experience in other fields that open reviews can work out if set up carefully. In our case, we mix old-style with open, by still soliciting reviews, and by giving solicited reviewers the option to stay anonymous, if they see a need for this protection. We Editors-in-Chief also “steer” the journal in the sense that we have rather clear strategic targets, e.g. in terms of scope and quality, which we’re trying to meet. In short: rather than experimenting with radical changes, we mildly introduce a new but essential component – open reviewing – in a traditional scientific publishing process. That way, it will work.

Question: But isn’t anonymous reviewing necessary to protect the reviewers and in order to get objectively critical reviews?

Hitzler: Sometimes. That’s why it’s good that solicited reviewers can opt to stay anonymous. Open reviewing – like any form of assessment in science – isn’t perfect, and has its drawbacks. However, the current reality in Computer Science is that reviewing processes are often extremely poor and decision processes are not very transparent. For conferences, reviewer discussions and rebuttal phases were introduced some time ago to improve the decision making. Open reviews simply go a step further.

Question: Aren’t potential authors afraid of getting a public bashing in the review process?

Hitzler: Reviewers typically won’t bash if they sign with their name. And in fact, we monitor the reviews in order to make sure that they adhere to a certain minimal scientific standard. At the same time, it’s probably just as well if our public process makes people more reluctant to submit papers which are not yet mature enough for publication. We wouldn’t want to publish them anyway. And in order to protect authors of rejected submissions, we actually remove the corresponding papers and reviews from the website after some time.

While I understand that some people may be more reluctant to put their work out in the open before it’s been accepted through a review process, we have to be aware that many quality journal publications, like the ones we’re striving for, are extended versions of high-quality conference publications: so they have indeed already been through a review process. Furthermore, submitting to our journal gives added visibility for the work, since it’s up for public review on our website.

Question: Your journal also publishes papers which are not standard research papers. Aren’t you compromising scientific rigor by doing this?

Hitzler: Times are changing. The prime purpose of a scientific journal is to disseminate results to other researchers, and to do so through a quality filter. Traditionally, this dissemination was restricted to focused research contributions, targeted at other researchers working in the same narrow area as the author(s). Semantic Web as a field, however, is extremely diverse and comprises researchers and practitioners from many other communities. Consequently, high-quality tools, systems, ontologies, introductory surveys and application reports are very much needed for the dissemination of advances in our field to all interested parties. As for research papers, the role of the journal for these other types of papers is primarily quality assurance. And consequently, we have clearly formulated the evaluation criteria for different types of papers. A report on a high impact tool, for example, is thus not a direct research contribution in the traditional sense. But if the tool enables further developments in the field, then it is worth reporting, and it indirectly makes a contribution to scientific progress.

Question: Why are you still publishing through a commercial publishing house?

Hitzler: Because it helps. A lot. It’s easy to underestimate the amount of work which needs to be put into running a journal, and going with a commercial publisher rids the Editors-in-Chief and the Editorial Board from a lot of tasks which are not directly related with quality assurance. Open review does not mean that this kind of professional support is no longer needed. And we are glad that we have found a publishing house which is very accommodating to our ideas.

Question: What are plans for the immediate future?

Hitzler: We currently have more than 30 papers up for review, most of them responses to two recent calls, one on tools and systems papers, and one on applications of OWL – and some of the submissions seem rather prominent. We also have several special issues lined up, most of them have not been announced yet. The first issue will appear towards the end of the year and contain vision statements by the EB members – we do not normally publish vision statements, but this seemed an appropriate way to introduce the journal. Considering that the journal has been launched only 7 months ago, this means that we are already very well under way in pursuing our goal of establishing a high-quality scientific outlet in the field.

[author: Pascal Hitzler]

1. Mobile
2. Search and the Semantic Web
3. Search and the Semantic Web
4. Virtualization and cloud computing
5. Virtualization and cloud computing

These are reasonable choices, thought I’ve have not done the double counting and added “machine learning applied to the massive amounts of Web data now available” and “social computing”.

But it’s gratifying to see the Semantic Web in the list. Here’s some of what he he has to say about search and the Semantic Web.

The relationship between search technology and the Semantic Web is a perfect illustration of how a small sustaining technology, such as a basic search feature on an operating system, will eventually be eaten up by a larger disruptive technology, such as the Semantic Web. The Semantic Web has the potential of acting like a red giant star by expanding at exponential rates, swallowing whole planets of existing technology in the process.

The technology started as a simple group of secure, trusted, linked data stores. Now Semantic Web technologies enable people to create data stores on the Web and then build vocabularies or write rules for handling the data. Because all the data by definition is trusted, security is often less of a problem.

The task of turning the World Wide Web into a giant dynamic database is causing a shift among traditional search engines because products such as Apture, by Apture Inc. of San Francisco, Calif., let content publishers include pop-up definitions, images or data whenever a user scrolls over a word on a Web site. The ability to categorize content in this manner could have significant implications not only for Web searches but also for corporate intranets and your desktop PC.

These types of products will continue to expand, initially in the publishing industry and then to most industries on the Web in the next two to three years.

For example, human resources sites could use them to pop up a picture and a résumé blip when a recruiter drags a mouse over an applicant’s name. Medical and financial sites such as the National Institutes of Health could use it to break down jargon and help with site exploration.
…
Government sites around the world, such as Zaragoza, Spain, and medical facilities, such as the Cleveland Medical Clinic, are using the vocabulary features of the Semantic Web to create search engines that reach across complex jargon and tech silos to offer a high degree of automation, full integration with external systems and various terminologies, in addition to the ability to accurately answer users’ queries.
…”

(h/t @FrankVanHarmele)

Warning: Visiting this site may harm your computer!

The website at planetrdf.com contains elements from the site bin.clearspring.com, which appears to host malware – software that can hurt your computer or otherwise operate without your consent. Just visiting a site that contains malware can infect your computer.

For detailed information about the problems with these elements, visit the Google Safe Browsing diagnostic page for bin.clearspring.com.

Learn more about how to protect yourself from harmful software online.

[ ] I understand that visiting this site may harm my computer. PROCEED

Clearspring claims it’s a technical problem, although they admit they were using a service that was compromised with files redirecting users to a certain malware domain. I’m a bit fuzzy on what clearspring does and where they are being used on the Planet RDF site. I don’t see it in the page source, for example.

update: Maybe the problem stems from flash cookies in blog content being syndicated by Planet RDF that have flash objects mediated by clearspring.