One immediate distinguishing feature of hakia Search Box is its flexibility to search in multiple domains as shown below.

The second distinguishing feature is its sentence highlighting and semantic precision (especially with complex, long-tail, and unusual queries) as shown below. Note the uninterrupted text snippets (no ellipses) for Pubmed and health searches.

There are several ASP and PHP examples on the page with design options as outlined below:

- Web Plus Search (multiple domains as shown above)
- Site search (pick a site to search only its content)
- Pubmed search (search results from 10 million pubmed articles)
- Health search (search results from credible Web sources on health)

It is free up to 30,000 searches per day (which is the highest number offered to date).

Why do you need a good search box on your site? Well, you don’t want those curious visitors to leave your site and go to a search engine. With a good search box, you will keep them on your Web property.

If you already have a search box on your Web site and you are not sure what to do, you can add hakia’s search box as a semantic search option.

Give it a try and let us know.

At 5pm local time on Friday, January 30, each site will be told the parameters of the game they all must produce. Participants pitch ideas, form teams, and get to work producing the best game they can in 48 hours. The UMBC site will have a good mix of computers and development platforms including Windows (XP), Mac (Leopard), XBox 360 (with Creators Club), PlayStation 3 (running Linux) with a diverse software environment that inlcludes Visual Studio, Maya, XNA Game Studio, NVIDIA PhysX and Adobe Creative Suite. For more information see the UMBC Global Game Jam page.

The Global Game Jam participants do not have to be UMBC students, and the Jam is open to participants of all levels of skill and experience. There is no registration fee for the Baltimore Jam site at UMBC, but space is limited so advance registration is required.

This event is sponsored by the UMBC Games, Animation and Interactive Media program, an innovative academic program with tracks available for students pursuing a degree in computer science or a degree in visual arts.

The resolved issue that was most likely to have frustrated users was broken import behavior when using the the Pellet command line with the Jena loader. In addition, our effort to support the built-ins listed in the SWRL submission is closer to complete now that the team has added implementation of the built-ins for URIs.

Special thanks to the users who reported issues since most of the changes in this release were made in response to user identified problems. Keep up the good work by sending your bug reports to the Pellet users mailing list.

Olga Vladi Ratsimor, Opportunistic Bartering of Digital Goods and Services in Pervasive Environments, Ph.D. Dissertation in Computer Science, University of Maryland, Baltimore County, August 2007.

The vision of mobile personal devices querying peers in their environment for information such as local restaurant recommendations or directions to the closest gas station, or traffic and weather updates has long been a goal of the pervasive research community. However, considering the diversity and the personal nature of devices participating in pervasive environments it is not feasible to assume that these interactions and collaborations will take place with out economically-driven motivating incentives.

This dissertation presents a novel bartering communication model that provides an underlying framework for incentives for collaborations in mobile pervasive environments by supporting opportunistic serendipitous peer-to-peer bartering for digital goods such as ring tones, MP3’s and podcasts.

To demonstrate viability and advantages of this innovative bartering approach, we compare and contrast the performances of two conventional, frequently employed, peer-to-peer interaction approaches namely Altruists and FreeRiders against two collaborative strategies that employ the Double Coincidence of Wants paradigm from the domain of barter exchanges. In particular, we present our communication framework that represents these collaborative strategies through a set of interaction policies that reflect these strategies. Furthermore, we present a set of results from our in-depth simulation studies that compare these strategies.

We examine the operation of the nodes employing our framework and executing these four distinct strategies and specifically, we compare the performances of the nodes executing these strategies in homogeneous and heterogeneous networks of mobile devices. We also examine the effects of adding InfoStations to these networks. For each of the strategies, we observe levels of gains and losses that nodes experience as result of collaborative digital good exchanges. We also evaluate communication overhead that nodes incur while looking for possible collaborative exchange. Furthermore, this dissertation offers an in-depth study of the swarm-like inter-strategy dynamics in heterogeneous networks populated with diverse nodes displaying varying levels of collaborative interaction attitudes. Further, the bartering framework is extended by incorporating value-sensitive bartering models that incorporate digital goods and content valuations into the bartering exchange process. In addition, the bartering model is extended by integration of socially influenced collaborative interaction that exploit role based social relationships between mobile peers that populate dynamic mobile environments.

Taken as a whole, the novel research work presented in this dissertation offers the first comprehensive effort that employs and models opportunistic bartering-based collaborative methodology in the context of serendipitous encounters in dynamic mobile peer-to-peer pervasive environments where mobile entities negotiate and exchange digital goods and content.

“Blogging and message boards have played a substantial role in allowing communication among those who would do the United States harm,” DHS said in a recent notice.

Julian Sanchez notes on Ars Technica that the story is not new.

“The story is actually pegged to a Sources Sought Notice posted by the Department of Homeland Security back in October. Our colleagues at Wired reported on it at the time.”

A Mobile Ad-hoc NETwork (MANET), as its name suggests, has no fixed infrastructure, and is generally composed of a dynamic set of cooperative peers, which are willing to share their wireless transmission power with other peers so that indirect communication can be possible between nodes that are not in the radio range of each other . The nature of MANETs, such as node mobility, unreliable transmission medium and restricted battery power, makes them extremely vulnerable to a variety of node misbehaviors. Wireless links, for instance, are generally prone to both passive eavesdropping and active intrusion. Another security concern in ad hoc networks is caused by the cooperative nature of the nodes. Attacks from external adversaries may disturb communications, but the external intruder generally cannot directly participate in the cooperative activities among the nodes, such as routing, because they do not possess the proper secure credentials, such as shared keys. However, compromised nodes, which are taken over by an adversary, are capable of presenting the proper secure credentials, and consequently can interfere with almost all of the network operations, such as route discovery, key management and distribution, and packet forwarding. Hence, it is essential to cope with node misbehaviors so as to secure mobile ad hoc networks.

In this dissertation, we address the question of how to ensure that a MANET will properly operate despite the presence of various node misbehaviors. We propose to build a framework that can cope with various node misbehaviors in a wise and adaptive manner. The main purpose of our proposed framework is to provide a platform so that the components that identify and respond to misbehaviors can better cooperate with each other and quickly adapt to the changes of network context. Therefore, policies are planned to be utilized in our framework in order to make those components correctly function in different network contexts. Besides the policy component, there are three other components, which fulfill the tasks of misbehavior detection, trust and reputation management, and route management, respectively. To validate and evaluate our proposed framework, we plan to implement our framework based on simulator.

In particular, the contributions of this dissertation are (i) Develop a framework to combine the functionalities of surveillance and detection of misbehavior, trust and reputation management, route management, and policy management so as to provide a high-level solution to cope with various misbehaviors in MANETs in an intelligent and adaptive manner (ii) Propose and implement a misbehavior detector based on the gossip-based outlier detection method, which relies on neither any pre-defined threshold nor any training data (iii) Take into account both first-hand information (direct observation) and second-hand information (indirect observation) during both misbehavior detection and trust management processes, in which first-hand information and second-hand information are merged by some intelligent methods (iv) Specify and enforce policies in the proposed framework, which makes the framework promptly adapt to the rapidly changing network context.

octo.py is a very simple MapReduce like system inspired by Ruby’s Starfish.

“Octo.py doesn’t aim to meet all your distributed computing needs, but its simple approach is amendable to a large proportion of parallelizable tasks. If your code has a for-loop, there’s a good chance that you can make it distributed with just a few small changes. If you’re already using Python’s map() and reduce() functions, the changes needed are trivial!”

triangular.py is the simple example given in the documentation that is used with octo.py to compute the first 100 triangular numbers.

# triangular.py compute first 100 triangular numbers. Do
# 'octo.py server triangular.py' on server with address IP
# and 'octo.py client IP' on each client. Server uses source
# & final, sends tasks to clients, integrates results. Clients
# get tasks from server, use mapfn & reducefn, return results.

source = dict(zip(range(100), range(100)))

def final(key, value):
    print key, value

def mapfn(key, value):
    for i in range(value + 1):
        yield key, i

def reducefn(key, value):
    return sum(value)

Put octo.py on all of the machines you want to use. On the machine you will use as a server (with ip address <ip>), also install triangular.py, and then execute:

     python octo.py server triangular.py &

On each of your clients, run

     python octo.py client <ip> &

You can try this out using the same machine to run the server process and one or more client processes, of course.

When the clients register with the server, they will get a copy of triangular.py and wait for tasks from the server. The server access the data from source and distributed tasks to the clients. These in turn use mapfn and reducefn to complete the tasks, returning the results. The server integrates these and, when all have completed, invokes final, which in this case just prints the answers, and halts. The clients continue to run, waiting for more tasks to do.

Octo.py is not a replacement for more sophisticated frameworks like Hadoop or Disco but if you are working in Python, its KISS approach is a good way to get started with the MapReduce paradigm and might be all you need for a small projects.

(Note: The package has not been updated since April 2008, so it’s status is not clear. But further development would run the risk of making it more complex and would be self-defeating.)

We're wrapping up the year with 9,000 registered users of the Calais service who are submitting one to two million transactions each day. You've built dozens, if not hundreds, of innovative applications and you've provided regular feedback to help us make the service more useful. Thank you. We truly could not have done it without you.

Now we'd like to ask for a little more help. We're pulling together our roadmap for 2009 and want to hear directly from you about what you'd like to see. As a user-driven project, Calais needs your feedback to make sure we're on the right path.

Please drop us a note at 2009@opencalais.com and let us know two things:

First: What would you like to see delivered in 2009? New extractions? Languages? New tools or API capabilities? Integration with your favorite content management system? Let us know - nothing is off the table.

Second: How are you using Calais? Even if it's experimental and not for public consumption, we'd love to hear what you're up to - or even what you're thinking about.

Again, thanks for your support. There are some exciting announcements coming with Release 4 of Calais in mid-January and we're looking forward to a great 2009. Your input will help make it so.

Our best wishes for happy and healthy new year,

The Calais Team

• Best paper: Travis Kriplean, Ivan Beschastnikh, David McDonald, Wikipedian Self-Governance in Action: Motivating the Policy Lens
• Invited talk: Brad Fitzpatrick, Decentralization and Interop: past, present, future
• Invited talk: David Sifry, Scaling Innovation in a Network Effect World
• Invited talk: Marc Smith, Social dimension of social media
• Tutorial: Mary McGlohon, Graph Mining Techniques for Social Media Analysis
• Tutorial: Jan Wiebe, Subjectivity and Sentiment Analysis
• James Caverlee, A Large-Scale Study of MySpace: Observations and Implications for Online Social Networks
• Aaditeshwar Seth, Jie Zhang, A Social Network Based Approach to Personalized Recommendation of Participatory Media Content
• Michael Gamon, Danyel Fisher, Sumit Basu, Dmitriy Belenko, Matthew Hurst, Arnd Christian König, BLEWS: Using Blogs to Provide Context for News Articles
• Jiang Yang, Lada Adamic, Mark Ackerman : Competing to Share Expertise: the Taskcn Knowledge Sharing Community
• Jaime Arguello, Jonathan Elsas, Jamie Callan, Jaime Carbonel : Document Representation and Query Expansion Models for Blog Recommendation
• Noor Ali-Hasan, Exploring Social Media Scenarios for the Television
• Hikaru Yamamoto, Naohiro Matsumura, Daisuke Tomozawa, Finding Influencers and Consumer Insights in the Blogosphere
• Mikhail Bautin, Lohit Vijayarenu, Steven Skiena, International Sentiment Analysis for News and Blogs
• Ramesh Nallapati, William Cohen, Link-PLSA-LDA: A new unsupervised model for topics and influence of blogs
• Ian Soboroff, On TREC Blog Track
• Jason Kessler, Polling the Blogosphere: a Rule-Based Approach to Belief Classification
• Yi-Chia Wang, William Cohen, Mahesh Joshi, Carolyn Rosé : Recovering Implicit Thread Structure in Newsgroup Style Conversations
• Danyel Fisher, Marc Smith, Tammara Combs Turner, Space Planning for Online Community
• Kristin Stecher, Scott Counts, Spontaneous Inference of Personality Traits from Online Profiles
• Gina Walejko, Thomas Ksiazek, The Politics of Sourcing: A Study of Journalistic Practices in the Blogosphere
• Nairan Ramírez-Esparza, Cindy Chung, Ewa Kacewicz, James W. Pennebaker, The Psychology of Word Use in Depression Forums in English and in Spanish
• Kristin Stecher, Scott Counts, Thin Slices of Online Profile Attributes
• David Evans, Sam Gosling, Anthony Carroll, What Elements of an Online Social Networking Profile Predict Target-Rater Agreement in Personality?
• Zareen Syed, Wikipedia as an Ontology for Describing Documents

UMBC tied for first place with University of Texas at Dallas (B Team) in the sixth and final round of the three-day 2008 Pan-Am Championship which was held in Dallas. This year 29 four-person college teams competed in the annual event which is known as the “World Series of College Chess“. UMBC has now won the Pan-Am tournament a record eight times. The final standings are available at swchess.

The two first place winners will meet again with the third and fourth place teams, the University of Texas Brownville and Stanford, in the special Final Four of Chess tournament, which is held in spring 2009.

The UMBC chess team: front row, L to R: WGM Sabina Foisor, GM Timur Gareev, GM Sergey Erenburg, and GM Leonid Kritz, board one, Back row: UMBC coaches GM Sam Palantnik and NM Igor Epshteyn. Photo Alexey Root.

Wikirage lists the 100 Wikipedia pages that are being heavily edited over any of six time periods from the last hour to the last month. You can see the top 100 by your choice of six metrics: number of quality edits, unique editors, total edits, vandalism, reversions, or undos. Clicking on a result shows a monthly summary for the article, for example, December 2008 Gaza Strip airstrikes, which is at the top of today’s list for number of edits as I write. I understand the Gaza article, but what’s up with the Tasmanian tiger?

The interface has some other nice features, such as marking pages in red that have high revision, vandalism or undo rates and showing associated Wikipedia flags that indicating articles that need attention or don’t live up to standards. Wikirage is also available for the English, Japanese, Spanish, German and French language Wikipedias.

Wikirage was developed by Craig Wood and is a nicely done system.

(via the Porn Sex Viagra Casino Spam site)

Akshay Java, Mining Social Media Communities and Content, Ph.D. Dissertation, Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, December 1, 2008. Available at http://ebiquity.umbc.edu/paper/html/id/429/Mining-Social-Media-Communities-and-Content.

Social Media is changing the way people find information, share knowledge and communicate with each other. The important factor contributing to the growth of these technologies is the ability to easily produce “user-generated content”. Blogs, Twitter, Wikipedia, Flickr and YouTube are just a few examples of Web 2.0 tools that are drastically changing the Internet landscape today. These platforms allow users to produce and annotate content and more importantly, empower them to share information with their social network. Friends can in turn, comment and interact with the producer of the original content and also with each other. Such social interactions foster communities in online, social media systems. User-generated content and the social graph are thus the two essential elements of any social media system.

Given the vast amount of user-generated content being produced each day and the easy access to the social graph, how can we analyze the structure and content of social media data to understand the nature of online communication and collaboration in social applications? This thesis presents a systematic study of the social media landscape through the combined analysis of its special properties, structure and content.

First, we have developed a framework for analyzing social media content effectively. The BlogVox opinion retrieval system is a large scale blog indexing and content analysis engine. For a given query term, the system retrieves and ranks blog posts expressing sentiments (either positive or negative) towards the query terms. Further, we have developed a framework to index and semantically analyze syndicated1 feeds from news websites. We use a sophisticated natural language processing system, OntoSem, to semantically analyze news stories and build a rich fact repository of knowledge extracted from real-time feeds. It enables other applications to benefit from such deep semantic analysis by exporting the text meaning representations in Semantic Web language, OWL.

Secondly, we describe novel algorithms that utilize the special structure and properties of social graphs to detect communities in social media. Communities are an essential element of social media systems and detecting their structure and membership is critical in several real-world applications. Many algorithms for community detection are computationally expensive and generally, do not scale well for large networks. In this work we present an approach that benefits from the scale-free distribution of node degrees to extract communities efficiently. Social media sites frequently allow users to provide additional meta-data about the shared resources, usually in the form of tags or folksonomies. We have developed a new community detection algorithm that can combine information from tags and the structural information obtained from the graphs to effectively detect communities. We demonstrate how structure and content analysis in social media can benefit from the availability of rich meta-data and special properties.

Finally, we study social media systems from the user perspective. In the first study we present an analysis of how a large population of users subscribes and organizes the blog feeds that they read. This study has revealed interesting properties and characteristics of the way we consume information. We are the first to present an approach to what is now known as the “feed distillation” task, which involves finding relevant feeds for a given query term. Based on our understanding of feed subscription patterns we have built a prototype system that provides recommendations for new feeds to subscribe and measures the readership based influence of blogs in different topics.

We are also the first to measure the usage and nature of communities in a relatively new phenomena called Microblogging. Microblogging is a new form of communication in which users can describe their current status in short posts distributed by instant messages, mobile phones, email or the Web. In this study, we present our observations of the microblogging phenomena and user intentions by studying the content, topological and geographical properties of such communities. We find that microblogging provides users with a more immediate form of communication to talk about their daily activities and to seek or share information.

The course of this research has highlighted several challenges that processing social media data presents. This class of problems requires us to re-think our approach to text mining, community and graph analysis. Comprehensive understanding of social media systems allows us to validate theories from social sciences and psychology, but on a scale much larger than ever imagined. Ultimately this leads to a better understanding of how we communicate and interact with each other today and in future.

I learned one interesting thing from the article about the length of SMS messages. I’d never thought much about where the limit on the number of characters came from. According to the article, the limit is 160 (7 bit) characters because that’s what will fit into the control channel messages that mobile phones exchange with cell towers.

“The lucrative nature of that revenue increase cannot be appreciated without doing something that T-Mobile chose not to do, which is to talk about whether its costs rose as the industry’s messaging volume grew tenfold. Mr. Kohl’s letter of inquiry noted that “text messaging files are very small, as the size of text messages are generally limited to 160 characters per message, and therefore cost carriers very little to transmit.” A better description might be “cost carriers very, very, very little to transmit.”

A text message initially travels wirelessly from a handset to the closest base-station tower and is then transferred through wired links to the digital pipes of the telephone network, and then, near its destination, converted back into a wireless signal to traverse the final leg, from tower to handset. In the wired portion of its journey, a file of such infinitesimal size is inconsequential. Srinivasan Keshav, a professor of computer science at the University of Waterloo, in Ontario, said: “Messages are small. Even though a trillion seems like a lot to carry, it isn’t.”

Perhaps the costs for the wireless portion at either end are high — spectrum is finite, after all, and carriers pay dearly for the rights to use it. But text messages are not just tiny; they are also free riders, tucked into what’s called a control channel, space reserved for operation of the wireless network. That’s why a message is so limited in length: it must not exceed the length of the message used for internal communication between tower and handset to set up a call. The channel uses space whether or not a text message is inserted.”

There’s a lot more to the protocols, of course. The Wikipedia SMS article looks like a good place to start.

I learned one interesting thing from the article about the length of SMS messages. I’d never thought much about where the limit on the number of characters came from. According to the article, the limit is 160 (7 bit) characters because that’s what will fit into the control channel messages that mobile phones exchange with cell towers.

“The lucrative nature of that revenue increase cannot be appreciated without doing something that T-Mobile chose not to do, which is to talk about whether its costs rose as the industry’s messaging volume grew tenfold. Mr. Kohl’s letter of inquiry noted that “text messaging files are very small, as the size of text messages are generally limited to 160 characters per message, and therefore cost carriers very little to transmit.” A better description might be “cost carriers very, very, very little to transmit.”

A text message initially travels wirelessly from a handset to the closest base-station tower and is then transferred through wired links to the digital pipes of the telephone network, and then, near its destination, converted back into a wireless signal to traverse the final leg, from tower to handset. In the wired portion of its journey, a file of such infinitesimal size is inconsequential. Srinivasan Keshav, a professor of computer science at the University of Waterloo, in Ontario, said: “Messages are small. Even though a trillion seems like a lot to carry, it isn’t.”

Perhaps the costs for the wireless portion at either end are high — spectrum is finite, after all, and carriers pay dearly for the rights to use it. But text messages are not just tiny; they are also free riders, tucked into what’s called a control channel, space reserved for operation of the wireless network. That’s why a message is so limited in length: it must not exceed the length of the message used for internal communication between tower and handset to set up a call. The channel uses space whether or not a text message is inserted.”

There’s a lot more to the protocols, of course. The Wikipedia SMS article looks like a good place to start.

Yongmei Shi, An Investigation of Linguistic Information for Speech Recognition Error Detection, Ph.D. Dissertation, Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, October 2008.

After several decades of effort, signi?cant progress has been made in the area of speech recognition technologies, and various speech-based applications have been developed. However, current speech recognition systems still generate erroneous output, which hinders the wide adoption of speech applications. Given that the goal of error-free output can not be realized in near future, mechanisms for automatically detecting and even correcting speech recognition errors may prove useful for amending imperfect speech recognition systems. This dissertation research focuses on the automatic detection of speech recognition errors for monologue applications, and in particular, dictation applications.

Due to computational complexity and ef?ciency concerns, limited linguistic information is embedded in speech recognition systems. Furthermore, when identifying speech recognition errors, humans always apply linguistic knowledge to complete the task. This dissertation therefore investigates the effect of linguistic information on automatic error detection by applying two levels of linguistic analysis, speci?cally syntactic analysis and semantic analysis, to the post processing of speech recognition output. Experiments are conducted on two dictation corpora which differ in both topic and style (daily of?ce communication by students and Wall Street Journal news by journalists).

To catch grammatical abnormalities possibly caused by speech recognition errors, two sets of syntactic features, linkage information and word associations based on syntactic dependency, are extracted for each word from the output of two lexicalized robust syntactic parsers respectively. Con?dence measures, which combine features using Support Vector Machines, are used to detect speech recognition errors. A con?dence measure that combines syntactic features with non-linguistic features yields consistent performance improvement in one or more aspects over those obtained by using non-linguistic features alone.

Semantic abnormalities possibly caused by speech recognition errors are caught by the analysis of semantic relatedness of a word to its context. Two different methods are used to integrate semantic analysis with syntactic analysis. One approach addresses the problem by extracting features for each word from its relations to other words. To this end, various WordNet-based measures and different context lengths are examined. The addition of semantic features in con?dence measures can further yield small but consistent improvement in error detection performance. The other approach applies lexical cohesion analysis by taking both reiteration and collocation relationships into consideration and by augmenting words with probability predicted from syntactic analysis. Two WordNet-based measures and one measure based on Latent Semantic Analysis are used to instantiate lexical cohesion relationships. Additionally, various word probability thresholds and cosine similarity thresholds are examined. The incorporation of lexical cohesion analysis is superior to the use of syntactic analysis alone. In summary, the use of linguistic information as described, including syntactic and semantic information, can provide positive impact on automatic detection of speech recognition errors.

This list complements Akshay Java’s Social Media Events calendar which is focused mostly on research conferences. He also invites suggestions which you can submit by email or through comments.

Michael Nielsen has a nice post on using your laptop to compute PageRank for millions of webpages. His posts reviews PageRank and how to compute it and shows a short, but reasonably efficient, Python program that can easily do a graph with a few million nodes. While not sufficient for many applications, like the Web, there are lots of interesting and significant graphs this small Python program can handle — Wikipedia pages, DBLP publications, RDF namespaces, BGP routers, Twitter followers, etc.

The post is part of a series Nielsen is making on the Google Technology Stack including PageRank, MapReduce, BigTable, and GFS. The posts are a byproduct of a series of weekly lectures he’s giving starting earlier this month in Waterloo. Here’s the way that Nielsen describes the series.

“Part of what makes Google such an amazing engine of innovation is their internal technology stack: a set of powerful proprietary technologies that makes it easy for Google developers to generate and process enormous quantities of data. According to a senior Microsoft developer who moved to Google, Googlers work and think at a higher level of abstraction than do developers at many other companies, including Microsoft: “Google uses Bayesian filtering the way Microsoft uses the if statement” (Credit: Joel Spolsky). This series of posts describes some of the technologies that make this high level of abstraction possible.”

Videos of the first two lectures, Introducion to PageRank and Building our PageRank Intuition) are available online. Nielsen illustrates the concepts and algorithms with well-written Python code and provides exercises to help readers master the material as well as “more challenging and often open-ended problems” which he has worked on but not completely solved.

Nielsen was trained as a as a theoretical Physicist but has shifted his attention to “the development of new tools for scientific collaboration and publication”. As far as I can see, he is offering these as free public lectures out of a desire to share his knowledge and also to help (or maybe force) him to deepen his own understanding of the topics and develop better ways of explaining them. In both cases, it an admirable and inspiring example for us all and appropriate for the holiday season. Merry Christmas!

We built my.hakia.com, which does exactly what is described above. A screenshot is shown below.

The screenshot above tells the whole story except one important differentiator.

Semantic technology of hakia allows high-level of precision compared to any other similar platform. This enables the user to park highly specific questions against the emerging news. Therefore, my.hakia.com can be considered as “intelligence gathering dashboard”. Let us tell you how.

If you search Google news for Obama’s strategy for the new team, you will see that the results are mostly irrelevant. Try to create a Google alert for this query and see the results for yourself on a continuous basis.

The same search at hakia for Obama’s strategy for the new team produces dead-on results. This is because semantic technology does not need “link referrals” to pull relevant results unlike Google-esque search engines. For dynamic content like news, there is no time to collect “Link-referral” statistics and that is why Google-esque search engines fail beyond simple triggers. Try the same query at Yahoo, it displays the same confusion.

This fundamental differentiation is a valuable asset for my.hakia.com users because they will be getting precise results for specific interests that they cannot get it anywhere else. Some ideal cases are outlined below;

- Monitor your business competitors
- Get information on latest progress in the treatment of diseases
- Keep an eye on your favorite artists by activity (like album releases)
- Stay in touch with particular economic developments (such as in real estate in your city)

Try my.hakia.com, and tell us if we have met your expectations.

Happy holidays

Videolectures captured the slides for most of the presentations (which are available for downloading) and their site shows both the the speaker’s video and slides in synchronization. Videolectures used three camera crews in parallel so were able to capture almost all of the presentations. Here are some highlights from the ~90 videos to whet your appetite.

• Introduction to the Semantic Web: An all-day tutorial featuring ten top researchers covering all aspects of the Semantic Web.
• How to Publish Linked Data on the Web: A three-hour tutorial by five researchers explaining how LOD works and how you can exploit it.
• Freebase: An Open, Writable Database of the World’s Information: A keynote by MetaWeb CTO John Giannandrea on Freebase.
• Semantic Web Challenge & Billion Triple Challenge: The session on the Semantic Web Challenge finalists.
• Semantic Wikis: Fusing the two strands of the Semantic Web: Mark Greaves on Semantic Wikis.
• Requirements Critic: A Tool for Automatically Analyzing Software Requirements Documents: The winner of the best paper award for the In Use track.
• Laconic and Precise Justifications in OWL: Winner of the best paper award for the Research Track.
• An OWL 2 Far?: A panel that takes up the question of whether having standard languages based on formal methods with steadily increasing power is the right way to support the Semantic Web.

Thomas H. Briggs, Constraint Generation and Reasoning in OWL, 2008.

The majority of OWL ontologies in the emerging SemanticWeb are constructed from properties that lack domain and range constraints. Constraints in OWL are different from the familiar uses in programming languages and databases. They are actually type assertions that are made about the individualswhich are connected by the property. Because they are type assertions these assertions can add vital information to the individuals involved and give information on how the defining property may be used. Three different automated generation techniques are explored in this research: disjunction, least-common named subsumer, and vivification. Each algorithm is compared for the ability to generalize, and the performance impacts with respect to the reasoner. A large sample of ontologies from the Swoogle repository are used to compare real-world performance of these techniques. Using generated facts is a type of default reasoning. This may conflict with future assertions to the knowledge base. While general default reasoning is non-monotonic and undecidable a novel approach is introduced to support efficient contraction of the default knowledge. Constraint generation and default reasoning, together, enable a robust and efficient generation of domain and range constraints which will result in the inference of additional facts and improved performance for a number of Semantic Web applications.

“We see that platforms for distributed computing will be of such high importance in the future that it is crucial to have a wide variety of different approaches which produces healthy competition and co-evolution between the projects. In this respect, Hadoop and Disco can be seen as complementary projects, similar to Apache, Lighttpd and Nginx.

It is a matter of taste whether Erlang and Python are more suitable for the task than Java. We feel much more productive with Python than with Java. We also feel that Erlang is a perfect match for the Disco core that needs to handle tens of thousands of tasks in parallel.

Thanks to Erlang, the Disco core remarkably compact, currently less than 2000 lines of code. It is relatively easy to understand how the core works, and start experimenting with it or adapt it to new environments. Thanks to Python, it is easy to add new features around the core which ensures that Disco can respond quickly to real-world needs.”

The Disco tutorial uses the standard word counting task to show how to set up and use Disco on both a local cluster and Amazon EC2. There is also homedisco, which lets programmers develop, debug, profile and test Disco functions on one local machine before running on a cluster. The word counting example from the tutorial is certainly nicely compact:

from disco.core import Disco, result_iterator

def fun_map(e, params):
    return [(w, 1) for w in e.split()]

def fun_reduce(iter, out, params):
    s = {}
    for w, f in iter:
        s[w] = s.get(w, 0) + int(f)
    for w, f in s.iteritems():
        out.add(w, f)

results = Disco(”disco://localhost”).new_job(
		name = “wordcount”,
                input = ["http://discoproject.org/chekhov.txt"],
                map = fun_map,
		reduce = fun_reduce).wait()

for word, frequency in result_iterator(results):
	print word, frequency

Back in May

In the intervening months many including myself used Ed’s work as a pointer to how useful publicly available data could, with the use of open Linked Data principles, become a valuable part of sites and services across the globe.   For instance another Talking with Talis interviewee Martin Malmsten, from the Royal Library of Sweden, almost immediately made use of the links to the LCSH data.  Ed went on to get lots of feedback, and wrote a paper which he then presented at DC2008.

It is therefore with great disappointment that I read this on the lcsh.info site the other day:

On December 18th I was asked to shut off lcsh.info by the Library of Congress. As an LC employee I really did not have much choice other than to comply.

As a LC employee he was put in an untenable position when they obviously decided that they didn’t like this useful service based on publicly available data being delivered from a domain that doesn’t end in loc.gov.  I wonder if there are any other Linked Data enthusiasts, not held back by who their employer is, who would pick up from where he left off?

Ed goes on to say:

It was always my intention for concept URIs at lcsh.info to be cool. I advertised the service as ‘experimental’ and indicated it was going to hopefully inform the development of a similar continually updated service at LC where I work. …  My thought was I could leave the service running until there was something similar at LC that I could redirect the concept URIs to. After a year or two when people had rewritten there data to point at loc.gov I could retire lcsh.info. I never imagined I would be asked by LC to take it down.

LOC should have listened to Ed in the first place and taken the high ground in leading the work in to creating a semantic web of data with their valuable publicly available data.  At the end of his post Ed hints that LC is still considering running a service like lcsh.info at loc.gov, but it’s not there yet.  Why-o-why did they not learn from his work and ride the wave of introducing their own service based on his great initiative.  Instead they present to the world a short-termist not-invented-here attitude, that reminds me of other well established leviathans of the world of library metadata.

Let’s hope that Ed’s hint is correct and we will soon be able to welcome the release of Open Linked LCSH and other Data from the electronic portals of the LofC.

Traffic Squad Police (LOC) image published in the The Library of Congress’ photostream on Flickr

Share This

• RIF Use Cases and Requirements: minor changes
• RIF Core: new design to support both BLD and PRD
• RIF Datatypes and Built-Ins 1.0: various improvements
• RIF Production Rule Dialect (PRD): operational semantics are complete
• RIF Test Cases: early stages of test suite

For some people, the point of the Semantic Web is distributed, web-friendly knowledge management and knowledge representation. Generally we’re in that camp. But that camp breaks down into several factions, and it’s useful to be clear about which faction we’re in.

There is a spectrum that runs from Maximum Fidelity to Maximum Scalability. Given our roots in Description Logic, we lie somewhere in-between these two poles. Notice that I have intentionally avoided calling these “extremes”; they are endpoints, and perfectly respectable, useful ones, depending on who you are and what you’re trying to achieve.

The Max Fidelity folks want to model as closely as possible some world-chunk in as fine-grained and faithful manner as is possible. This often means that they are at least first order logic fans, and sometimes higher-order logic users. They debate edge cases, corner cases, alternate and competing semantics and logics in an attempt to ever more faithfully mirror reality. The price they pay is, generally, computability. For some use cases, that price is perfectly acceptable. For other use cases, that price is entirely too high, since the most perfect representation of the world is useless if you can’t practically compute with it—at least, that’s how Max Fidelity often looks to us.

At the far end of the spectrum we have Max Scalability folks, for whom the point of the Semantic Web is rather more the “Web” than the “Semantic” part—we might playfully call them the “semantic WEB” crowd, in order to reflect their ideal ratio. Here the point isn’t to model perfectly; but, rather, to do something with lots and lots of data, ideally Webfuls of data. This means, in the argot of current tech choices, that they tend to be RDF and Linked Data fans and users, since that’s just about the only approach to doing anything at all interesting with Webfuls of data. The price they pay, of course, is expressivity. For some use cases, that’s just fine, since you don’t always need a lot or even much semantic fidelity to get the job done. Sometimes we build applications for customer that take this approach. But, as above, for other use cases, this is simply a killer, because without enough or the right semantics, you don’t get the right kind of help from the machine in figuring out complex stuff.

So what do we have so far? First, we have a notional (and idealized) spectrum that runs from Webfuls of data to, roughly, at least first order logic. Second, we have obviously tons of interesting use cases at (probably) every point along this spectrum. And, third, we have the suggestion that we aim for some kind of sweet spot in the middle—where “sweet spot” and “in the middle” are not absolute notions, but are interest-relative and goal-specific, and where the interests and goals we care about are, surprise-surprise, ours.

(In other words, I’ve setup a little fantasy where we are the Heroes—where we naturally occupy the “sweet spot”—but then, since I’m not a complete jerk, I’ve ironized or called into question that very fantasy in an effort to suggest that we, just like everyone else, try to spin things to make ourselves look smart, cool, and useful.)

And—will miracles never cease?—that’s just about where Description Logic fits along such an idealized spectrum. Technically, it’s the decidable subset of first order logic, which means that we try to balance Fidelity and Scalability in a way where we can get some of both.

The Max Fidelity folks are forever poking us with sticks to the effect that we can’t model world-chunks nearly as faithfully as they can. Well, no crap, of course we can’t! Then the Max Scalability folks poke us with different sticks to the effect that we can’t scale to Webfuls of data—again, no duh!

And then we poke back at both camps—hey, they started it!—to the effect that we can model far better than Max Scalers and we can scale far further than Max Fideliters (yes, I just made that word up…Rock!)...

Finally, a word about how this positioning issue plays out in our approach to modeling. In short, we model such that we get the right inferences, since getting the inferences is typically what our kind of applications (analysis, decision support kinds of apps, in short) are all about. So that means some edge or corner cases, even if they fit into DL, get ignored or dropped out or even distorted when there’s no point—given requirements analysis—to fidelity for its own sake. And it means, on the flip side, that we don’t worry too much that that inference over Webfuls of data is not realistically achievable anytime soon. Fast enough for the customer’s data is sufficient scalability in most cases for us.

Related Articles

• Thinking ‘Inside the Box’ with Description Logics
• The Tribes of the Semantic Web

The perceived utility of automated reasoning for a wide range of applications matters to us greatly, which makes sense, given that our biz proposition is “semantic infrastructure OEM”. In other words, we’re trying to make money by licensing reasoning infrastructure, and related pieces, for semantic applications to other developers to use in their apps. With the right APIs and tool maturity, as well as supporting materials, our customers should be able to treat automated reasoning as a black box—not a black art.

A problem with demonstrating automated reasoning’s utility is that automated reasoning is complex, with non-trivial logical background and framework, including oodles of domain-specific vocabulary. Another problem is that automated reasoning is, in the end, just a kind of mechanical term rewriting often according to, considered individually, quite trivial rules. (Pellet isn’t really a rules engine, but we’ll talk about that another time.)

That means that for toy cases, which is what most people new to the subject are ready for, it seems dull and unimpressive. And for the hard cases? Well, most people aren’t ready for hard cases, so they simply tune out. And who can blame them, really? It’s like my example about Emma and Jack. I mean, that example really sucked, but what’s the alternative?

This is not an easy problem to solve.

My approach, rather than showing more toy or real examples, is just to talk about the utility of automated reasoning in plain language, in an attempt to communicate not so much specific details as the general mindset or approach to solving particular sorts of problems using automated reasoning. This approach to marketing mirrors our approach to technology development: both are iterative and experimental, but not just for us. As the man said, even a blind pig occasionally finds an acorn.

Related Articles

• Why Reasoning Matters: It’s Not What You Know… (2)
• Why Reasoning Matters: Explanations (3)
• Upcoming on This Blog…

I don’t have slides for my time at the APQC in Houston, I was not slated to present, so no cool slide widget with my presentation in this post.  I was merely there to observe and learn, and maybe answer some questions about POPS.

As Kendall mentioned, POPS was nominated as a best practice as part of NASA JPL’s overall efforts in Knowledge Management. The meeting at APQC was for all the nominees to give a short talk and to hear the overall findings of the study conducted by APQC, which in this case was on Expertise Location and Social Networking.

I got to see some great presentations by folks from IBM, Sun, Pratt-Whitney, Rockwell Collins, and Mitre and get a lot of insight into what they’re doing with Expertise Location and Social Networking; challenges they faced in the past, lessons learned, and what they’re doing now, and in the future, to continue their efforts in these areas.

It was a great experience, the people from APQC were fantastic, very friendly and put on a great event, and all the nominees and study partners, a group which included L3, Marathon Oil, ExxonMobil, and Wyeth, were all great and added a lot to the discussions.

Hopefully I get the chance to participate or work with APQC in the future.

“A journal’s Eigenfactor score is our measure of the journal’s total importance to the scientific community. With all else equal, a journal’s Eigenfactor score doubles when it doubles in size. Thus a very large journal such as the Journal of Biological Chemistry which publishes more than 6,000 articles annually, will have extremely high Eigenfactor scores simply based upon its size. Eigenfactor scores are scaled so that the sum of the Eigenfactor scores of all journals listed in Thomson’s Journal Citation Reports (JCR) is 100.
…
A journal’s Article Influence score is a measure of the average influence of each of its articles over the first five years after publication. Article Influence measures the average influence, per article, of the papers in a journal. As such, it is comparable to Thomson Scientific’s widely-used Impact Factor. Article Influence scores are normalized so that the mean article in the entire Thomson Journal Citation Reports (JCR) database has an article influence of 1.00.”

For example, here are the ISI-indexed journals in the AI subject category ranked by the Article Influence score for 2006.

The site makes good use of GoogleDoc’s motion charts to visualize the changes of metrics for top journals in a subject area. You can also interactively explore maps that show the influence of different subject categories on one another as estimated from journal citations.

The details of the approach and algorithms are available in various papers by Bergstrom and his colleagues, such as

M. Rosvall and C. T. Bergstrom, Maps of random walks on complex networks reveal community structure, Proceedings of the National Academy of Sciences USA. 105:1118-1123. Also arXiv physics.soc-ph/0707.0609v3 [PDF]

(spotted on Steve Hsu’s blog)

Here are the guidelines from the RNA Biology site:

“To be eligible for publication the Supplementary Material must contain: (1) a link to a Wikipedia article preferably in a User’s space. Upon acceptance this can easily be moved into Wikipedia itself together with a reference to the published article.
…
At least one stub article (essentially an extended abstract) for the paper should be added to either an author’s userspace at Wikipedia (preferred route) or added directly to the main Wikipedia space (be sure to add literature references to avoid speedy deletion). This article will be reviewed alongside the manuscript and may require revision before acceptance. Upon acceptance the former articles can easily be exported to the main Wikipedia space. See below for guidelines on how to do this. Existing articles can be updated in accordance with the latest published results.”

This is definitely an interesting and forward looking idea. Yet, I can not help having the cynical thought that it’s also a great way for the journal to boost it’s page rank.