Neomeme

I was lucky enough to get an invite to the alpha of Freebase, the semantic Web 3.0 knowledge application that everyone’s talking about. I’ve been eagerly playing with it for most of last night and some of last morning, and I like what I see. Tim O’Reilly gives a good overview of the concepts behind Freebase, but he does not really demonstrate the killer app, the features that make Freebase so damn… cool.

Basically, Freebase is a fully object-oriented Wikipedia. The concept will be instantly familiar to OOP programmers- every entry is an object. As an object, it has attributes called fields, such as “Name” and descends from classes called types, like “Film” or “Person”. An article will have different fields based on the type it is- for example, Person will have fields like birthdate (in turn an object of type “date”). Objects can inherit characterisics from multiple classes, like Person and Blogger. Metaweb, the company behind Freebase aptly terms it an open-data project.

Okay, enough with the geeky stuff. What are some of the cool things we can do with Freebase right now, the “killer apps” that make it shine?

Today’s Topic:

Using Freebase as a Hyper-Powerful Search Engine

Say I remember hearing a friend, who is a huge fan of 80’s metal, play an album called “Metal”, and I want to find out more about it. I don’t remember the name of the artist or the exact names of any songs(this has happened to me before, so the situation is very plausible). A Google search for “metal“, or even “metal album” will result in information on music of the metal genre, but not the specific album I was looking for. A Wikipedia search will first lead me to the page for the element, and then to this disambiguation page. Finally! Two albums called “Metal” are listed, one by Preston Reed(1995) and one by Annihilator(2007). But wait… my friend,ever the purist, listens exclusively to 80s metal. According to the Wikipedia page, there were only two albums called “Metal”, neither released in the 80s. Did I misunderstand? Is this album just a figment of my imagination? For all intents and purposes, there is no way for me to find this album. Now, here’s what I do with Freebase:

I simply type “Metal” into the search box, and I am instantly presented with everything that Metal could be:

You can already see that Metal, the 1982 album by Manilla Road is in the list. I know it is a musical album because Freebase tells me its type under the name, “Musical Album”.

But what if it wasn’t at the top of list? Instead of manually crawling through my results, as I would do with a Google query for a word with multiple meanings, I can filter my results by type, so that only musical albums matching “Metal” are shown:

And the album I was looking for will be first in the list.

Incidentally, Wikipedia does have an entry for the album I was looking for. But, the human editors of Wikipedia forgot to add this obscure album to the disambiguation page for Metal. Freebase’s algorithms make no such mistakes.

But searching by types, which is like Dumbfind on steroids, is only the tip of the Freebase search iceberg. The creators of Freebase super-charged its search by including something called Lucene Search Syntax. Basically, this is an extremely flexible and powerful search syntax that is ages ahead of anything Google or Wikipedia have implemented. First of all, it supports wildcard searches, so a search for politic* returns results like “politician” and “politics”. Users have been clamoring for wildcard search of the web for years, with no results. This is particularly useful for searching for words you don’t know how to spell, which Wikipedia offers no help with. Fuzzy searches, which direct the user to linguistically similar terms are also supported and rather well-implemented.

Although some more advanced searches are rather slow, the search functionality shows great promise. Already, it can be used much more effectively than the current Google+Wikipedia hybrid which dominates our search for knowledge.

Stay tuned for Part II..many more exciting and unprecedented uses for semantic databases like Freebase await!