A short note on Semantic Search Engines

Lars featured an article on Hakia, a 'semantic search engine' in his blog today.
Contrarywise to Google, Hakia uses natural language processing (NLP) to 'understand' search queries given in natural language (and not as plain keywords). Ok, this is now new.
Just remember AskJeeves, a.k.a. Ask.com. But, in difference to that search engine, Hakia claims to perform a 'semantic search' (and not a keyword based search). If you read a little bit further in their technical description at Hakia labs you will find thet they are using a parser called 'OntoSem', which as they claim is able to perform a 'deep semantic analysis' of sentences. This parser is used for query string analysis and also for the generation of the search index. But, their search index is different to that we are used from 'traditional' search engines.
Just remember, traditional keyword based search engines extract so called 'descriptors' from web pages that are used to describe the content of those pages. These descriptors are managed within an inverted index. Thus, by accessing the inverted index with a descriptor (=keyword in query string), a list of web pages will be returned together with some weights indicating the relevance of the descriptor for the page.
So, how does the index work in Hakia? They clain, that their parser is analyzing all web pages to be indexed sentence by sentence. In effect, all possible questions that can be posed for each sentence are generated, forming what they call the 'QDEX data'. All possible questions for all sentences in all pages have to be stored in an index-like data structure (simply for fast and efficient access). If now a query string contains a question, this question is mapped against the index, resulting in a large number of 'relevant' (questions, sentences and in the end...) web pages. Now they apply a 'smart' algorithm called 'Semantic Rank' which orders the resulting list of documents according to their relevance wrt. the question given in the query string. More details about the technique is not published (at least not to my knowledge).

Thus, the only way to find out about the quality of their approach is to try out their search engine. A nice feature is that they have included small example applications where you may try out OntoSem or QDEX interactively by yourself. I have only tried OntoSem (because for QDEX you have to sign up a request) and the result was not really different from any standard NLP-Parser (and thus not really convincing. I will sign up and give QDEX a try..and of course I will post the result).

If I compare their approach to Google's, the problem is that Google is also claiming to incorporate semantic technology altogether. So, again I can only compare the results of my queries. I have tried several queries and have come to the following results:

• in general the results from hakia do look very promising!


• in comparison to Google results, they are not really that different this may come because of my 'queries' and thus, my results are probably not really objective).
  Let me give you an example: I was asking both (Google and Hakia) the question 'When did the Semantic Web start?'. Ok, it's not so easy to understand at all. First, it didn't really start (as an being implementation), but the topic itself startet several years ago...and so I was curious about the results:
  
  
  
  • Hakias result list presented an ecrim article (being co-authored by Tim Berners-Lee) entitled 'The semantic web lifts off' as top result.
  
  
  • Google's result list on the other hand presented (as second result) an artice entitled 'When did the Semantic Web enter our lexicons?', which at least answered one part of my original question.
  
  

Ok, as stated before, this is not representative at all, but I will keep an eye on Hakia anyway. As soon as I have more representative experiences (and hopefully some comments or other interesting comparisons), I will write about.

One thing in the end. After all, what I have read about Hakia and on their web site, they seem to have a completely different notion of 'Semantic Search'. What they do is to apply (enhanced) information retrieval techniques based on natural language processing. They do NOT evaluate any additional given semantic annotation (RDF, OWL, etc...), which might be addid into the web pages or connected to them. They claim (same as Peter Norwig) that the average user is much too stupid to supply semantic annotation (because for doing that he needs to be a linguist as they say).
Of course, if you are following the PLAIN trail of the W3C and encode everything by hand (HTML, XML, RDF, OWL, SWRL, etc...) then you have to be some real expert. But today, you (at least most of you) don't encode HTML by your own. Remember, there are lots of real nice WYSIWYG editors for doing that AND (even more) there are several simplifications (just think of writing your blog). In the same way there will be smart user interfaces and editors providing help in generating semantic annotations for your texts. First and most simple step is providing labels and keywords (just 'tags') for your blog posts. Most people do that...just because (1) they want to put some order into the set of their postings and (2) they want their posting to be found.

Document Retrieval vs. Fact Retrieval - In Search for a Qualified User Interface

Today, if you are looking for information in the Web, you enter a set of keywords (query string) into a search engine and in return you will receive a list (= ordered set) of documents that are supposed to contain those keyword(s) (or their word stem). This list of documents (therefore 'document retrieval') is ordered according to the document's relevance with respect to the user's query string. 'Relevance' - at least for Google - refers to PageRank. To make it short, PageRank reflects the number of links referring to the document under consideration, each link weighted with its own relevance being adjusted by the number of total links starting at the document that contains this link (in addition with some black magic that is still under copyright restriction, see U.S. Patent 6285999).
But, is this list really what the user expects for an answer? O.k. meanwhile, we - the users - have become used to this kind of search engine interface. In fact, there exist books and courses about how to use search engines in order to get the information you want. Interesting fact is that it is the user, who has to get adapted to the search engine interface....and not vice versa.
Instead it should be the other way around. The search engine interface should get adapted to the user - and even better to each different user! But, how then should a search engine interface should look like? In fact, there are already search engines that are able to give the answer to simple questions ('What is the capital of Italy?'). But, they stil fail in answering more complex questions ('What was the reason for Galileo's house arrest?').

In real life - at least if you happen to have one - if you are in need for information, you have different possibilities to get it:

1. If there is somebody you can ask, then ask.
   
2. If there is nobody to ask, then look it up (e.g. in a book).
   
3. If there is nobody to ask, and if there is no way to look it up, then think!
   

Let's consider the first two possibilities. Both do also have their drawbacks: Asking somebody is only helpful, if the person being asked does know the answer. (O.k., there is also the social aspect that you might get another reward just by making social contact...instead of getting the answer). If the person does not know the answer, maybe she/he knows, whom to ask or where to look it up. But we might consider this fact as being a kind of referential answer. On the other hand, even if the person does know the answer, she/he might not be able to communicate the answer. Maybe you speak different languages (not necessarely different languages in the sense of 'English' and 'Suaheli', but also consider a philosopher answering the question of an engineer...). Sometimes you have to read in between the lines to understand somebody's answer. At least, in some sense we have to 'adapt' to the way the other person is giving the answer to understand the answer.
Considering the other possibility of looking up the information, we have the same situation as if asking the www search engine. E.g., if we look up an article in an encyclopedia, we use our knowledge of how to access the encyclopedia (alphabetical order of entries, reading the article, considering links to other articles...being able to read...).
Have you realized that in both cases we have to adapt ourselves to an interface. Even when asking sombody, we have to adopt to way this person is talking to us (her/his level of expertise, background, context, language, etc.). From this point of view, adapting to the search engine interface of Google seems not to be such a bad thing at all....

If it comes to fact retrieval, the first thing to do is to understand the user's query. To understand an ordinary query (and not only a list of interconnected query keywords), natural language processing is the key (or even as they say the 'holy grail'). But even, if the query phrase can be parsed correctly, we have to consider (a) context and (b) the user's background knowledge. While the context helps to disambiguate and to find the correct meaning of the user's query, the user's background determines its level of expertise and the level of detail in which the answer is best suited for the user.

Thus, I propose that there is no such thing as 'the perfect user interface'. Anyway, different kind of interfaces might serve for different users in different situations. No matter how the interface will look like, we - the users - will adapt (because we are used to do that and we learn very quickly). Of course, if the search engine is able to identify the circumstances of the user (maybe she/he's retrieving information orally with a cell phone or the user is sitting in front of a keyboard with a huge display) the search engine may choose (according to the user's infrastructure) the suitable interface for entering the query as well as for presenting the answer...