Find What I Mean, Not What I Type

The following are notes from a workshop given by Chris Sherman on 9/15/2009.

Many people today conduct an internet search like a slot machine - they drop a few keywords into Google, hit enter, and then hope to hit the jackpot.  What we really want is for our search engines to read our minds.

How do search engines search?  They use search algorithms that might include hundreds of ranking factors.  These algorithms rely heavily on text pattern matching, statistics, and observations of searcher behavior.  There is some semantic analysis, but it's very limited.  What we need is a true understanding of language and meaning.

WHAT IS "SEMANTIC SEARCH"?

• It focuses on delivering answers, not just search results
• Key:  Disambiguation of both queries and content on the web - helps you narrow down to precise question and answer you intended
  
• Puts a heavy emphasis on natural language processing and on understanding both the meaning AND the intention.

WHAT SEMANTIC SEARCH IS NOT:

• It is not just NLP - or natural language processing
• It is not a replacement for navigational queries
• It is not the semantic web
• It is not simply refining or filetering a search into categories, such as done by Northern Light and Vivisimo
  

Examples of search engines that use Semantic Search features:

•   Powerset   by Microsoft.  One of Chris's favorites.  (enter any search term, such as "global warming")  Powerset uses data from Wikipedia. Notice the relationships you uncover that you don't normally get in a routine search.
  
• Hakia - (still in beta at this time) "hakia’s semantic technology provides a new search experience that is focused on quality, not popularity. hakia’s quality search results satisfy three criteria simultaneously: They (1) come from credible Web sites recommended by librarians, (2) represent the most recent information available, and (3) remain absolutely relevant to the query."   Sometimes Hakia gets really bad results, but it is still in beta.
• True Knowledge - Probably the first true "answer engine".  Its API returns structured results from machine-based queries.   "A user can treat our site just as you did your previous search engine, but in addition to receiving a list of documents, your query will also be passed through our technology - if we can help more directly we will."  SAMPLES 
• Wolfram Alpha - Chris considered this one to have the best chance of surviving.  W/A can combine lots is disparate data types and extracts meaning and relationships on the fly. "We aim to collect and curate all objective data; implement every known model, method, and algorithm; and make it possible to compute whatever can be computed about anything."
• Transinsight (GoPubMed.org) - another of Chris's favorites. Very good at disambiguation of queries and categorization of search results.  Something very helpful is that it will identifiy the "top author" in a particular subject area.  But it only works with highly structured corpora, and the format is a little confusing.
• Lexee
  

THIS POST STILL IN PROGRESS