During the week of Valentine’s day earlier this month, IBM’s Watson, a Jeopardy playing supercomputer, handily beat two human contestants – Ken Jennings and Brad Rutter – who until now were the most successful winners. IBM announced that they will sell the question-and-answer natural language processing technology to hospitals and call centers.
IBM fed Watson the entire contents of Wikipedia, dictionary, thesaurus and the internet movie database, without any special processing or categorization. Watson’s technology enables it to assimilate the raw data and store it in a form that enables rapid retrieval. Contestants will ring in the buzzer as soon as Alex Trebek has finished asking the question so Watson has very few seconds to determine the right answer. While Ken and Brad had to listen to the question and manually ring the buzzer, Watson also had to manually ring the buzzer but it was fed the question in a text form as soon as it appeared to all contestants on the video display. Watson proved to be incredibly quick at the buzzer, especially in the first of two contests, but the point is not about winning. It is about how much natural language technology has progressed to the point that tricky questions with metaphorical and alliterative undertones can be parsed and processed by a computer. What does all this mean to internet technology?
First, I should point out that a San Francisco-based company, Powerset, released a search engine back in 2008 that used natural language technology to provide answers and relevant links. For instance, asking “How many days to christmas” will result in a number, not a page full of web links that the user has to view and hunt for an answer. Microsoft bought Powerset in 2008. Today when you search for the same query on Bing, you get back a page full of web links, as does Google. So for some reason, Microsoft has not meaningfully leveraged natural language search technology. Google and Microsoft should fear Watson. Google’s primary value proposition is serving relevant search results and it appears Watson is much more capable of meeting that customer need if deployed correctly.
Internet forums depend upon humans to respond to each other. Today, on average, there is a low probability of getting a timely and high quality answer. Ask Watson “What kinds of plants will do well in my backyard” and it gives you a useful set of recommendations, along with a confidence score, much like how it showed its confidence score for each of the top three answers it determined for every question on Jeopardy. Sites like Yahoo Answers, Amazon’s Askville and Ask Jeeves should be worried.
IBM has already announced it will serve call centers where this technology makes a lot of sense. In particular, this will be valuable to companies that have a wealth of knowledge base that Watson can mine. FAQs on websites will still have to be curated manually to meet stylistic guidelines and for now Watson is unlikely to encroach into that space.
Crab Tales 