When Smart Software Is Really Dense
May 4, 2010
Search and content processing has hidden rooms and musty basements. Many vendors put some apple cider in a pan, turn the burner on low, and suffuse the manse with a mouth-watering scent. Yep, that’s marketing.
“Rest in Peas: The Unrecognized Death of Speech Recognition” is not about cooking veggies. The write up does a good job of pointing out that the progress of taking a spoken string and figuring out what was said is going nowhere fast. The write up includes a number of useful comments, some tables, links to oh-so academic papers, and some blunt talk. One example:
To some, these developments are no surprise. In 1986, Terry Winograd and Fernando Flores audaciously concluded that “computers cannot understand language.” In their book, Understanding Computers and Cognition, the authors argued from biology and philosophy rather than producing a proof like Einstein’s demonstration that nothing can travel faster than light. So not everyone agreed. Bill Gates described it as “a complete horseshit book” shortly after it appeared, but acknowledged that “it has to be read,” a wise amendment given the balance of evidence from the last quarter century. Fortunately, the question of whether computers are subject to fundamental limits doesn’t need to be answered. Progress in conversational speech recognition accuracy has clearly halted and we have abandoned further frontal assaults.
I have tucked this write up in my reference folder. I also discussed the article with my trusty crew at lunch. What emerged from the conversation were three points.
First, speech recognition is just one small, dark, hidden room in search. There are many more. Hopefully Robert Fortner or a person with his knowledge will open the blinds and let the sun shine in.
Second, the write up strikes at the heart of Google’s method. At lunch, I said that Facebook uses humans to generate content and provide relevant results. Nothing could be more different from the math club approach used at Google. The references to algorithms in the write up reminded me that Google had a great idea a decade ago. That idea may be less and less useful in some of the dark rooms of search. Has Facebook found a way to open search doors? I don’t know but it is an interesting question, is it not?
Finally, I have seen other dead end charts. One example that struck me was the lack of progress in precision and recall tests. The US government’s search “competitions” are not in the public relations business. I have heard that improvements in the formal precision and recall scores of some of the participants are * very * hard to improve. We are not talking 90 percent and up. We are talking 75 percent and higher with 90 percent a darned wonderful score. How many years? A decade, give or take a year or two. Another musty room? Maybe.
To wrap up, search is a tough problem. Search is not getting any easier either.
Stephen E Arnold, May 4, 2010
Unsponsored post.
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
