IBM Watson Performance: Just an IBM Issue?
September 6, 2017
I read “IBM Pitched its Watson Supercomputer As a Revolution in Cancer Care. It’s Nowhere Close.” Here in Harrod’s Creek, doubts about IBM Watson are ever present. It was with some surprise that we learned:
But three years after IBM began selling Watson to recommend the best cancer treatments to doctors around the world, a STAT investigation has found that the supercomputer isn’t living up to the lofty expectations IBM created for it. It is still struggling with the basic step of learning about different forms of cancer. Only a few dozen hospitals have adopted the system, which is a long way from IBM’s goal of establishing dominance in a multibillion-dollar market. And at foreign hospitals, physicians complained its advice is biased toward American patients and methods of care.
The write up beats on the lame horse named Big Blue. I would wager that the horse does not like being whipped one bit. The write up ignores a problem shared by many “smart” software systems. Yep, even those from the wizards at Amazon, Facebook, Google, and Microsoft. That means there are many more stories to investigate and recount.
But I want more of the “why.” I have some hypotheses; for example:
Smart systems have to figure out information. Now on the surface, it seems as if Big Data can provide as much input as necessary. But that is a bit of a problem too. Information in its various forms is not immediately usable in its varied forms. Figuring out what information to use and then getting that information into a form which the smart software can process is expensive. The processes involved are also time consuming. Smart software needs nannies, and nannies which know their stuff. If you have ever tried to hire a nanny who fits into a specific family’s inner workings, you know that the finding of the “right” nanny is a complicated job in itself.
Let’s stop. I have not tackled the mechanism for getting smart software to “understand” what humans mean with their utterances. These outputs, by the way, are in the form of audio, video, and text. To get smart software to comprehend intent and then figure out what specific item of tagged information is needed to deal with that intent is a complex problem too.
IBM Watson, like other outfits trying to generate revenue by surfing a trend, has been tossed off its wave rider by a very large rogue swell: Riffing on a magic system is a lot easier than making that smart software do useful work in a real world environment.
Enterprise search vendors fell victim to this mismatch between verbiage and actually performing in dynamic conditions.
Wipe out. (I hear the Safaris’ “Wipe Out” in my mind. If you don’t know the song, click here.)
IBM Watson seems to be the victim of its own over inflated assertions.
My wish is for investigative reports to focus on case analyses. These articles can then discuss the reasons for user dissatisfaction, cost overruns, contract abandonments, and terminations (staff overhauls).
I want to know what specific subsystems and technical methods failed or cost so much that the customers bailed out.
As the write up points out:
But like a medical student, Watson is just learning to perform in the real world.
Human utterances and smart software. A work in progress but not for the tireless marketers and sales professionals who want to close a deal, pay the bills, and buy the new Apple phone.
Stephen E Arnold, September 6, 2017
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
