IEEE Spectrum Embraces Business Analysis: IBM Watson and Health
April 8, 2019
I spotted a link to “How IBM Watson Overpromised and Under delivered on AI Healthcare.” I read the article and found it reasonably balanced. What surprised me was the fact that the editors of IEEE Spectrum believed that this particularly collection of information should be published for the magazine’s and online audience. My suspicion is that IBM was promoting its technology in a manner that was egregious. IEEE was reminding its readers about veering from technical facts into the wild and crazy world of toothpaste and dandruff shampoo marketing. Then I realized the IEEE Spectrum was explaining an example digital snake oil:
I circled in Big Blue marker this passage:
Outside of corporate headquarters, however, IBM has discovered that its powerful technology is no match for the messy reality of today’s health care system. And in trying to apply Watson to cancer treatment, one of medicine’s biggest challenges, IBM encountered a fundamental mismatch between the way machines learn and the way doctors work.
Translation: Reality is different from a demo. When demos are built on software which has proven problematic for decades, one wonders how the square peg in the round hole gets funded.
I circled this statement:
… Even today’s best AI struggles to make sense of complex medical information. And encoding a human doctor’s expertise in software turns out to be a very tricky proposition. IBM has learned these painful lessons in the marketplace, as the world watched. While the company isn’t giving up on its moon shot, its launch failures have shown technologists and physicians alike just how difficult it is to build an AI doctor.
IEEE Spectrum does not use the word “desperation” but it applies. The reality, from my point of view, is that finding information and answering questions is difficult. Google pulls off a version of question answering by hooking relevance to behavior and possibly relevant advertisements. Precision and recall are not part of Google or other commercial search vendors’ vocabulary today.
But answering questions doesn’t work all that well today. Sorry Google.
“Regular” search— particularly search based on open source software, some home brew code, and acquired technology — is difficult to make work across different types of content and use cases. The dust up between HP and Autonomy is one example of what happens when “logical” explanations don’t apply to search and retrieval. There are other examples too. Just ask a Fast Search & Transfer executive who skirted serious jail time.
IEEE Spectrum’s article drives home failure this way:
In a final blow to the dream of an AI super doctor, researchers realized that Watson can’t compare a new patient with the universe of cancer patients who have come before to discover hidden patterns.
Translation: Watson doesn’t work. But the article finds some sparkles in the mine tailings. Note: A few sparkles.
The print version of the article is titled, “Watson, Heal Thyself.”
The title should be: “IBM: Stick with What Works”. The mainframes are okay. The i2 and Cybertap technology is pretty good.
The Watson thing. Wow, pretty crazy expensive and sufficiently off the rails to motivate IEEE Spectrum to embrace the baloney making methods of the Harvard Business Review.
My take on the essay? IEEE Spectrum is saying, “EEs, don’t do this hyperbole charged approach when pushing your technology toys.” News flash: The EEs will ignore this plea when big money is on the table.
Stephen E Arnold, April 8, 2019
Comments
2 Responses to “IEEE Spectrum Embraces Business Analysis: IBM Watson and Health”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
[…] even points to Watson’s Jeopardy championship as evidence the technology is here to stay. Gee, I wonder why a more recent Watson success story wasn’t cited? And how about those IBM financials? Watson, […]
[…] its troubles, Watson lives on. ZDNet reports, “IBM Updates Watson Studio.” The AI-model-building platform […]