IEEE Spectrum Embraces Business Analysis: IBM Watson and Health

April 8, 2019

I spotted a link to “How IBM Watson Overpromised and Under delivered on AI Healthcare.” I read the article and found it reasonably balanced. What surprised me was the fact that the editors of IEEE Spectrum believed that this particularly collection of information should be published for the magazine’s and online audience. My suspicion is that IBM was promoting its technology in a manner that was egregious. IEEE was reminding its readers about veering from technical facts into the wild and crazy world of toothpaste and dandruff shampoo marketing. Then I realized the IEEE Spectrum was explaining an example digital snake oil:

image

I circled in Big Blue marker this passage:

Outside of corporate headquarters, however, IBM has discovered that its powerful technology is no match for the messy reality of today’s health care system. And in trying to apply Watson to cancer treatment, one of medicine’s biggest challenges, IBM encountered a fundamental mismatch between the way machines learn and the way doctors work.

Translation: Reality is different from a demo. When demos are built on software which has proven problematic for decades, one wonders how the square peg in the round hole gets funded.

I circled this statement:

… Even today’s best AI struggles to make sense of complex medical information. And encoding a human doctor’s expertise in software turns out to be a very tricky proposition. IBM has learned these painful lessons in the marketplace, as the world watched. While the company isn’t giving up on its moon shot, its launch failures have shown technologists and physicians alike just how difficult it is to build an AI doctor.

IEEE Spectrum does not use the word “desperation” but it applies. The reality, from my point of view, is that finding information and answering questions is difficult. Google pulls off a version of question answering by hooking relevance to behavior and possibly relevant advertisements. Precision and recall are not part of Google or other commercial search vendors’ vocabulary today.

But answering questions doesn’t work all that well today. Sorry Google.

“Regular” search— particularly search based on open source software, some home brew code, and acquired technology — is difficult to make work across different types of content and use cases. The dust up between HP and Autonomy is one example of what happens when “logical” explanations don’t apply to search and retrieval. There are other examples too. Just ask a Fast Search & Transfer executive who skirted serious jail time.

IEEE Spectrum’s article drives home failure this way:

In a final blow to the dream of an AI super doctor, researchers realized that Watson can’t compare a new patient with the universe of cancer patients who have come before to discover hidden patterns.

Translation: Watson doesn’t work. But the article finds some sparkles in the mine tailings. Note: A few sparkles.

The print version of  the article is titled, “Watson, Heal Thyself.”

The title should be: “IBM: Stick with What Works”. The mainframes are okay. The i2 and Cybertap technology is pretty good.

The Watson thing. Wow, pretty crazy expensive and sufficiently off the rails to motivate IEEE Spectrum to embrace the baloney making methods of the Harvard Business Review.

My take on the essay? IEEE Spectrum is saying, “EEs, don’t do this hyperbole charged approach when pushing your technology toys.” News flash: The EEs will ignore this plea when big money is on the table.

Stephen E Arnold, April 8, 2019

IBM and Oldsters

March 29, 2019

I can hear the question posed to IBM Watson now, “Watson, is it okay to fire older employees in order the make room for younger, less expensive workers?”

I even can anticipate the IBM Watson answer, “Yes.”

IBM Watson is smart software, but it does not do as well providing human resource outputs as it does with generating recipes which require tamarind.

How do I know?

I read “IBM Sued By Former Employees For Alleged Illegal Firing.” I learned from the article:

IBM is being sued by a group of its former employees for allegedly laying them off for their age.

The write up added what seems obvious to a human like me but probably a nuance unnoticed by IBM’s Watson:

The lawyers of the complainants added that their main case against IBM would be a major age-discrimination lawsuit. They said that top executives of the company “took the calculated risk of openly breaking the law” in order to cover up substantial, targeted layoffs of its older workers.

IBM Watson may need a bit more training, particularly information related to employment laws and regulations.

Stephen E Arnold, March 29, 2019

RedMonk and Its Assessment of IBM as an Open Source Leader

March 24, 2019

I read “The RedMonk Programming Language Rankings: January 2019.” The analysis was interesting and contained one remarkable assertion and one probably understandable omission. The guts of the report boiled down, in my opinion, to a reminder to job hunters. If you want to increase your chances of getting hired, know:

1 JavaScript
2 Java
3 Python
4 PHP
5 C#

But the surprising statement in the write up was this one:

IBM remains at the forefront of open source innovation.

Now the omission. If IBM is in the forefront, where is Amazon? The company has made an effort to support most of the widely used open source software. Plus, the company appears to be taking tactical steps to close or capture open source.

From my vantage point, Amazon is taking a more “innovative” approach to open source. Granted Amazon’s “approach” may be a milestone in the company’s enhanced walled garden approach to core software systems. IBM’s approach seems little more than Big Blue’s attempt to give back and convince the open source community that it is not the IBM of its mainframe heritage.

Stephen E Arnold, March 24, 2019

New CIA Chief Information Officer: Watson, Who Is It?

March 19, 2019

The answer comes not from IBM Watson. “CIA Announces New Chief Information Officer” reveals that Juliane Gallina, an IBM professional has landed the job. DarkCyber finds this interesting for three reasons.

image

Aurora, which means dawn and $500 million for one system, may be a new technology the CIA explores.

First, Amazon’s policeware found some traction in that government agency. IBM covets US government work. Amazon may find that Gallina may ask different questions in her tenure.

Second, IBM Federal Systems is the poster child for old-school government contracting. The idea within some sectors of the US government is to find a new-school approach. Gallina may have some interesting ideas about how next-generation systems are selected, shaken down, and made operational.

Third, Gallina has intelligence sector experience. Presumably that experience will make it easier to determine which units can best be served by specific technologies. Will that insight match the diverse community of interests within the CIA?

The appointment is going be one closely watched by those within and outside the Beltway. Perhaps there will be a new technology dawn at the agency. Aurora, it’s called.

Stephen E Arnold, March 19, 2019

IBM and Its Farm Team Model

March 6, 2019

IBM and SUNY Poly have pooled their talents to create an innovating institute: a brand new artificial intelligence lab in Albany, New York. Times Union has the details in the article, “IBM, SUNY Poly Creating An Artificial Intelligence Center In Albany.” The new AI lab is only one part of a $2 billion commitment IBM has made to New York University.

The deal outlines that Empire State Development will provide SUNY with a $300 million grant over five years for the AI Hardware Center. As a trade, IBM will remain at SUNY Poly’s Center for Semiconductor Research until 2023. IBM’s spending over the period will amount to $2 billion and will make New York one of the top places for AI.

We learned:

“ ‘New York has always been at the forefront of emerging industries, and this private sector investment to create a hub for artificial intelligence research will attract world-class minds and drive economic growth in the region,’ Gov. Andrew M. Cuomo said in a statement. “Artificial intelligence has the potential to transform how we live and how businesses operate, and this partnership with IBM will help New York stay on the cutting edge developing innovative technologies.’”

The AI Hardware Center is only one of many AI institutes, among them are Rensselaer Polytechnic Institute, Applied Materials, and Tokyo Electron. IBM will also spend $30 million on AI programs at SUNY schools, while SUNY will contribute $25 million as well. SUNY wants all of its schools to have closer collaboration and it will be part of a new education model.

Is this a “farm to table” approach to innovation? Watson, what’s the answer?

Whitney Grace, March 6, 2019

IBM: Excited about the Press Coverage of Think

March 5, 2019

Interesting back patting in this IBM publicity about IBM getting publicity. You can find the happy happy information in “How Press Reacted to the Data and AI News from Think 2019.” DarkCyber was disappointed in the coverage of the Watson vs human debate. Unlike Jeopardy, post production was not available. The human judges decided the human beat IBM Watson.

However, DarkCyber provided an analysis of the debate. The human judges were like the three stooges. The debate should have been judged by artificial intelligence systems from Google, Amazon, and Microsoft. The final tally would have fallen to Facebook’s system.

If you missed our analysis, you can find it at this link.

Stephen E Arnold, March 5, 2019

Oldster Teaches Young Dogs Some Tricks

March 1, 2019

It can be easy to forget just how long IBM has been around compared to other huge tech companies, but that venerable giant was incorporated in 1911. An article at The Conversation examines “Lessons from IBM for Google, Amazon and Facebook.” Writer and former IBM employee James Cortada, author of the recently published book, IBM: The Rise and Fall and Reinvention of a Global Icon, shares his observations. He observes:

“There is a difference between individual products – successive models of PCs or typewriters – and the underlying technologies that make them work. Over 130 years, IBM released well over 3,600 hardware products and nearly a similar amount of software. But all those items and services were based on just a handful of real technological advances, such as shifting from mechanical machines to those that relied on computer chips and software, and later to networks like the internet. The transitions between those advances took place far more slowly than the steady stream of new products might suggest. These transitions from the mechanical, to the digital, and now to the networked reflected an ever-growing ability to collect and use greater amounts of information easily and quickly. IBM moved from manipulating statistical data to using technologies that teach themselves what people want and are interested in seeing.”

The write-up goes into more depth on the progression of IBM advances, emphasizing that the company’s success comes more from developing technologies over time than from sudden breakthroughs. Cortada notes that, unlike IBM, Microsoft, and Apple, Amazon, Google, and Facebook have yet to evolve away from their original functions. Those internet-born companies, he advises, can last out the century if, and only if, they adapt to evolving technologies as IBM has done.

Cynthia Murrell, March 1, 2019

Watson Weakly: Recruitment the Smart Way

February 26, 2019

IBM is working overtime to become the cloud alternative to Amazon. IBM Watson is back to recipes, health care, and background noise. IBM, however, knows how to capture the attention of the DarkCyber and Beyond Search team in rural Kentucky.

We noted an article in the Register, an online publication, with the interesting title “IBM So Very, Very Sorry after Jobs Page Casually Asks Hopefuls: Are You White, Black… or Yellow?”

The Register asserts:

IBM has apologized after its recruitment web pages asked applicants whether their ethnicity was, among other options, the racial slurs Yellow and Mulatto.

The article describes the wording as a “baffling error.” My hunch is that either IBM Watson or one of his acolytes consumed outputs with the diligence once expects of millennials and smart software, possibly working in tandem.

An IBM professional is quoted as telling the Register:

“Those questions were removed immediately when we became aware of the issue and we apologize. IBM hiring is based on skills and qualifications. We do not use race or ethnicity in the hiring process and any responses we received to those questions will be deleted. IBM has long rejected all forms of racial discrimination and we are taking appropriate steps to make sure this does not happen again.”

Watson? What about that cancer diagnosis? What about inappropriate questions? What about those old people who used to work in personnel?

Stephen E Arnold, February 26, 2019

IBM Debate Contest: Human Judges Are Unintelligent

February 12, 2019

I was a high school debater. I was a college debater. I did extemp. I did an event called readings. I won many cheesey medals and trophies. Also, I have a number of recollections about judges who shafted me and my team mate or just hapless, young me.

I learned:

Human judges mean human biases.

When I learned that the audience voted a human the victor over the Jeopardy-winning, subject matter expert sucking, and recipe writing IBM Watson, I knew the human penchant for distortion, prejudice, and foul play made an objective, scientific assessment impossible.

ibm debate

Humans may not be qualified to judge state of the art artificial intelligence from sophisticated organizations like IBM.

The rundown and the video of the 25 minute travesty is on display via Engadget with a non argumentative explanation in words in the write up “IBM AI Fails to Beat Human Debating Champion.” The real news report asserts:

The face-off was the latest event in IBM’s “grand challenge” series pitting humans against its intelligent machines. In 1996, its computer system beat chess grandmaster Garry Kasparov, though the Russian later accused the IBM team of cheating, something that the company denies to this day — he later retracted some of his allegations. Then, in 2011, its Watson supercomputer trounced two record-winning Jeopardy! contestants.

Yes, past victories.

Now what about the debate and human judges.

My thought is that the dust up should have been judged by a panel of digital devastators; specifically:

  • Google DeepMind. DeepMind trashed a human Go player and understands the problems humanoids have being smart and proud
  • Amazon SageMaker. This is a system tuned with work for a certain three letter agency and, therefore, has a Deep Lens eye to spot the truth
  • Microsoft Brainwave (remember that?). This is a system which was the first hardware accelerated model to make Clippy the most intelligent “bot” on the planet. Clippy, come back.

Here’s how this judging should have worked.

  1. Each system “learns” what it takes to win a debate, including voice tone, rapport with the judges and audience, and physical gestures (presence)
  2. Each system processes the video, audio, and sentiment expressed when the people in attendance clap, whistle, laugh, sub vocalize “What a load of horse feathers,” etc.
  3. Each system generates a score with 0.000001 the low and 0.999999 the high
  4. The final tally would be calculated by Facebook FAIR (Facebook AI Research). The reason? Facebook is among the most trusted, socially responsible smart software companies.

The notion of a human judging a machine is what I call “deep stupid.” I am working on a short post about this important idea.

A human judged by humans is neither just nor impartial. Not Facebook FAIR.

An also participated award goes to IBM marketing.

participant meda

IBM snagged an also participated medal. Well done.

Stephen E Arnold, February 13, 2019

With Q, Whither Watson?

January 20, 2019

It was a trying summer for IBM’s Watson as its Oncology software received some harsh criticism. Will new leadership improve the program? We learn IBM replaced Watson Health’s former head, Deborah DiSanzo, with John Kelly, previously their senior V.P. of Cognitive Solutions and Research, from Computerworld’s article, “Did IBM Overhype Watson Health’s AI Promise?” We would say the answer to that rhetorical question is, you bet!

Writer Lucas Mearian describes the troublesome July report published in Stat News. That report revealed that Watson Oncology, which is being used in several real-life healthcare facilities, had recommended “unsafe and incorrect” treatments for hypothetical patients. The Computerworld article touches on the company’s defense (or denial, depending on who you listen to), as well as covering some early problems that plagued the program. See the piece for those details. We are reminded that it takes time to fully train machine learning software, and told the AI has simply not had enough time or quality data to meet the hype generated by the sales team. Not yet.

The article closes with a look ahead, citing IBM’s relatively recent purchases of healthcare data analytics-firm Explorys, patient communications company Phytel, and Truven Health Analytics. Quoting Cynthia Burghard of IDC Health Insights, Mearian writes:

Upon completing all three acquisitions, IBM boasted its Watson Health Cloud housed “one of the world’s largest and most diverse collections of health-related data, representing an aggregate of approximately 300 million patient lives acquired from three companies. They all in their own right, before they were acquired, were very successful companies and had good, strong, loyal client bases and were plugging along.”… In late October, IBM announced plans to seed its new hybrid cloud model for Watson by first moving data from insurance payer systems. For that, Truven will be key. Once payer data is moved to the hybrid cloud, the electronic medical records (EMRs) acquired through the Explorys acquisition will follow, Kelly said.

IBM is not the first company to have its sales team outrun its developers, with an IBM quantum computer ready for prime time, what happens when one combines the two?

One possible answer is more marketing.

Cynthia Murrell, January 20, 2019

« Previous PageNext Page »

  • Archives

  • Recent Posts

  • Meta