The Old Bayesian Recipe: Burning the Predictive Reality Cupcakes

June 7, 2013

I don’t have any comment about the alleged surveillance conducted by governments or the comments of giant online vendors alleged interactions with governments. I will leave the subject and speculations to those younger than I and possibly — just possibly — less well informed folks.

I do want to call attention to the write up “How Likely Is the NSA PRISM Program to Catch a Terrorist?” The source is the Bayesian Biologist. I know less about PRISM, biology, and Bayes than my neighbors here in Harrod’s Creek, Kentucky.

Here’s the snippet I noted in the “How Likely” story:

for every positive (the NSA calls these ‘reports’) there is only a 1 in 10,102 chance (using our rough assumptions) that they’ve found a real bad guy.Big brother is always watching, but he’s still got a needle in a haystack problem.

I think that there might be some fascinating marketing hype, fear, and salami in the digital blender.

In my most recent lecture about Big Data and the limitations of today’s software:

Collecting is one thing. Finding is another.

Search, content processing, and analytics work well in certain circumstances; for example, trimmed data sets which match the textbook checklists for valid inputs and when key facts are known such as the name and aliases of an entity.

Today’s systems — no matter what the marketers say — have been designed to work within some constraints. Marketers and fear mongers don’t have to cope with computational realities.

Stephen E Arnold, June 7, 2013


Comments are closed.

  • Archives

  • Recent Posts

  • Meta