AI Has Become Better at Predicting Your Actions Than You Are
January 1, 2018
It’s official, AI has become smarter than us. Well, maybe. It certainly is more sophisticated about human patterns than we ourselves are. We learned just how advanced in a recent Phys.org article, “Can Math Predict What You’ll Do Next?”
According to the piece:
When making predictions, scientists have historically been limited by a lack of complete data, relying instead on small samples to infer characteristics of a wider population.
But in recent years, computational power and methods of collecting data have advanced to the point of creating a new field: big data. Thanks to the huge availability of collected data, scientists can examine empirical relationships between a wide variety of variables to decipher the signal from the noise.
For example, Amazon uses predictive analytics to guess which books we may like based on our prior browsing or purchase history. Similarly, automated online advertisement campaigns tell us which vehicles we may be interested in based on vehicles sought out the day before.
Not convinced? Consider this story about how Carnegie Mellon’s AI recently won a Texas Hold ‘em Tournament. Poker, of course, is based on subtle human cues, bluffing, and psychology. So, if an AI system is on target there, imagine what it would do if the attention was focused on us?
Patrick Roland, January 1, 2018
Watson and CDC Research Blockchain
December 29, 2017
Oh, Watson! What will IBM have you do next? Apparently, you will team up with the Centers for Disease Control and Prevention to research blockchain benefits. The details about Watson’s newest career are detailed in Fast Company’s article, “IBM Watson Health Team With the CDC To Research Blockchain.” Teaming up with the CDC is an extension of the work IBM Watson is already doing with the Food and Drug Administration by exploring owned-mediated data exchange with blockchain.
IBM chief science officer Shahram Ebadollahi explained that the research with the CDC and FDA with lead to blockchain adoption at the federal government level. By using blockchain, the CDC hopes to discover new ways to use data and expedite federal reactions to health threats.
Blockchain is a very new technology developed to handle sensitive data and cryptocurrency transactions. It is used for applications that require high levels of security. Ebadollahi said:
‘Blockchain is very useful when there are so many actors in the system,’ Ebadollahi said. ‘It enables the ecosystem of data in healthcare to have more fluidity, and AI allows us to extract insights from the data. Everybody talks about Big Data in healthcare but I think the more important thing is Long Data.’
One possible result is that consumers will purchase a personal health care system like a home security system. Blockchain could potentially offer a new level of security that everyone from patients to physicians is comfortable with.
Blockchain is basically big data, except it is a more specific data type. The applications are the same and it will revolutionize the world just like big data.
Whitney Grace, December 29, 2017
Turning to AI for Better Data Hygiene
December 28, 2017
Most big data is flawed in some way, because humans are imperfect beings. That is the premise behind ZDNet’s article, “The Great Data Science Hope: Machine Learning Can Cure Your Terrible Data Hygiene.” Editor-in-Chief Larry Dignan explains:
The reality is enterprises haven’t been creating data dictionaries, meta data and clean information for years. Sure, this data hygiene effort may have improved a bit, but let’s get real: Humans aren’t up for the job and never have been. ZDNet’s Andrew Brust put it succinctly: Humans aren’t meticulous enough. And without clean data, a data scientist can’t create algorithms or a model for analytics.
Luckily, technology vendors have a magic elixir to sell you…again. The latest concept is to create an abstraction layer that can manage your data, bring analytics to the masses and use machine learning to make predictions and create business value. And the grand setup for this analytics nirvana is to use machine learning to do all the work that enterprises have neglected.
I know you’ve heard this before. The last magic box was the data lake where you’d throw in all of your information–structured and unstructured–and then use a Hadoop cluster and a few other technologies to make sense of it all. Before big data, the data warehouse was going to give you insights and solve all your problems along with business intelligence and enterprise resource planning. But without data hygiene in the first place enterprises replicated a familiar, but failed strategy: Poop in. Poop out.
What the observation lacks in eloquence it makes up for in insight—the whole data-lake concept was flawed from the start since it did not give adequate attention to data preparation. Dignan cites IBM’s Watson Data Platform as an example of the new machine-learning-based cleanup tools, and points to other noteworthy vendors investigating similar ideas—Alation, Io-Tahoe, Cloudera, and HortonWorks. Which cleaning tool will perform best remains to be seen, but Dignan seems sure of one thing—the data that enterprises have been diligently collecting for the last several years is as dirty as a dustbin lid.
Cynthia Murrell, December 28, 2017
Big Data Used to Confirm Bad Science
November 30, 2017
I had thought we had moved beyond harnessing big data and were now focusing on AI and machine learning, but Forbes has some possible new insights in, “Big Data: Insights Or Illusions?”
Big data is a tool that can generate new business insights or it can reinforce a company’s negative aspects. The article consists of an interview with Christian Madsbjerg of ReD Associates. It opens with how Madsbjerg and his colleagues studied credit card fraud by living like a fraudster for a while. They learned some tricks and called their experience contextual analytics. This leads to an important discussion topic:
Dryburgh: This is really interesting, because it seems to me that big data could be a very two-edged sword. On the one hand you can use it in the way that you’ve described to validate hypotheses that you’ve arrived at by very subjective, qualitative means. I guess the other alternative is that you can use it simply to provide confirmation for what you already think.
Madsbjerg: Which is what’s happening, and with the ethos that we’ve got a truth machine that you can’t challenge because it’s big data. So you’ll cement and intensify the toxic assumptions you have in the company if you don’t use it to challenge and explore, rather than to confirm things you already know.
This topic is not new. We are seeing unverified news stories reach airwaves and circulate the Internet for the pure sake of generating views and profit. Corporate entities do the same when they want to churn more money into their coffers than think of their workers or their actual customers. It is also like Hollywood executives making superhero movies based on comic heroes when they have no idea about the medium’s integrity.
In other words, do not forget context and the human factor!
Whitney Grace, November 30, 2017
The Thing Holding AI Back Is the Thing It Needs Most, Data
November 30, 2017
Here’s an interesting problem: for artificial intelligence and machine learning to thrive, it needs a massive amount of information. However, they need so much data that it causes hiccups in the system. Google has a really interesting solution to this problem, as we learned in the Reuter’s article, “Google’s Hinton Outlines New AI Advance That Requires Less Data.”
The bundling of neurons working together to determine both whether a feature is present and its characteristics also means the system should require less data to make its predictions.
The leader of Google Brain said, “The hope is that maybe we might require less data to learn good classifiers of objects, because they have this ability of generalizing to unseen perspectives or configurations of images.
Less data for big data? It’s just crazy enough to work. In fact, some of the brightest minds in the business are trying to, as ComputerWorld said, “do less with more.” The piece focuses on Fuzzy LogiX and their attempts to do exactly what Google is hypothetically saying. It will be interesting to see what happens, but we are betting on technology cracking this nut.
Patrick Roland, November 30, 2017
The Worlds Wealthiest People Should Fear Big Data
November 24, 2017
One of the strengths that the planets elite and wealthy have is secrecy. In most cases, average folks and media don’t know where big money is stored or how it is acquired. However, that recently changed for The Queen of England, several Trump cabinet members, and other powerful men and women. And they should be afraid of what big data and search can do with their info, as we learned in the Guardian’s piece, “Paradise Papers Leak Reveals Secrets of the World’s Elite Hidden Wealth.”
The story found a lot of fishy dealings with political donors and those in power, Queen Elizabeth having tax-free money in the Caymans and more. According to the story:
At the centre of the leak is Appleby, a law firm with outposts in Bermuda, the Cayman Islands, the British Virgin Islands, the Isle of Man, Jersey and Guernsey. In contrast to Mossack Fonseca, the discredited firm at the centre of last year’s Panama Papers investigation, Appleby prides itself on being a leading member of the “magic circle” of top-ranking offshore service providers.
Appleby says it has investigated all the allegations, and found “there is no evidence of any wrongdoing, either on the part of ourselves or our clients”, adding: “We are a law firm which advises clients on legitimate and lawful ways to conduct their business. We do not tolerate illegal behaviour.
Makes you wonder what would happen if some of the brightest minds in search and big data got ahold of this information? We suspect a lot of the financial knots this money ties to keep itself concealed would untangle. In an age of increasing transparency, we wouldn’t be shocked to see that happen.
Patrick Roland, November 24, 2017
Spark: An Easy Way to Burn Through Big Data?
November 14, 2017
I read “What is Apache Spark? The Big Data Analytics Platform Explained.” Interesting approach. The publishing outfit IDC seized upon the idea that the Wikipedia entry for Spark was not making the open source project easy enough to understand. I know that Wikipedia is chock full of craziness, but the Spark write up in the free encyclopedia struck me as reasonably good as far as Wikipedia content goes. There are code samples, links, and statements which balance the wonderfulness of open source with the grim realities of fiddling with the goodies the community provides. If I were a college professor (which I most certainly am not!), I would caution my students about applying the tenants of recycling to their class assignments. Apparently the old fashioned ideas I have are irrelevant.
Let’s look at three points from the IDC “explainer” that I found intriguing:
Apache Spark is the leading platform for large-scale SQL, batch processing, stream processing, and machine learning
The statement seems to be factual. I would ask, from my shack in rural Kentucky, what is the source of data backing up this claim. I hate to rain on everyone’s parade, but I was under the impression that the numero uno tool for wrestling with data was Excel. There are some software solutions which are popular among the crunching crowd; for example, the much loved SAS and SPSS systems. And there are others. Many others.
A second interesting statement warranted a blue circle on my printed copy of the article:
The second advantage is the developer-friendly Spark API. As important as Spark’s speed-up is, one could argue that the friendliness of the Spark API is even more important.
If I understand the title, the write up is about making Spark easy. The explanation of “easy” is to use the “developer friendly Spark AI.” Easy means friendly. Hmmm.
The third statement I noted was:
By providing bindings to popular languages for data analysis like Python and R, as well as the more enterprise-friendly Java and Scala, Apache Spark allows everybody from application developers to data scientists to harness its scalability and speed in an accessible manner.
It seems that “easy” means that one needs knowledge of specific programming languages. Yep, easy. For “everybody” too.
What a simple thing is Spark! I will stick with Wikipedia. Maybe IDC should too?
Stephen E Arnold, November 14, 2017
Big Data Less Accessible for Small and Mid-Size Businesses
October 31, 2017
Even as the term “Big Data” grows stale, small and medium-sized businesses (SMB’s) are being left behind in today’s data-driven business world. The SmartData Collective examines the issue in, “Is Complexity Strangling the Real-World Benefits of Big Data for SMB’s?” Writer Rehan Ijaz supplies this example:
Imagine a local restaurant chain fighting to keep the doors open as a national competitor moves into town. The national competitor will already have a competent Cloud Data Manager (CDM) in place to provide insight into what should be offered to customers, based on their past interactions. A multi-million-dollar technology is affordable, due to scale, for a national chain. The same can’t be said for a smaller, mom and pop type restaurant. They’ve relied on their gut instinct and hometown roots to get them this far, but it may not be enough in the age of Big Data. Large companies are using their financial muscle to get information from large data sets, and take targeted action to outmaneuver local competitors.
Pointing to an article from Forbes, Ijaz observes that the main barrier for these more modestly-sized enterprises is not any hesitation about the technology itself, but rather a personal issue—their existing marketing employees were not hired for their IT prowess, and even the most valuable data requires analysis to be useful. Few SMB’s are eager to embrace the cost and disruption of hiring data scientists and reorganizing their marketing teams; they have to be sure it will be worth the trouble.
Ijaz hopes that the recent increase in scalable, cloud-based analysis solutions will help SMB’s with these challenges. The question is, he notes, whether it is too late for many SMB’s to recover from their late foray into Big Data.
Cynthia Murrell, October 31, 2017
A Handy Collection of References on AI Topics
October 24, 2017
Ever wish there were a centralized resource with all you need to know about AI, clearly presented? If so, check out the “Cheat Sheets for AI, Neural Networks, Machine Learning, Deep Learning & Big Data” at Becoming Human. Chatbot pro-Stefan Kojouharov shares his selections of graphic aids and includes a summary list of links at the end. He briefly introduces his assemblage:
Over the past few months, I have been collecting AI cheat sheets. From time to time I share them with friends and colleagues and recently I have been getting asked a lot, so I decided to organize and share the entire collection. To make things more interesting and give context, I added descriptions and/or excerpts for each major topic. This is the most complete list and the Big-O is at the very end, enjoy…
The offerings begin with illustrations of neural networks and machine learning in general, then progress to detailed information on relevant software, like Python for Data Science and TensorFlow, and topics like data wrangling and data visualization. As promised, graphics on Big-O notation conclude the presentation. This is a page to bookmark; it could save some time hunting for the right resource down the line, if not today.
Cynthia Murrell, October 24, 2017
Big Data Might Just Help You See Through Walls
October 18, 2017
It might sound like science fiction or, worse, like a waste of time, but scientists are developing cameras that can see around corners. More importantly, these visual aids will fill in our human blind spots. According to an article in MIT News, “An Algorithm For Your Blind Spot,” it may have a lot of uses, but needs some serious help from big data and search.
According to the piece about the algorithm, “CornerCameras,”
CornerCameras generates one-dimensional images of the hidden scene. A single image isn’t particularly useful since it contains a fair amount of “noisy” data. But by observing the scene over several seconds and stitching together dozens of distinct images, the system can distinguish distinct objects in motion and determine their speed and trajectory.
Seems like a pretty neat tool. Especially, when you consider that this algorithm could help firefighters find people in burning buildings or help bus drivers spot a child running onto the street. However, it is far from perfect.
The system still has some limitations. For obvious reasons, it doesn’t work if there’s no light in the scene, and can have issues if there’s low light in the hidden scene itself. It also can get tripped up if light conditions change, like if the scene is outdoors and clouds are constantly moving across the sun. With smartphone-quality cameras the signal also gets weaker as you get farther away from the corner.
Seems like they have a brilliant idea in need of a big data boost. We can envision a world where these folks partner with big data and search giants to help fill in the gaps of the algorithm and provide a powerful tool that can save lives. Here’s to hoping we’re not the only ones making that connection.
Patrick Roland, October 18, 2017
-
- Subscribe to Beyond Search
Feature archive
News archive
-
