AI Oh-Oh: Innovation Needed Now
December 27, 2024
This blog post is the work of an authentic dinobaby. No smart software was used.
I continue to hear about AI whiz kids “running out of data.” When people and institutions don’t know what’s happening, it is easy to just smash and grab. The copyright litigation and the willingness of AI companies to tie up with content owners make explicit that the zoom zoom days are over.
A smart software wizard is wondering how to get over, under, around, or through the stone wall of exhausted content. Thanks, Grok, good enough.
“The AI Revolution Is Running Out of Data. What Can Researchers Do?” is a less crazy discussion of the addictive craze which has made smart software or — wait for it — agentic intelligence the next big thing. The write up states:
The Internet is a vast ocean of human knowledge, but it isn’t infinite. And artificial intelligence (AI) researchers have nearly sucked it dry.
“Sucked it dry” and the systems still hallucinate. Guard rails prevent users from obtaining information germane to certain government investigations. The image generators refuse to display a classroom of student paying attention to mobile phones, not the teacher. Yep, dry. More like “run aground.”
The fix to running out of data, according to the write up, is:
plans to work around it, including generating new data and finding unconventional data sources.
One approach is to “find data.” The write up says:
one option might be to harvest non-public data, such as WhatsApp messages or transcripts of YouTube videos. Although the legality of scraping third-party content in this manner is untested, companies do have access to their own data, and several social-media firms say they use their own material to train their AI models. For example, Meta in Menlo Park, California, says that audio and images collected by its virtual-reality headset Meta Quest are used to train its AI.
And what about this angle?
Another option might be to focus on specialized data sets such as astronomical or genomic data, which are growing rapidly. Fei-Fei Li, a prominent AI researcher at Stanford University in California, has publicly backed this strategy. She said at a Bloomberg technology summit in May that worries about data running out take too narrow a view of what constitutes data, given the untapped information available across fields such as health care, the environment and education.
If you want more of these work arounds, please, consult the Nature article.
Several observations are warranted:
First, the current AI “revolution” is the result of many years of research and experimentation, The fact that today’s AI produces reasonably good high school essays and allows people to interact with a search system is a step forward. However, like most search-based innovations, the systems have flaws.
Second, the use of neural networks and the creation by Google (allegedly) of the transformer has provided fuel to fire the engines of investment. The money machines are chasing the next big thing. The problem is that the costs are now becoming evident. It is tough to hide the demand for electric power. (Hey, no problem how about a modular thorium reactor. Yeah, just pick one up at Home Depot. The small nukes are next to the Honda generators.) There is the need for computation. Google can talk about quantum supremacy, but good old fashioned architecture is making Nvidia a big dog in AI. And the cost of people? It is off the chart. Forget those coding boot camps and learn to do matrix math in your head.
Third, the real world applications like those Apple is known for don’t work very well. After vaporware time, Apple is pushing OpenAI to iPhone users. Will Siri actually work? Apple cannot afford to whiff to many big plays. Do you wear your Apple headset or do you have warm and fuzzies for the 2024 Mac Mini which is a heck of a lot cheaper than some of the high power Macs from a year ago? What about Copilot in Notebook. Hey, that’s helpful to some Notepad users. How many? Well, that’s another question. How many people want smart software doing the Clippy thing with every click?
Net net: It is now time for innovation, not marketing. Which of the Big Dog AI outfits will break through the stone walls? The bigger question is, “What if it is an innovator in China?” Impossible, right?
Stephen E Arnold, December 27, 2024
Comments
Got something to say?
-
- Subscribe to Beyond Search
Feature archive
News archive
-
