Some AI Wisdom: Is There a T Shirt?
January 14, 2025
Prepared by a still-alive dinobaby.
I was catching up with newsfeeds and busy filtering the human output from the smart software spam-arator. I spotted “The Serious Science of Trolling LLMs,” published in the summer of 2024. The article explains that value can be derived from testing large language models like ChatGPT, Gemini, and others with prompts to force the software to generate something really stupid, off base, incorrect, or goofy. I zipped through the write up and found it interesting. Then I came upon this passage:
the LLM business is to some extent predicated on deception; we are not supposed to know where the magic ends and where cheap tricks begin. The vendors’ hope is that with time, we will reach full human-LLM parity; and until then, it’s OK to fudge it a bit. From this perspective, the viral examples that make it patently clear that the models don’t reason like humans are not just PR annoyances; they are a threat to product strategy.
Several observations:
- Progress from my point of view with smart software seems to have slowed. The reason may be that free and low cost services cannot affords to provide the functionality they did before someone figured out the cost per query. The bean counters spoke and “quality” went out the window.
- The gap between what the marketers say and what the systems do is getting wider. Sorry, AI wizards, the systems typically fail to produce an output satisfactory for my purposes on the first try. Multiple prompts are required. Again a cost cutting move in my opinion.
- Made up information or dead wrong information is becoming more evident. My hunch is that the consequence of ingesting content produced by AI is degrading the value of the models originally trained on human generated content. I think this is called garbage in — garbage out.
Net net: Which of the deep pocket people will be the first to step back from smart software built upon systems that consume billions of dollars the way my French bulldog eats doggie treats? The Chinese system Deepseek’s marketing essentially says, “Yo, we built this LLM at a fraction of the cost of the spendthrifts at Google, Microsoft, and OpenAI. Are the Chinese AI wizards dragging a red herring around the AI forest?
To go back to the Lcamtuf essay, “it’s OK to fudge a bit.” Nope, it is mandatory to fudge a lot.
Stephen E Arnold, January 14, 2025
Comments
Got something to say?
-
- Subscribe to Beyond Search
Feature archive
News archive
-
