Smart Software: Reproducibility Is Not Part of the Game Plan, Thank You
March 24, 2023
Note: The essay below has been crafted by a real, still-alive dinobaby. No smart software required, thank you.
I love it when outfits suggest one thing and do another. I was tempted to write about some companies’ enthusiastic support for saving whales and their even more intense interest in blocking the ban on “forever chemicals.” But whales are one thing and smart software is another.
Specifically, the once open OpenAI is allegedly embracing the proprietary and trade secret approach to technology. “OpenAI’s Policies hinder Reproducible Research on Language Models” reports:
On Monday [March 20. 2023], OpenAI announced that it would discontinue support for Codex by Thursday. Hundreds of academic papers would no longer be reproducible: independent researchers would not be able to assess their validity and build on their results. And developers building applications using OpenAI’s models wouldn’t be able to ensure their applications continue working as expected.
The article elaborates on this main idea.
Several points:
- Reproducibility means that specific recipes have to be known and then tested. Who in Silicon Valley wants this “knowledge seeping” to take place when demand for the know how is — as some may say — doing the hockey stick chart thing.
- Good intentions are secondary to money, power, and control. The person or persons who set thresholds, design filters, orchestrate what content and when is fed into a smart system, and similar useful things want their fingers on the buttons. Outsiders in academe or another outfit eager to pirate expertise? Nope.
- Reproducibility creates opportunities for those not in the leadership outfit to find themselves criticized for bias, manipulation, and propagandizing. Who wants that other than a public relations firm?
Net net: One cannot reproduce much flowing from today’s esteemed research outfits. Should I mention the president of Stanford University as the poster person for intellectual pogo stick hopping? Oh, I just did.
Stephen E Arnold, March 24, 2023
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
