Cheap Training for Machine Learning Is Not Hyped Enough. Believe It or Not
December 6, 2022
I read an interesting article titled “Counting the Cost of Training Large Language Models.” The write up contains a statement which provides insight into the type of blind spots that plague whiz bang smart software companies. Here’s the statement which struck me as amusing and revelatory:
It has been becoming increasingly clear – anecdotally at least – just how expensive it is to train large language models and recommender systems…
Two points. Anyone who took the time to ask about the cost of retraining a Bayesian and neurolinguistic system from the late 1990s would have learned: [a] Smart software, even relatively simple implementations, require refined and curated training data before a system is deployed. This work is tedious and requires subject matter specialists. Then there is testing and fiddling knobs and dials before the software becomes operational. [b] The smart software requires retraining with updated data sets, calibration, and testing on a heartbeat. For some Autonomy plc type systems, the retraining could be necessary every 180 days or when “drift” became evident. Users complain, and that’s how one knows the system is lost in the tiny nooks and crannies of lots of infinitesimals adding up to a dust pile in a dark corner of a complex system.
After three decades of information available about the costs of human centric involvement in making smart software less stupid, one would think that the whiz kids would have done some homework. Oh, right. If the information is not in the first 15 items in a Google search result, there are no data. Very modern.
The write up identifies a number of companies with ways to chop down training costs. To be clear, the driving idea for Snorkel from the Stanford AI Lab is reducing the costs of building training sets. The goal is to be “close enough for horseshoes” or “good enough.” Cut the costs and deal with issues with some software wrappers. Package up the good enough training data and one has a way to corner the market for certain ML applications. But it’s not just the Google. Amazon AWS is in the hunt for this off-the-shelf approach to machine learning. I think of it as the 7-11 approach to getting a meal: Cheap, quick, and followed by a Big Gulp.
The write up has a number of charts. These are okay, but I am not sure about the provenance of the data presented. But that’s just my skepticism for content marketing type write ups. There are even “cost per one million parameters” data. Interesting but who compiled the data, what methods were used to generate the numbers, and who vetted the project itself? Annoying questions? Sure. Important? Not to true believers.
But I know the well educated, well informed funding sources and procurement officials will love this conclusion:
some people will rent to train, and then as they need to train more and also train larger models, the economics will compel them to buy.
Yep, but what about the issue of “close enough for horseshoes”? Yep, here’s another annoying question: Is this article the kick off for another hype campaign? My initial reaction is, “Yes.”
Stephen E Arnold, December 2022
Apple Factoid or Why a US Company Shows Affection for Pandas (Digital and Furry)
December 6, 2022
I spotted an article with a killer title: “Apple Reaches Highest Ever Monthly Market Share in China.” What’s the factoid? The write up provides what may be a semi credible factoid:
One in every four devices sold in China during October 2022 was an iPhone.
Here’s a passage from the write up I found intriguing:
Apple has been reaching new heights in terms of market share in China during the last two years. It reached a record monthly market share in November and December 2020, and in October, November and December 2021. Notably, 2020 was also the year when US sanctions were imposed on Huawei.
The article provides no information about why a US company is thriving in an environment of restrictions on certain Chinese-US interactions. Perhaps there is information to be found, but it is not in reports of what appear to be significant sales by a US firm in the Middle Kingdom.
Stephen E Arnold, December 6, 2022
FTX: What Does B Stand For?
December 2, 2022
I am not a krypto kiddie. After the mysterious Nakamoto white paper became available, I made an informed judgment: Bad actors will love this crypto thing. My hunch was correct. The meltdown of a crypto wizard and his merry band of tea totaling worker bees have demonstrated that cyber fraud can be entertaining.
I read “Does B Stand for Bankman-Fried or Bankruptcy?” The write up asks a simple question. I noted this passage from the “real” Silicon Valley write up:
SBF said FTX failed on risk management and he didn’t “knowingly co-mingle funds.”
There you go.
Now what does B stand for? Here are my suggestions:
bamboozle – to rip off, fool, or deceive
bane – a source or ruin, harm, or evil
baseborn – a nice way to question one’s family position in society
bebotherer – one who brings trouble
besotted – drunk and incoherent
bonkers — a few cans short of a six pack
brock—a nasty, little, furred creature
I am leaning toward bamboozle but I think brock has a certain charm. Perhaps a combo; to wit:
The brock bamboozled himself and others.
Close enough for horseshoes as the “we’re not talking” analytics folks like to say among friends at lunch.
Stephen E Arnold, December 2, 2022
Why Did Smart People Fall For The FTX Scam? Uh, Maybe Greed?
November 29, 2022
When we hear how people fall for scams, we tell ourselves that we are too smart and will never become a victim of one. Despite our intelligence, we all become scam victims at some point in our lives. Hopefully, the aftermath is no more devastating than broken pride and a well-learned lesson. Unfortunately, investors in the FTX crypto currency have lost everything like people in the 1930s Depression. The Guardian expresses why FTX lured so many people: “Why Were So Many Smart People So Dumb About FTX? Did They Seriously Just Like Sam Bankman-Fried’s Vibe?”
Area Mahdawi wrote the editorial about crypto currencies and she immediately rips into Sam Bankman-Fried’s unprofessionalism. The FTX inventor played videogames during business meetings. That does not inspire confidence. Large investors threw money at him and he was described as the “next Warren Buffett” and it was believed he could become the world’s first trillionaire.
Behind the proverbial curtain, Bankman-Fried was pulling a typical scam: shifting funds from FTX to his other company, Alameda Research. He then made risky risky trades and lost billions! His net worth fell from $16.2 billion to $3. Bankman-Fried lived a luxurious life a lá Anna Delvey in the Bahamas with his nine employees and they all had various romantic relationships with each other.
Mahdawi thinks people fell for Bankman-Fried for two reasons:
- They didn’t understand what he was selling, so that meant he was a genius.
- They liked his charisma.
Some investors were impressed that Bankman-Fried played videogames during meetings. Why? Maybe he conveyed an autistic savant vibe that appeared he could crunch the numbers and do magic tricks. Mahdawi said this would not have happened to other people, especially women:
“I don’t know about you, but I’m having one of those want-to-tear-my-hair-out-with-frustration moments right now. Can you imagine a woman playing video games in a meeting and being handed billions by investors? That would never happen. Last year, female founders secured only 2% of venture capital in the US and I’ll bet you everything I have that those founders were as buttoned-up as you can get. I’ll bet you they didn’t get a billion dollars because people “just liked their vibe”.”
She’s right, but also wrong. It depends on the person handing out the loans and the office politics. As to why the people invested their money with the scam artist, they wanted to make more money. Chalk it up to simple greed. Greed is good too.
Whitey Grace, November 30, 2022
The Collision of Nation State Bias and High School Science Club Management
November 28, 2022
CNN offered some interesting pictures of the labor management misunderstanding in Zhengzhou, China. Even though I have been to China several times, I was not sure what made Zengzhou different from other “informed” cities struggling with what may be an ill-advised approach to Covid. In fact, the images of law enforcement and disgruntled individuals are not particularly unique. These images are more interesting when a blurry background of Apple and a Taiwanese company add a touch of chiaroscuro to the scenes.
What is interesting is that “Apple Has a Huge Problem with an iPhone Factory in China” mentions the “Taiwan contract manufacturing firm Foxconn.” CNN, however, does not offer any information about the involvement of individuals who want to create issues for Foxconn. China and Taiwan sort of coexist, but I am not certain that the Chinese provincial government either in Henan or the national government in Beijing are particularly concerned about what happens to either Apple of Foxconn.
The fact that workers suddenly became upset suggests that I have to exercise a willing suspension of disbelief and assume the dust up was spontaneous. Sorry, a “Hey, this just happened because of pay” or some similar dismissive comment won’t make me feel warm and fuzzy.
The write up asserts:
The Zhengzhou campus has been grappling with a Covid outbreak since mid-October that caused panic among its workers. Videos of people leaving Zhengzhou on foot went viral on Chinese social media in early November, forcing Foxconn to step up measures to get its staff back…. But on Tuesday [November 22, 2022] night, hundreds of workers, mostly new hires, began to protest against the terms of the payment packages offered to them and also about their living conditions. Scenes turned increasingly violent into the next day as workers clashed with a large number of security forces. By Wednesday [November 23] evening, the crowds had quieted, with protesters returning to their dormitories on the Foxconn campus after the company offered to pay the newly recruited workers 10,000 yuan ($1,400), or roughly two months of wages, to quit and leave the site altogether.
Seems straightforward. A confluence of issues culminated in a protest.
Now let’s think about the issue this way. These are my working hypotheses.
First, Foxconn may not perceive the complaints of its employees as important. Sure, the factory workers have to do their job, but these are Chinese factory workers. Foxconn has a Taiwan spin. This may translate into Chinese government passivity. Let the Taiwan managers deal with the problems.
Second, Apple is a US outfit and it embraces some of the tenets of the high school science club management method. The kernel of the HSSCMM is that science club members know best. Others do not; therefore, if something is not on the radar of the science club, that “something” is irrelevant, silly, or just plain annoying.
Third, the workers have some awareness of the financial resources of Foxconn and Apple. Thus, like workers from an Apple store to the quiet halls of the Apple core spaceship, money talks.
Fourth, Covid. Yep, not going away it seems.
What happens when China is not too interested in Foxconn, Foxconn is not too interested in Chinese workers, and Apple is busy inventing ways to prevent people from upgrading the Mac computers?
That’s what CNN understands. Protests, clashes, and violence. Toss in some Covid fear and one has the exciting story for consumers of CNN “real” news.
Is there are fix? For China and its attitude to Taiwanese businesses which allegedly exploit Chinese workers, sure. I won’t explore that solution. For Foxconn, sure, but it will take time for Foxconn to de-China its production operations. For Apple, not really. The company will follow the logic of the science club: Find some people who will work for less.
Net net: Apple and its HSSCMM will probably not find too many fans in the Middle Kingdom. And Foxconn? Do China and Apple care? Apple cares about money. China cares about the Middle Kingdom. Foxconn cares about what? Building plants in the US… soon?
Stephen E Arnold, November 28, 2022
Snorkel: Now Humans Are a Benefit?
November 23, 2022
Snorkel emerged from Stanford University’s AI lab. Some at the Google are ga-ga over Snorkel’s approach to reducing the cost of creating training sets for machine learning. If you are not paying attention to the expense of training models the old-fashioned way, when humans do the work, months or years of effort are required. Then — surprise — after operating in the real world for six months (plus or minus depending on the use case), the model has to be retrained.
Snorkel wants to get subject matter experts to build a training set one time. Then the numerical recipes will harvest additional information and automatically update the training set. Imagine better, faster, cheaper. Well, that’s the theory. Thus the entire AI industry push for finding short cuts to deal with the need for building training sets for initial model training and the work needed to make sure the model does not drift off into craziness. (I won’t mention the name of any search vendors, but a number of these outfits have performed oblation for their VC gods. Why? The results of the user’s query returned garbage. Confusing the information in a PowerPoint pitch with returning relevant and precise results for a user’s query is a bit like resolving the conflicts between Newtonian and quantum physics.)
I read “AI Startup Snorkel Preps a New Kind of Expert for Enterprise AI.” My immediate reaction was a question, “Why didn’t Google buy the company?” Hmmm. Now Snorkel is going to push to be a commercial success, perhaps like DeepDyve, an outfit which used or uses Snorkel technology.
The write up says:
Snorkel’s Data-centric Foundation Model Development, as the offering is called, is an enhancement to the startup’s flagship Snorkel Flow program. The new features let companies write functions that automatically create labeled training data by using what are called foundation models, the largest neural nets that exist, such as OpenAI’s GPT-3. The new functions in Snorkel Flow let a person who is a domain expert but not a programmer create a workflow that will then automatically generate labeled data sets that can be used to train the foundation programs for specific tasks.
The base technology emerged from projects guided in part by Christopher Ré. The work goes back more than a decade. Snorkel itself has been a start up for several years.
Smart software is getting a lot of tire kicking action by large companies. My hunch is that Snorkel wants to sell its methods to the firms just now having a bean counter come to a meeting and saying, “Have you taken a look at how much money our AI teams need to retrain our models?”
Then a whiz kid — possibly a graduate of Stanford — says, “Get Snorkel!”
Well, that’s my hunch. Will the models avoid the horrible fate of self immolating smart software which just gets stuff wrong? Probably not. But the PowerPoints and Zoom presentations will explain that Snorkel does not go “under water.” Snorkel lets an apoplectic accountant breathe somewhat more easily until the next quarterly analysis of smart software expenses.
Stephen E Arnold, November 23, 2022
Academic Publisher eLife Shifting to Peer Review Model
November 17, 2022
The racket, er, field of academic journalism has needed a shakeup for quite some time. Will this be the move that does it? Science reports, “Journal Seeks to Upend Scientific Publishing by Only Reviewing—Not Accepting—Manuscripts.” The non-profit, online-only eLife hopes the change will offer readers more nuance than the traditional accept-or-reject dichotomy. The free-to-read journal used to charge writers $3000 if it accepted and published their paper. Writer Jeffrey Brainard relates:
“Under the new approach, eLife will charge authors $2000 if they accept the publisher’s offer to have a submitted manuscript undergo peer review. Regardless of whether the critiques are positive or negative, the manuscript and its associated, unsigned peer-review statements will be posted online and be free to read. If the author revises the paper to address the comments, eLife will post the new version.
Since eLife was founded in 2012, it has tried other innovations. In 2020, for example, it started to require all submitted manuscripts be published as preprints. Abandoning the ‘accept’ stamp is a logical next step, says eLife’s editor-in-chief, biologist Michael Eisen of the University of California, Berkeley.
Eisen, who co-founded the open-access Public Library of Science journals in 2003, says the detailed critiques written by reviewers that eLife recruits are its main contribution to the scientific process. The reviews, he says, are ‘more nuanced, more informative, and more useful to the community than our thumbs-up or thumbs down publishing decision.’ He also argues that the new model will speed up a peer-review process that at other journals is often opaque and slow because it can involve multiple rounds.”
The plan is similar to a practice already put into place by open-research platform F1000Research, which allows readers to review manuscripts posted by researchers. Eisen, however, expects to offer higher quality critiques on his site. Some details are still being ironed out, including how to decide which papers to invite for review. The new policy is to be implemented in January 2023. Researchers funded by the NIH will be glad to know they can declare a reviewed manuscript the final version of record, allowing it to be indexed by the PubMed search engine (a funding requirement). Ultimately, says Eisen, the new approach will push the publisher to the background and researchers’ work to the fore. We wonder how other academic journals feel about that philosophy.
Cynthia Murrell, November 17, 2022
Post Pandemic Blues: Tablets and Chromebooks Struggle
November 17, 2022
Might smartphones make some devices irrelevant? We learn from The Register that “Tablet, Chromebook Shipments Come Crashing Down.” The article examines IDC’s report of third-quarter shipments. It states a mere 38.6 million tablets were shipped between July 01 and September 30, a decline of almost 9% since the previous year. Only Huawei grew its sales as demand escalated in China and Russia, where sanctions barred the way for Western tech. Writer Paul Kunert reports:
“Apple saw sales decline 1.1 percent to 14.5 million, according to IDC estimates. Samsung was down 4 percent to 7.1 million, Amazon fell 8.1 percent to 4.3 million, Lenovo shipments dropped 36.6 percent to 2.7 million, and Huawei grew 2 percent to 2.4 million. In its results filed late last week, Apple said iPad sales to end users were up 21 percent to $8.3 billion in Q4 of its fiscal ’22 ended 30 September despite supply constraints. IDC tracks sales into the channel, hence the difference in the figures. Chromebook shipments fell at a far faster rate, down 34.4 percent year-on-year to 4.3 million devices. This was the fifth straight decline for this sector of the PC industry. The downward trajectory began in the US, which accounted for 70 percent of global shipments. … IDC placed Acer as market leader with shipments of 1 million, albeit down 23.8 percent on a year ago. Dell shrank 19.9 percent to 900,000 units, HP was down 26.8 percent to 800,000, Lenovo plunged 54.8 percent to 700,000, and Samsung was down 37 percent to 300,000.”
Researchers point out Chromebook sales spiked during the pandemic as students connected from home, so its decline is simply a return to normal levels. As for the rest, a tough economy was likely at play. Apparently one can endure a slightly smaller small screen when fuel and groceries are difficult to afford.
Cynthia Murrell, November 17, 2022
Math: Who Needs It to be a CEO? Answer: Maybe FTX?
November 16, 2022
Let me be clear. I don’t think much about crypto. One outfit pitched me to invest in a way for lawyers to transfer a deceased client’s digital currency to the heirs. Nope. Don’t care. Another eager young MBA wanted me to get involved in a deal offering investors shares in virtual cows. Nope. Really didn’t care.
Nevertheless, I find the ups and downs, ins and outs of the crypto world interesting. I enjoy learning how specialist firms deanonymize crypto transactions. The idea of putting a bad actor in front of a judge makes me happy. However, listening to a crime analyst suggest that more than 50 percent of digital currency transactions are related to illegal activities makes me some what sad.
I absorbed the information in “‘The Bottom Has Dropped Out’: Study Confirms Fears of Growing Learning Gaps.” Let’s assume the data in the article are close enough horseshoes, which is the modus operadi today I believe. The write up says:
In the earliest weeks of the pandemic, researchers associated with NWEA made two jaw-dropping predictions. The first — that school closures would lead to lower math and reading scores — has been borne out over and over since then. The second — that the already broad range of academic levels within classrooms would yawn wider — has now been confirmed.
The article grinds through data which make one thing clear: Quite a few students cannot do math particularly well.
I found this statement in the estimable publication The Daily Mail’s article “Harry Potter Fan Ex girlfriend 28 of Founder Sam Bankman-Fried Bragged She Only Needed Elementary School Math to Be CEO of His Start Up, Despite Being Propped Up by Funds from His Failed Sister Crypto Exchange” fascinating:
The write up states:
Ellison [the alleged friend of FTX’s founder] disliked common trading safeguards such as stop-loss orders, a way of capping losses and reducing risk.
Yep, grade school math. I think that going forward more exciting things will surface. Oh, one plus one equals two, not one plus one equals a Mississippi River flow of money. A magic wand is real too. Wave it an a million people will be really happy.
Stephen E Arnold, November 16, 2022
What Goes Up Must Come Down Even in Zuckland
November 15, 2022
Facebook used to be the indomitable ruler of social media, then its popularity plummeted in the face of older users and other platforms. Zuckerberg is facing a similar decline with his Meta company, but the plunge hits his deep pockets. Techspot explains what is going on with Zuckerberg and Meta in the article, “Meta Value Down $520 Billion Last Year, Threatening Its Position As a Top 20 Company.”
Meta has less net profits because of the economic downturn in the United States. Companies are spending less money are advertisements through Meta’s products. Meta’s investors are also worried, because the company is funneling billions into the VR/MR “metaverse” project. Meta’s VR/MR branch is called Reality Labs and it lost $10.2 billion in 2021. Reality Labs’s losses are expected to increase in 2023. In 2022, the losses are expected to be $85-87 billion.
Facebook hit the $1 trillion market cap in June 2021 more quickly than any company before. At the beginning of 2022, Facebook was the sixth biggest company in the US. Since Zuckerberg, however, renamed his company Meta its worth has fallen and it could secure the twentieth spot in the biggest company list.
Zuckerberg continues to push VR agenda:
“Despite losing billions and an analyst’s prediction that many business projects in this area will close by 2025, Zuckerberg is doubling down on the metaverse. ‘Look, I get that a lot of people might disagree with this investment, but from what I can tell, I think this is going to be a very important thing,’ he said. ‘People will look back a decade from now and talk about the importance of the work being done here.’”
Zuckerberg was a visionary with Facebook. Is he replicating his visions with the metaverse? He is losing billions of dollars, but it could pay off or it will be another blunder in technology history.
Whitney Grace, November 15, 2022
-
- Subscribe to Beyond Search
Feature archive
News archive
-
