Google Smart Software: Bert and Ernie Subbed for Joe Fourier
May 24, 2021
The story “Google Replaces Bert Self Attention with Fourier Transform: 92% Accuracy, 7 Times Faster on GPUs” may be more smart software hog wash. Against the colorful background of Google’s artificial intelligence melodrama, now Bert (shorthand for the wonderfully named Bidirectional Encoder Representations from Transformers) has some flaws. It seems accuracy and performance are among the ones which caught someone’s attention at the Google.
What’s up next?
The 19th century methods of Joe Fourier, who dabbled in fancy math in the 18th and 19th century until his vibrations petered out in 1830. Joe may be gone, but this math has new life. Why? Bert, it appears, was not exactly a high performer in the 100 meter sprints needed to make ad centric smart software work.
The write up is filled with jargon. That’s okay since anyone who reads about Fourier transforms knows something about the methods. However, what’s left out is an explanation of how the DeepMind research team has reacted to the Sesame Street infused acronym.
My hunch is that there are some sharp differences of opinion because, well, Google, smart software, algorithmic issues, and management methods.
Stephen E Arnold, May 24, 2021
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
