Smart Software Project Road Blocks: An Up-to-the-Minute Report
October 1, 2024
![green-dino_thumb_thumb_thumb_thumb_t[2]_thumb_thumb](https://arnoldit.com/wordpress/wp-content/uploads/2024/09/green-dino_thumb_thumb_thumb_thumb_t2_thumb_thumb-2.gif "green-dino_thumb_thumb_thumb_thumb_t[2]_thumb_thumb")
This essay is the work of a dumb dinobaby. No smart software required.
I worked through a 22-page report by SQREAM, a next-gen data services outfit with GPUs. (You can learn more about the company at this buzzword dense link.) The title of the report is:
The report is a marketing document, but it contains some thought provoking content. The “report” was “administered online by Global Surveyz [sic] Research, an independent global research firm.” The explanation of the methodology was brief, but I don’t want to drag anyone through the basics of Statistics 101. As I recall, few cared and were often good customers for my class notes.
Here are three highlights:
- Smart software and services cause sticker shock.
- Cloud spending by the survey sample is going up.
- And the killer statement: 98 percent of the machine learning projects fail.
Let’s take a closer look at the astounding assertion about the 98 percent failure rate.
The stage is set in the section “Top Challenges Pertaining to Machine Learning / Data Analytics.” The report says:
It is therefore no surprise that companies consider the high costs involved in ML experimentation to be the primary disadvantage of ML/data analytics today (41%), followed by the unsatisfactory speed of this process (32%), too much time required by teams (14%) and poor data quality (13%).
The conclusion the authors of the report draw is that companies should hire SQREAM. That’s okay, no surprise because SQREAM ginned up the study and hired a firm to create an objective report, of course.
So money is the Number One issue.
Why do machine learning projects fail? We know the answer: Resources or money. The write up presents as fact:
The top contributing factor to ML project failures in 2023 was insufficient budget (29%), which is consistent with previous findings – including the fact that “budget” is the top challenge in handling and analyzing data at scale, that more than two-thirds of companies experience “bill shock” around their data analytics processes at least quarterly if not more frequently, that that the total cost of analytics is the aspect companies are most dissatisfied with when it comes to their data stack (Figure 4), and that companies consider the high costs involved in ML experimentation to be the primary disadvantage of ML/data analytics today.
I appreciated the inclusion of the costs of data “transformation.” Glib smart software wizards push aside the hassle of normalizing data so the “real” work can get done. Unfortunately, the costs of fixing up source data are often another cause of “sticker shock.” The report says:
Data is typically inaccessible and not ‘workable’ unless it goes through a certain level of transformation. In fact, since different departments within an organization have different needs, it is not uncommon for the same data to be prepared in various ways. Data preparation pipelines are therefore the foundation of data analytics and ML….
In the final pages of the report a number of graphs appear. Here’s one that stopped me in my tracks:
The sample contained 62 percent user of Amazon Web Services. Number 2 was users of Google Cloud at 23 percent. And in third place, quite surprisingly, was Microsoft Azure at 14 percent, tied with Oracle. A question which occurred to me is: “Perhaps the focus on sticker shock is a reflection of Amazon’s pricing, not just people and overhead functions?”
I will have to wait until more data becomes available to me to determine if the AWS skew and the report findings are normal or outliers.
Stephen E Arnold, October 1, 2024
FOGINT: Telegram Changes Its Tune
October 1, 2024
![green-dino_thumb_thumb_thumb_thumb_t[2]_thumb](https://arnoldit.com/wordpress/wp-content/uploads/2024/09/green-dino_thumb_thumb_thumb_thumb_t2_thumb-2.gif "green-dino_thumb_thumb_thumb_thumb_t[2]_thumb")
This essay is the work of a dumb dinobaby. No smart software required.
Editor note: The term Fogint is a way for us to identify information about online services which obfuscate or mask in some way some online activities. The idea is that end-to-end encryption, devices modified to disguise Internet identifiers, and specialized “tunnels” like those associated with the US MILNET methods lay down “fog”. A third-party is denied lawful intercept, access, or monitoring of obfuscated messages when properly authorized by a governmental entity. Here’s a Fogint story with the poster boy for specialized messaging, Pavel Durov.
Coindesk’s September 23, 2024, artice “Telegram to Provide More User Data to Governments After CEO’s Arrest” reports:
Messaging app Telegram made significant changes to its terms of service, chief executive officer Pavel Durov said in a post on the app on Monday. The app’s privacy conditions now state that Telegram will now share a user’s IP address and phone number with judicial authorities in cases where criminal conduct is being investigated.
Usually described as a messaging application, Telegram is linked to a crypto coin called TON or TONcoin. Furthermore, Telegram — if one looks at the entity from 30,000 feet — consists of a distributed organization engaged in messaging, a foundation, and a recent “society” or “social” service. Among the more interesting precepts of Telegram and its founder is a commitment to free speech and a desire to avoid being told what to do.
Art generated by the MSFT Copilot service. Good enough, MSFT.
After being detained in France, Mr. Durov has made several changes in the way in which he talks about Telegram and its precepts. In a striking shift, Mr. Durov, according to Coindesk:
said that “establishing the right balance between privacy and security is not easy,” in a post on the app. Earlier this month, Telegram blocked users from uploading new media in an effort to stop bots and scammers.
Telegram had a feature which allowed a user of the application to locate users nearby. This feature has been disabled. One use of this feature was its ability to locate a person offering personal services on Telegram via one of its functions. A person interested in the service could use the “nearby” function and pinpoint when the individual offering the service was located. Creative Telegram users could put this feature to a number of interesting uses; for example, purchasing an illegal substance.
Why is Mr. Durov abandoning his policy of ignoring some or most requests from law enforcement seeking to identify a suspect? Why is Mr. Durov eliminating the nearby function? Why is Mr. Durov expressing a new desire to cooperate with investigators and other government authority?
The answer is simple. Once in the custody of the French authorities, Mr. Durov learned of the penalties for breaking French law. Mr. Durov’s upscale Parisian lawyer converted the French legal talk into some easy to understand concepts. Now Mr. Durov has evaluated his position and is taking steps to avoid further difficulties with the French authorities. Mr. Durov’s advisors probably characterized the incarceration options available to the French government; for example, even though Devil’s Island is no longer operational, the Centre Pénitentiaire de Rémire-Montjoly, near Cayenne in French Guiana, moves Mr. Durov further from his operational comfort zone in the Russian Federation and the United Arab Emirates.
The Fogint team does not believe Mr. Durov has changed his core values. He is being rational and using cooperation as a tactic to avoid creating additional friction with the French authorities.
Stephen E Arnold, October 1, 2024
Surveillance Watch Maps the Surveillance App Ecosystem
October 1, 2024
Here is an interesting resource: Surveillance Watch compiles information about surveillance tech firms, organizations that fund them, and the regions in which they are said to operate. The lists, compiled from contributions by visitors to the site, are not comprehensive. But they are full of useful information. The About page states:
“Surveillance technology and spyware are being used to target and suppress journalists, dissidents, and human rights advocates everywhere. Surveillance Watch is an interactive map that documents the hidden connections within the opaque surveillance industry. Founded by privacy advocates, most of whom were personally harmed by surveillance tech, our mission is to shed light on the companies profiting from this exploitation with significant risk to our lives. By mapping out the intricate web of surveillance companies, their subsidiaries, partners, and financial backers, we hope to expose the enablers fueling this industry’s extensive rights violations, ensuring they cannot evade accountability for being complicit in this abuse. Surveillance Watch is a community-driven initiative, and we rely on submissions from individuals passionate about protecting privacy and human rights.”
Yes, the site makes it easy to contribute information to its roundup. Anonymously, if one desires. The site’s information is divided into three alphabetical lists: Surveilling Entities, Known Targets, and Funding Organizations. As an example, here is what the service says about safeXai (formerly Banjo):
“safeXai is the entity that has quietly resumed the operations of Banjo, a digital surveillance company whose founder, Damien Patton, was a former Ku Klux Klan member who’d participated in a 1990 drive-by shooting of a synagogue near Nashville, Tennessee. Banjo developed real-time surveillance technology that monitored social media, traffic cameras, satellites, and other sources to detect and report on events as they unfolded. In Utah, Banjo’s technology was used by law enforcement agencies.”
We notice there are no substantive links which could have been included, like ones to footage of the safeXai surveillance video service or the firm’s remarkable body of patents. In our view, these patents represent an X-ray look at what most firms call artificial intelligence.
A few other names we recognize are IBM, Palantir, and Pegasus owner NSO Group. See the site for many more. The Known Targets page lists countries that, when clicked, list surveilling entities known or believed to be operating there. Entries on the Funding Organizations page include a brief description of each organization with a clickable list of surveillance apps it is known or believed to fund at the bottom. It is not clear how the site vets its entries, but the submission form does include boxes for supporting URL(s) and any files to upload. It also asks whether one consents to be contacted for more information.
Cynthia Murrell, October 1, 2024
-
- Subscribe to Beyond Search
Feature archive
News archive
-
