Big Numbers But What Is the End Game for Software Quality?
January 11, 2021
I cannot define quality without context. Furthermore, I am skeptical of really big round numbers. How does two trillion sound? Pretty suspicious, right? How does $2.08 trillion sound? Much more credible, right? A report from upscale universities and a standards group offers up the $2.08 trillion number. My problem is that this number appears to be pulled from thin air, and it may be too small. In short, the cost risk of lousy software is under-stated.
Let’s be honest. Exactly how big is two trillion? I know from experience that big numbers are designed to impress, but the reality is that big numbers don’t do much more than cause a person to dis-associate from the main point.
That’s the major flaw in “The Cost of Poor Software Quality in the US: A 2020 Report.” The numbers can be dismissed because software engineers, technical experts, and teenaged wizards laboring in the vineyards of the Google have created a bit of a problem.
What do I mean? I will try to answer this question, after looking at several points set forth in the 46 page document.
First, the report informs me that software quality is bound up with Covid. Yeah, fine. Site5.com, a hosting provider, offered this argument to me when their servers crashed for the sixth time in the last eight weeks. Sorry, Covid is not software unless one considers IBM’s assertion that supercomputers in Tennessee would identify drugs likely to deal with Covid? How is that working out exactly? The cost? Let’s make up some numbers?
Second, there are apparently four categories of crappy software that impose costs. These are, and I quote:
- Cost of unsuccessful IT / software projects
- Cost of poor quality in legacy systems
- Cost of operational software failures
- Cost of cybersecurity and technical debt.
The point about failed projects seems obvious. However, what about failed projects in US, state, country, and local governments systems. What is failure? What is the cost of a life when law enforcement systems cannot communicate and exchange information in near real time? Was that number included? And what about the cost of software which seems to work but levies a massive cost upon users? What’s the “cost” of phone home software or malware not detected by software systems purpose built to detect cyber breaches?
Let’s think about legacy systems at the IRS, those which manage air line reservations and flight control data, and the IBM machines chugging along in large financial institutions. Not only have the big academic brains and the whiz kids failed to create reliable methods for migrating or remediating legacy software, there has been virtually zero progress in the last few decades on using automated mechanisms for improving legacy code. Want an example? How about the failure of New Jersey to have sufficient COBOL programmers to deal with the mess in the state’s labor-related systems.
And those operational failures. It is easy for Amazon to assert that outage X cost us Y in sales. But what about the costs of delayed flights because the systems supporting the Chicago ARTCC functions go down or the rail freight routing systems hiccups and puts tens of thousands of empty freight cars in Texas. What’s the cost of Gmail outage? What’s the cost of glitch in the SWIFT financial system and its impact on a small business awaiting confirmation of a successful financial transaction.
Now we come to the cost of the cybersecurity thing. What’s the cost of the SolarWinds’ misstep? My hunch is that the number is very big, possibly equivalent to the economy of a pick up truck filled with mid sized EC countries GDP. And then the report addresses technical debt, I noted this statement:
In 2018, we reported the amount of software technical debt in the US was approximately $1.145 trillion, which represented just the debt principal without accumulated interest. At that time, we assumed a code growth rate of 35 billion LOC per year, projecting that there would be 1.455 trillion LOC worldwide (US share of 31%). We have since seen that code growth is now up to ~100 billion new LOC per year, or ~7% growth per year. Projecting those figures to 2020, and assuming that very little code has since been retired, there would now be 1.655 trillion LOC worldwide and 513 billion in the US. The US figure for technical debt in 2020 would therefore be $1.31 trillion.
Many numbers which ignore the dependent consequences of software which is either not maintained, maintained at the lowest possible cost, or just not maintained. Isn’t legacy software a component of technical debt? In fact, each day forward for an outfit like Google, the firm’s technical debt goes up. Modern software is often a combination of:
- Undocumented or poorly document software fixes
- Software wrappers which minimize the issues with flawed legacy code well enough to move on … until the next issue arises
- Minimal changes made by contractors who are alleged specialists in legacy code or marginalized code
- Changes introduced by essentially unmanaged, security free offshore “experts.”
But the numbers look interesting and big.
Read the report yourself and answer these questions:
- How much does the report understate the fully loaded cost of lousy software?
- Why is lousy software produced by graduates of prestigious institutions the norm?
- What is the definition of lousy software? (The VC who makes money thinks whatever software is deployed as a zippy Azure security solution is just the greatest thing since sliced bread.)
What’s the fix?
Well, that’s the problem, isn’t it? There is none. There are free hacking courses, junior college Zoom courses, and fora available via interesting Web sites accessible via Tor or i2p. Certifications are possibly helpful if there are national standards. You know. Like the outstanding standards for USB support or the brilliant smart software which is amply documented in Weapons of Math Destruction.
That’s the point. Lousy software has “eaten the world.” There are fix skirmishes. Sometimes the fix wins and sometimes it doesn’t.
The report makes a big deal about numbers. These are the result of spreadsheet fever induced with long Excel sessions. The issue is that the number of two trillion is too small. And the academics yapping about quality. Check out what your students do when unbounded by ethical constraints?
Stephen E Arnold, January 11, 2021
DarkTrace: A Controversial View
January 6, 2021
I spotted that a post about Darktrace had been removed from Reddit. I became curious because the comment thread was on Reddit when I checked today (January 4, 2021). I located the original Darktrace post on the Archive.org site at this link. This content may be disappeared, and some of the points run counter to the rah rah write ups about the company. Here are some of the factoids and assertions which caught my attention:
- A Darktrace initial public offering is likely to take place in the near future
- 10 members of the Darktrace executive team allegedly had ties to Autonomy, the search and content management vendor acquired by HP
- Michael Lynch is part of an investment firm which funded Darktrace
- Goldman Sachs snubbed the Darktrace float.
None of the information in the Reddit post struck me as controversial. The data appear to come from a variety of open sources, including the Darktrace Web site, news reports, LinkedIn biographies, and public documents.
Why did I chase down the original post? The removal of the information from the threat sparked a number of interesting Reddit comments about Darktrace, the company’s business tactics, and the cyber security sector.
With the SolarWinds’ misstep still in the news cycle, it strikes me that cyber security related posts provide additional color about the products and services some of the higher profile vendors are offering.
Reddit obviously does not agree.
Stephen E Arnold, January 6, 2021
Greed, Security, and MBAs: Compromising Security for Yachts, Snazzy Cars, and Big Houses?
January 5, 2021
I read “How to Get Rich Sabotaging Nuclear Weapons Facilities.” The title is snappy. The blend of sabotage, nuclear weapons, and money is a spicy blend. I have been critical of cyber security firms’ marketing. I’ve mentioned their lingo, the nifty exhibits at law enforcement and intelligence conferences, and the endless reports about data for sale on the Dark Web.
I admit that I have focused on the flashier side of the business. I leave the specifics of repurposing open source software wrapped in scripts to others. I also have not linked obvious financial plays like the sale of 4iQ to Alto Analytics or the Recorded Future tie up with Insight Partners or any of the other mergers and roll ups emerging from the cyber security gold rush.
Why? I have been commenting about the craziness of MBAs for years, and — guess what? — no one cares. When I worked for some archetypal MBAs at assorted financial institutions, to a person the individuals agreed. I recall one flashy MBA as saying to me, “That’s right. I want money. Lots of money.” That fine individual asked me to pay for lunch because he left his wallet in his desk.
The write up about sabotage and nuclear weapons seems to be getting traction. In the aftermath of the SolarWinds’ misstep, this passage has more meaning to the average thumb typer and social media maven:
Cybersecurity is a very weird area, mostly out of sight yet potentially very deadly. Anonymous groups can turn off power plants, telecom grids, or disrupt weapons labs, as Israel did when it used a cyber-weapon to cripple Iranian nuclear facilities in 2010. Bank regulators have to now consult with top military leaders about whether deposit insurance covers incidents where hackers destroy all bank records, and what that would mean operationally. It’s not obvious whether this stuff is war or run-of-the-mill espionage, but everyone knows that the next war will be chock full of new tactics based on hacking the systems of one’s adversary, perhaps using code placed in those systems during peacetime.
The high-flying SolarWinds sparked this comment:
SolarWinds didn’t bother to hire a senior official to focus on security until 2017, and then only after it was forced to do so by European regulations. Even then, SolarWinds CEO, Kevin Thompson, ignored the risk. As the New York Times noted, one security “adviser at SolarWinds, said he warned management that year that unless it took a more proactive approach to its internal security, a cybersecurity episode would be “catastrophic.”
What was the root cause? The write up points the finger at a roll up specialist called Bravo. I learned:
After its IPO, SolarWinds followed Ellison’s advice, became a merger machine, buying a dozen companies from 2011-2014, including Pingdom, Confio and N-Able Technologies. In 2015, Thoma Bravo Partners (along with Silver Lake) bought the company, and loaded it up with $2 billion of debt to finance the purchase. (Yes, this was one of those purchases in which the private equity buyer bought the company with the company’s own money.) Under Bravo’s control, SolarWinds engaged in more mergers, buying companies who made threat monitoring software, email security, database performance monitoring, and IT support firms. SolarWinds sought to become a one-stop-shop in its niche, not particular good at quality, but with everything a customer might need. Of course, the Federal Trade Commission and the European Competition Commission allowed these deals; just a month before the hack was revealed, the FTC approved yet another acquisition by SolarWinds.
What happened?
The misstep. The write up points out:
But in some ways it’s not that complex; the problem isn’t that Russians are good at hacking and U.S. defenses are weak, it’s that financiers in America make more money by sabotaging key infrastructure than by building it.
The root cause, therefore, is that which generates revenue in an environment in which regulators are asleep at the switch, MBAs plot their next big deal, and those who assume that whiz bang, smart security systems actually work.
Stephen E Arnold, January 5, 2021
Palantir Founder European Investments
December 22, 2020
Just a suggestion for those who want to know what Peter Thiel finds financially interesting. Point your browser thing at “Here’s Where Peter Thiel Is Investing in Europe.” The write up includes some biographical details about one of Palantir’s founders. Then the guts of the write up is a list of companies in which Mr. Thiel’s funds have invested. The list is organized by country so one can see that Belgium has only one outfit with Thiel appeal, Topcompare.be. Germany and the ever interesting UK each have about a dozen hoped to be winners. One of Mr. Thiel’s UK investments is the money sucking Google DeepMind. There’s a pony in there somewhere at the digital stables.
Stephen E Arnold, December 22, 2020
A Plan for a Recurring Google Tax Takes Shape
December 16, 2020
I spotted what looked like another ho hum the EC wants to penalize Google again story. “Tech Giants Face Fine of Up to 10% of Turnover for EU Rule Breaches -EU Source” contains a couple of nuggets. The first is that not just Google is a target. Now the goal is a company defined as a “technology” firm is fair game. With companies explaining that their operation is based on information, it is possible for the Google Tax to apply to companies different from the Google; for example, a health care company or a logistics outfit.
Second, this passage opens the door to financial and market data disclosures and may institutionalize a permanent penalty tax, maybe a tariff to just operate in the ED:
The rules, known as the Digital Markets Act, set out a list of dos and don’ts for online gatekeepers to ensure a level playing field for rivals and users. This could include requiring dominant companies to share certain kinds of data with rivals and regulators while practices such as companies favoring their own services could be outlawed.
This is likely to give some other nation states ideas for institutionalizing additional fees on “technology” companies. Who will pay these fees? Probably users.
Also, the write up does not identify a source. This is an interesting way to create “real” news when one is a trusted outfit. At least the source lives in the EC, maybe?
Stephen E Arnold, December 16, 2020
France: Know Your Anonymous Digital Currency Customers
December 14, 2020
I think this is a fine idea. France has many fine ideas. Do not say PC; say micro ordinature. Do not feed that chicken this; feed that chicken this. Do not confuse the right and left side of the Rhone.
“France Declares War on Crypto Anonymity, Cites ‘Terrorism’ in KYC Mandate” explains that the land of more than 200 cheese and a silky method of making friends in England wants crypto currency to be different. You know. Just not anonymous.
The write up states:
All virtual asset service providers must immediately begin checking their customers’ identities, verifying “beneficial owners” and prohibit anonymous crypto accounts, according to the press release from Finance Minister Bruno Le Maire. He called the action a necessary step in France’s fight against terrorism. The press release invoked a terrorist cell that apparently financed itself with crypto until its dismantling in September 2019. “We must drain the euro from all terrorist financing channels,” Le Maire declared in a tweet.
Yes, very French.
I must admit, however, that the French posture regarding crypto currency is one that seems okay with me. My research assistants remind me that more than half of anonymous Bitcoin transactions appear to be related to illegal activities.
The Dark Web would not have functioning markets for contraband without the now ubiquitous anonymous digital currency.
My hunch is that France’s announcement is a harbinger of similar actions from other nation states. The more quickly one of the lubricants of a range of illegal activities is linked to actual and verifiable identities certain types of crime will become closer to the long arm of the law.
Stephen E Arnold, December 14, 2020
How Will MindGeek Get Paid? Umm, Encrypted and Anonymous Digital Currencies Maybe
December 11, 2020
I have followed the strong MasterCard and Visa response to revelations about MindGeek’s less-than-pristine content offerings. The Gray Lady wrote about MindGeek and then other “real” news sites picked up the story. A good example is “Visa, MasterCard Dump Pornhub Over Abuse Video Claims.” The write ups appear to have sidestepped one question which seems obvious to me:
How will MindGeek collect money?
There are some online ad outfits which have been able to place ads on Dark Web sites and on some other sites offering specialized content, not very different from MindGeek’s glittering content array. Amped up advertising seems one play.
But what about MindGeek’s paying customers?
Perhaps MindGeek, nestled in the Euro-centric confines of Montréal, will come up with the idea to use a digital currency. Invoices can be disseminated in secret messaging systems like those favored by the Russian based Edward Snowden. The payments can flow via encrypted digital currencies. Now many transactions can be tracked by government authorities in a number of countries. Nevertheless, making this type of shift is likely to increase the burden on investigators.
Just as killing off Backpage created additional work for some law enforcement professionals. The MasterCard and Visa termination may have a similar effect. Yes, the backlog can be resolved. But that is likely to add friction to some enforcement activities. A failure by regulatory agencies to get a handle of payments systems (encrypted and unencrypted) is now evident to some.
Stephen E Arnold, December 11, 2020
Cryptocurrency Competition at the Tallest of Ivory Towers
December 7, 2020
Ah, what better use of the finest in education than financial manipulation? We learn from cryptocurrency and Web 3.0 news source Decrypt, “Oxford, Cambridge Students Compete to Make Money on Crypto Markets.” The competition was put to math and computer science students at the two prestigious UK universities by crypto algorithmic trading firm APEX:E3. The firm also brought Coinbase, FTX, SIX Digital Exchange, LMAX Digital, and ConsenSys in on the project. Reporter Robert Stevens writes:
“Since November 16, the students have been competing to make the most money from crypto exchanges Coinbase Pro and FTX, according to a press release APEX:E3 pushed out this morning. APEX:E3 said that students have already spent their days conjuring up algorithms that incorporate technologies and concepts as diverse as machine-learning, neural networks, and whale order trading. In return for their toils, APEX:E3 is providing Oxford and Cambridge students from math and computer science departments access to its APIs, technical support, mentorship, and an undisclosed amount of ‘seed funding.’ … When the competition concludes at some point next month, the clocks will stop and a panel of ‘industry leading judges’ will rate the teams on their algorithms.”
The winning team gets to keep that mysterious amount of seed funding plus their returns. Cambridge is billing the project as a “fun and risk-free way” for these handpicked students to learn about the cryptocurrency industry. It also looks like a cheap and salary-free way for the industry to pick some of the brightest young minds for profitable ideas. A win-win, I suppose.
Soon Facebook’s carpe diem will arrive and more excitement shall ensue.
Cynthia Murrell, December 7, 2020
Palantir Technologies Sparks a Controversial Metaphor
December 4, 2020
In an interview with a policeware/intelware vendor, I learned about a financial company’s view of Palantir Technologies. This is the 13 year old start up which recently went public. The company had an astounding 130 customers, about $600 million in revenue, and a modest $500 million in losses in 12 months.
Here’s the comment which I chased down in its original tweeter output glossiness:
The operative phrase is:
A full casino.
If I were a Palantirian unpacking boxes in Denver, Colorado, I would hit the Yellow Pages or the Seeing Stone. An attorney might have some thoughts about a malicious metaphor disseminated via the marvelous firm managed part time by a bearded CEO.
Palantir. A full casino. Whatever does that mean?
Stephen E Arnold, December 4, 2020
Some US Big Tech Outfits Say Laisse Tomber
December 2, 2020
The trusted “real news” outfit Thomson Reuters published “Amazon, Apple Stay Away from New French Initiative to Set Principles for Big Tech.” Quelle surprise! The “principle” is the silly notion of getting big US technology companies to pay their taxes, fair taxes. Incroyable? Companies not getting with the program allegedly include Apple, Facebook, Google, and Microsoft. These four firms are likely to perceive the suggestion of fairness as a demonstration of flawed logic. It is possible that the initiative may become a cause célèbre because money. France is a mere country anyway.
Stephen E Arnold, December 2, 2020
-
- Subscribe to Beyond Search
Feature archive
News archive
-
