Big Numbers But What Is the End Game for Software Quality?
January 11, 2021
I cannot define quality without context. Furthermore, I am skeptical of really big round numbers. How does two trillion sound? Pretty suspicious, right? How does $2.08 trillion sound? Much more credible, right? A report from upscale universities and a standards group offers up the $2.08 trillion number. My problem is that this number appears to be pulled from thin air, and it may be too small. In short, the cost risk of lousy software is under-stated.
Let’s be honest. Exactly how big is two trillion? I know from experience that big numbers are designed to impress, but the reality is that big numbers don’t do much more than cause a person to dis-associate from the main point.
That’s the major flaw in “The Cost of Poor Software Quality in the US: A 2020 Report.” The numbers can be dismissed because software engineers, technical experts, and teenaged wizards laboring in the vineyards of the Google have created a bit of a problem.
What do I mean? I will try to answer this question, after looking at several points set forth in the 46 page document.
First, the report informs me that software quality is bound up with Covid. Yeah, fine. Site5.com, a hosting provider, offered this argument to me when their servers crashed for the sixth time in the last eight weeks. Sorry, Covid is not software unless one considers IBM’s assertion that supercomputers in Tennessee would identify drugs likely to deal with Covid? How is that working out exactly? The cost? Let’s make up some numbers?
Second, there are apparently four categories of crappy software that impose costs. These are, and I quote:
- Cost of unsuccessful IT / software projects
- Cost of poor quality in legacy systems
- Cost of operational software failures
- Cost of cybersecurity and technical debt.
The point about failed projects seems obvious. However, what about failed projects in US, state, country, and local governments systems. What is failure? What is the cost of a life when law enforcement systems cannot communicate and exchange information in near real time? Was that number included? And what about the cost of software which seems to work but levies a massive cost upon users? What’s the “cost” of phone home software or malware not detected by software systems purpose built to detect cyber breaches?
Let’s think about legacy systems at the IRS, those which manage air line reservations and flight control data, and the IBM machines chugging along in large financial institutions. Not only have the big academic brains and the whiz kids failed to create reliable methods for migrating or remediating legacy software, there has been virtually zero progress in the last few decades on using automated mechanisms for improving legacy code. Want an example? How about the failure of New Jersey to have sufficient COBOL programmers to deal with the mess in the state’s labor-related systems.
And those operational failures. It is easy for Amazon to assert that outage X cost us Y in sales. But what about the costs of delayed flights because the systems supporting the Chicago ARTCC functions go down or the rail freight routing systems hiccups and puts tens of thousands of empty freight cars in Texas. What’s the cost of Gmail outage? What’s the cost of glitch in the SWIFT financial system and its impact on a small business awaiting confirmation of a successful financial transaction.
Now we come to the cost of the cybersecurity thing. What’s the cost of the SolarWinds’ misstep? My hunch is that the number is very big, possibly equivalent to the economy of a pick up truck filled with mid sized EC countries GDP. And then the report addresses technical debt, I noted this statement:
In 2018, we reported the amount of software technical debt in the US was approximately $1.145 trillion, which represented just the debt principal without accumulated interest. At that time, we assumed a code growth rate of 35 billion LOC per year, projecting that there would be 1.455 trillion LOC worldwide (US share of 31%). We have since seen that code growth is now up to ~100 billion new LOC per year, or ~7% growth per year. Projecting those figures to 2020, and assuming that very little code has since been retired, there would now be 1.655 trillion LOC worldwide and 513 billion in the US. The US figure for technical debt in 2020 would therefore be $1.31 trillion.
Many numbers which ignore the dependent consequences of software which is either not maintained, maintained at the lowest possible cost, or just not maintained. Isn’t legacy software a component of technical debt? In fact, each day forward for an outfit like Google, the firm’s technical debt goes up. Modern software is often a combination of:
- Undocumented or poorly document software fixes
- Software wrappers which minimize the issues with flawed legacy code well enough to move on … until the next issue arises
- Minimal changes made by contractors who are alleged specialists in legacy code or marginalized code
- Changes introduced by essentially unmanaged, security free offshore “experts.”
But the numbers look interesting and big.
Read the report yourself and answer these questions:
- How much does the report understate the fully loaded cost of lousy software?
- Why is lousy software produced by graduates of prestigious institutions the norm?
- What is the definition of lousy software? (The VC who makes money thinks whatever software is deployed as a zippy Azure security solution is just the greatest thing since sliced bread.)
What’s the fix?
Well, that’s the problem, isn’t it? There is none. There are free hacking courses, junior college Zoom courses, and fora available via interesting Web sites accessible via Tor or i2p. Certifications are possibly helpful if there are national standards. You know. Like the outstanding standards for USB support or the brilliant smart software which is amply documented in Weapons of Math Destruction.
That’s the point. Lousy software has “eaten the world.” There are fix skirmishes. Sometimes the fix wins and sometimes it doesn’t.
The report makes a big deal about numbers. These are the result of spreadsheet fever induced with long Excel sessions. The issue is that the number of two trillion is too small. And the academics yapping about quality. Check out what your students do when unbounded by ethical constraints?
Stephen E Arnold, January 11, 2021
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
