Big Numbers But What Is the End Game for Software Quality?

January 11, 2021

I cannot define quality without context. Furthermore, I am skeptical of really big round numbers. How does two trillion sound? Pretty suspicious, right? How does $2.08 trillion sound? Much more credible, right? A report from upscale universities and a standards group offers up the $2.08 trillion number. My problem is that this number appears to be pulled from thin air, and it may be too small. In short, the cost risk of lousy software is under-stated.

Let’s be honest. Exactly how big is two trillion? I know from experience that big numbers are designed to impress, but the reality is that big numbers don’t do much more than cause a person to dis-associate from the main point.

That’s the major flaw in “The Cost of Poor Software Quality in the US: A 2020 Report.” The numbers can be dismissed because software engineers, technical experts, and teenaged wizards laboring in the vineyards of the Google have created a bit of a problem.

What do I mean? I will try to answer this question, after looking at several points set forth in the 46 page document.

First, the report informs me that software quality is bound up with Covid. Yeah, fine. Site5.com, a hosting provider, offered this argument to me when their servers crashed for the sixth time in the last eight weeks. Sorry, Covid is not software unless one considers IBM’s assertion that supercomputers in Tennessee would identify drugs likely to deal with Covid? How is that working out exactly? The cost? Let’s make up some numbers?

Second, there are apparently four categories of crappy software that impose costs. These are, and I quote:

  1. Cost of unsuccessful IT / software projects
  2. Cost of poor quality in legacy systems
  3. Cost of operational software failures
  4. Cost of cybersecurity and technical debt.

The point about failed projects seems obvious. However, what about failed projects in US, state, country, and local governments systems. What is failure? What is the cost of a life when law enforcement systems cannot communicate and exchange information in near real time? Was that number included? And what about the cost of software which seems to work but levies a massive cost upon users? What’s the “cost” of phone home software or malware not detected by software systems purpose built to detect cyber breaches?

Let’s think about legacy systems at the IRS, those which manage air line reservations and flight control data, and the IBM machines chugging along in large financial institutions. Not only have the big academic brains and the whiz kids failed to create reliable methods for migrating or remediating legacy software, there has been virtually zero progress in the last few decades on using automated mechanisms for improving legacy code. Want an example? How about the failure of New Jersey to have sufficient COBOL programmers to deal with the mess in the state’s labor-related systems.

And those operational failures. It is easy for Amazon to assert that outage X cost us Y in sales. But what about the costs of delayed flights because the systems supporting the Chicago ARTCC functions go down or the rail freight routing systems hiccups and puts tens of thousands of empty freight cars in Texas. What’s the cost of Gmail outage? What’s the cost of glitch in the SWIFT financial system and its impact on a small business awaiting confirmation of a successful financial transaction.

Now we come to the cost of the cybersecurity thing. What’s the cost of the SolarWinds’ misstep? My hunch is that the number is very big, possibly equivalent to the economy of a pick up truck filled with mid sized EC countries GDP. And then the report addresses technical debt, I noted this statement:

In 2018, we reported the amount of software technical debt in the US was approximately $1.145 trillion, which represented just the debt principal without accumulated interest. At that time, we assumed a code growth rate of 35 billion LOC per year, projecting that there would be 1.455 trillion LOC worldwide (US share of 31%). We have since seen that code growth is now up to ~100 billion new LOC per year, or ~7% growth per year. Projecting those figures to 2020, and assuming that very little code has since been retired, there would now be 1.655 trillion LOC worldwide and 513 billion in the US. The US figure for technical debt in 2020 would therefore be $1.31 trillion.

Many numbers which ignore the dependent consequences of software which is either not maintained, maintained at the lowest possible cost, or just not maintained. Isn’t legacy software a component of technical debt? In fact, each day forward for an outfit like Google, the firm’s technical debt goes up. Modern software is often a combination of:

  • Undocumented or poorly document software fixes
  • Software wrappers which minimize the issues with flawed legacy code well enough to move on … until the next issue arises
  • Minimal changes made by contractors who are alleged specialists in legacy code or marginalized code
  • Changes introduced by essentially unmanaged, security free offshore “experts.”

But the numbers look interesting and big.

Read the report yourself and answer these questions:

  • How much does the report understate the fully loaded cost of lousy software?
  • Why is lousy software produced by graduates of prestigious institutions the norm?
  • What is the definition of lousy software? (The VC who makes money thinks whatever software is deployed as a zippy Azure security solution is just the greatest thing since sliced bread.)

What’s the fix?

Well, that’s the problem, isn’t it? There is none. There are free hacking courses, junior college Zoom courses, and fora available via interesting Web sites accessible via Tor or i2p. Certifications are possibly helpful if there are national standards. You know. Like the outstanding standards for USB support or the brilliant smart software which is amply documented in Weapons of Math Destruction.

That’s the point. Lousy software has “eaten the world.” There are fix skirmishes. Sometimes the fix wins and sometimes it doesn’t.

The report makes a big deal about numbers. These are the result of spreadsheet fever induced with long Excel sessions. The issue is that the number of two trillion is too small. And the academics yapping about quality. Check out what your students do when unbounded by ethical constraints?

Stephen E Arnold, January 11, 2021

 

Technical Debt with Cats: Lots of Cats

October 29, 2020

Cats are fine. Lots of cats can trigger a different reaction. I liked “Technical Debt: Why It’ll Ruin Your Software.” I ignored the cats and focused on the information payload of the article. The author does a good job of explaining what a number of people [a] ignore, [b] do not understand, and [c] miss the connection with cost and time over-runs.,

I circled three items in the write up:

First, I circled this passage:

The moment John chose the faster and easiest solution for him was the moment that the Technical Debt was inserted in the code.

The idea is that in order to “get ‘er done,” the Corona virus of cost, complexity, and chaos was let loose. The “faster and easiest” method is everywhere. Like a person with an addiction the individual will not admit, there is no single step toward remediation. The remediators will use the same method.

Second, I noted this diagram:

image

The chart makes clear what people under pressure often ignore. Costs are rising, and they may not be controllable. How much change has the core of Google search undergone in the last 20 years. Who wants to dig into the guts and deal with some of the interesting problems which exist? Answer: No one who wants a promotion and a chance to start a VC firm.

Third, the future is smart software:

In a realistic and respectable world, machines should take care of these situations, and not us.

Yep, and software will be just wonderful.

Stephen E Arnold, October 29, 2020

Snap: Has the Company Provided a Glimpse of the Future of Software?

August 14, 2020

A brief write-up at MakeUseOf alerts us to a novel approach to applications: “Snap Minis Are Bite-Sized Apps You Use in Snapchat.” Writer Dave Parrack tells us:

“These are miniature apps created by third-party developers that you can use in Snapchat without ever leaving the [Snapchat] app. Which could make Snapchat a solid choice for more than just teenagers. … The apps are all built directly into Snapchat using HTML5. This means you don’t have to leave the comfort of Snapchat to use them, they’re guaranteed to work for all users on all devices, and they don’t even need installing.”

Navigate to the post for details on how to use this feature in Snapchat, what Snap Minis are currently available, and which are in the works. Personally, we are more interested in the tactic itself. Parrack notes:

“This is part of a major effort by Snapchat to encourage developers to give users more to do inside Snapchat. Which should boost levels of engagement. The developers also gain access to millions of users who may then be tempted to install their full-featured apps. Making it a win-win for both parties. Snap clearly wants to make Snapchat a ubiquitous app you cannot afford not to have installed on your phone. Even if you aren’t interested in seeing what you would look like as a pug.”

Will Snapchat succeed in achieving ubiquity? Perhaps. Either way, the app-within-an-app concept offers new angles for platforms and developers alike.

Cynthia Murrell, August 14, 2020

Microsoft: Good Enough Is Not the Standard We Need

May 25, 2020

Imagine the topic options swirling around this weekend: A mass marketish iPhone jailbreak procedure, Amazon allegedly selling to blacklisted companies, Joe Rogan either pulling off the podcast coup of the year or falling into the black hole of irrelevance.

What catches DarkCyber’s eye?

Microsoft Acknowledges Internet Error in Windows 10 Cumulative Update KB4535996

Three points related to the allegedly accurate statement.

First, the problem affects some WFHers. Those are people who need the Internet to do work and get paid. Bad.

Second, the problem originated in February 2020, and it is only now (May 24, 2020) being “acknowledged.”

Third, Microsoft fouled up its magical online upgrade process.

So what?

Microsoft is gung-ho on the cloud, its “building” for the future, its reinvention of apps, and its partner flogging.

Maybe the company should consider that good enough is not good enough.

Even Amazon — a firm with some issues — steps up and says, “Hey, our vaunted speedy delivery is going to work like a horse drawn cart now.”

Microsoft appears to have embraced its good enough, and it is not.

I am tired of going to my office which has Linux, Mac, and Windows machines. There I see the Windows machine waiting for me to enter a secret code or press a button to update. Yesterday one of these machines reported that it couldn’t reach my Microsoft account?

These guys are going to do warfighting?

Good enough is not. Not for Google, not for Facebook, not for Amazon, and not for Microsoft.

Good enough. Does that mean excellence today?

Stephen E Arnold, May 25, 2020

Downloading Web Sites: Some Useful Information Available

February 20, 2020

Do you want to download a Web site or the content available from a specific url? What seems easy can become a tricky problem. For example, Google offers “feature” content which is more difficult to download than our DarkCyber video news program. Presumably flowing acrylic paint has more value than information about policeware software.

There are tools available; for example, Cyotek Web Copy and HTTrack, among others. But many of the available Web site downloaders often encounter problems with modern Web sites accessible via any “regular” browser. The challenges come from the general Wild West in which Internet accessible content resides.

One site ripping software goes an extra step. If you download the free version or pay for Microsys’ A1 Web Site Downloader, the developers have created a quite useful series of help pages. Many of the problems one can encounter trying suck down text, images, videos, or other content are addressed.

Navigate to the Microsys help pages and browse the list of topics available. Note that the help directs one to the A1 Web Site Downloader, but the information is likely to be useful if you are using another software or if you are trying to code your own site ripper.

The topics addressed by Microsys include:

  • Some basics like how to restrict how many pages are grabbed
  • Frequent problems encountered; for example, no urls located
  • The types of “options” available; for instance, dealing with Unicode. These “options” provide a useful checklist of important functions to include if you are rolling your own downloader. If you are trying to decide what alternative to AI Web Site Download, the list is useful.
  • A rundown of frequently encountered errors and response code; for example, hard and soft 404s
  • A summary of common platforms. (We liked the inclusion of information about EBay store data.)
  • General questions about the A1 software.

You can access the software and the useful help information via the Microsys Web site at this link. Version 1.0 is free. The current version is about US$40.

DarkCyber pays some attention to software which purports to download Web sites. If you want to download Dark Web sites or content accessible via an obfuscation system, you will have to look elsewhere or do your own programming.

Stephen E Arnold, February 20, 2020

VideoStudio 19 Ultimate Installation Failure: This Procedure May Help You

January 16, 2020

DarkCyber has never in our previous 16,000 posts provided a fix for a problem with commercial software. We are providing a fix for Corel’s VideoStudio 19 Ultimate for these reasons:

  1. Corel technical support responded to our trouble ticket but provided zero useful information
  2. There are problems on the disc with the packaged version of the software to make the resulting installation malfunction
  3. None of the posts on the Corel user forum provided a fix for what seems to be a common problem—No filters for special effects were installed
  4. The Corel download page for the program offers two files, but each is different. Careless, coincidence, lack of motivation, indifference, or some other reason? We don’t know.
Context

A local group was unable to install VideoStudeo 19 Ultimate VideoStudio Ultimate 2019 on their Windows 10 computer. The big problem was that the software  would appear to install on the boxed version of the software. On the machine used by the group, the install typically took two to three hours. But the software would not provide access the “filters,” which is Corel’s word for “video effects” or “FX.” Annoying. You bet. That’s what triggered a request for DarkCyber to help out this group.

Symptoms

1. Program crashed when started
2. Filters (video effects) were not available

After quite a bit of fooling around, we hit upon a solution for a computer running version 18362.

General Approach: A Bird’s-Eye View

1. Examine the failed installation
2. Install VideoStudio Ultimate 2019 on a different machine
3. Move the needed files to a USB stick
4. Copy the files from the working VideoStudio 2019 Ultimate installation to the installation of the software that did not work.

What You Need

This fix is not perfect, but it was one which worked for this particular volunteer group.

Here are the steps you may want to follow:

1. Install VideoStudio Ultimate 2019 on a Windows 10 computer. We used a spare from our shop which was a clean install; that is, no other software had been installed on the computer. No plug ins, third party anti virus software, nothing. Verify that the program loads and that the components are available; for example, filters (which VideoStudio sometimes calls FX).

2. Get a USB drive or comparable storage method and copy this one file
vfilter.rsf
and these two folders:

vft_plug
vfx_plug

Here’s where you should be able to find these files on your computer:
vfilter.rsf is located at:

[root drive] \Users\ [user name] \AppData\Roaming\Ulead Systems\Corel VideoStudio Pro (x64)\22.0\VFilter.rsf

For the machines we used, the default locations were that the vft_plug folder was at:

[root drive] \Program Files\Corel\Corel VideoStudio 2019\vft_plug

The location of vfx_plug was at:

[root drive] \Program Files\Corel\Corel VideoStudio 2019\Vfx_plug

The Fix Which Worked for Us

We had to do several steps. None was difficult once we determined the names of the files and their location. Corel does not provide this information to its customers. We don’t want to speculate about why the company does not address this problem. A number of VideoStudio 2019 Ultimate owners have this problem. Therefore, we are posting this as a report of what worked for the non profit group. We used a legal copy of the software. We used a “clean” Windows 10 machine. No clean Windows 10 computer? No legal copy of the software? Well, you may be out of luck.

The Steps We Followed

The procedure we followed for vfilter.rsf was:

a. Navigate to the location of vfilter.rsf
b. Rename the file to vfilter old.rsf
c. Copy the version of vifilter.rsf from the USB or storage device.

For clarity, you are substituting the files from an installation of VideoStudio Ultimate 2019 which works.

The procedure we followed for the folder and its files vft_plug was:

a. Navigate to the location of vft_plug
b. Rename the folder to vft_plug old. (There is no need to fiddle with the files in the folder)
c. Copy the version of the vft_plug folder from the USB or storage device.

For clarity, your are replacing the vft_plug folder from the machine with the working installation of VideoStudio Ultimate 2019.

The procedure we followed for the folder and its files vfx_plug was:

a. Navigate to the location of vfx_plug
b. Rename the folder to vfx_plug old (There is no need to fiddle with the files in the folder)
c. Copy the version of the vfx_plug folder from the USB or storage device.

For clarity, your are replacing the vfx_plug folder from the machine with the working installation of VideoStudio Ultimate 2019.

Wrap Up

Corel support was not helpful. Corel documentation was not helpful. Most of the citations returned from Bing, Google, Yandex, and Boardwatch queries were not helpful. Therefore, we had to find a solution because the non profit group lacked the funds to buy a more robust video editor or find staff to learn how to use one of the open source options available.

For us, we rebooted the computer and launched VideoStudio Ultimate 2019. The Filters (FX) were accessible under “My Favorites.” Note: Our fix did display animated icons in the Filters (FX) panel. The user has to click “My Favorites” to see the installed Filters (FX).

The other functions of the VideoStudio Ultimate 2019 software were working when we left the office of the group contacting us.

If you find a quicker or improved way to get this low cost video editing software to work, use the comments section of this blog.

Stephen E Arnold, January 16, 2020

Into R? A List for You

May 12, 2019

Computerworld, which runs some pretty unusual stories, published “Great R Packages for Data Import, Wrangling and Visualization.” “Great” is an interesting word. In the lingo of Computerworld, a real journalist did some searching, talked to some people, and created a list. As it turns out, the effort is useful. Looking at the Computerworld table is quite a bit easier than trying to dig information out of assorted online sources. Plus, people are not too keen on the phone and email thing now.

The listing includes a mixture of different tools, software, and utilities. There are more than 80 listings. I wasn’t sure what to make of XML’s inclusion in the list, but, the source is Computerworld, and I assume that the “real” journalist knows much more than I.

Two observations:

  • Earthworm lists without classification or alphabetization are less useful to me than listings which are sorted by tags and alphabetized within categories. Excel does perform this helpful trick.
  • Some items in the earthworm list have links and others do not. Consistency, I suppose, is the hobgoblin of some types of intellectual work
  • An indication of which item is free or for fee would be useful too.

Despite these shortcomings, you may want to download the list and tuck it into your “Things I love about R” folder.

Stephen E Arnold, May 12, 2019

Kentucky Technology: Stick with Horse Racing

February 21, 2018

Who knows if the information in “KFC: Enemy of Waistlines, AI, Arteries and Logistics Software” is steroid infused or faux chicken.

image

I loved the factoids in the write up for three reasons:

  1. I live in Harrod’s Creek, Kentucky, where fried squirrel is almost as popular as fried chicken with those herbs, spices, and what nots.
  2. Kentucky Fried Chicken has, according to local legend, had some squabbles in the software barnyard. Does Taco Bell system play nice with the fried chicken outlet systems? What about restaurant management software from folks in the even deeper South down Atlanta way?
  3. Kentucky Fried Chicken is famous in certain circles for the perfect celebratory feast. In Chicago, so the chatter goes, the buckets are a required food stuff after an event.

Here are the factoids I noted:

  • Self driving cars think the KFC logo is a stop sign. (I watched the Yandex self driving car video and it sailed right by what appeared to be fast food joints. If I spot a Yandex car braking for a bucket, I will pass along the information.)
  • A software glitch nuked some important information.
  • A shift to DHL from an outfit called Bidvest created a chicken shortage in some UK KFC outlets. No chicken? How does one fix this? Hit the Waitrose? Nah, shut the friend chicken shops.

Are you hungry for a two piece meal with the mandatory biscuit? Tip: Don’t tell the human at the counter to skip the biscuit. You have to wait if none are sitting on the ready line. Don’t like it? Hmmm.

Stephen E Arnold, February 21, 2018

Proprietary Software Cheats Users

November 16, 2017

Cory Doctorow is an outspoken defender of net neutrality, technology education, and user rights.  He has written and spoken about these subjects and shares his opinion on BoingBoing.  The science-fiction magazine Locus recently published one of his new essays,“Cory Doctorow: Demon-Haunted World.”  Doctorow discusses how software can be programmed to take out the human factor of like and steer things in favor of corporations who want to gobble down dollars.

Cheating is a well-established enterprise that originated long before the digital revolution, but it is getting smarter as technology advances.  While in the past it was cheating was more of a danger from outside forces, it is now nestled within the very things we own.

The software allows companies and literally anyone with the know how to cheat you out of money or precious time.  Rather than cheat en masse, the cheating is coming to your home because it is so much easier to infiltrate the individual now.  Even scarier is when he uses an alchemy metaphor, explaining how alchemists were cut-rate lab technicians who believed spirits, God, and demons influenced their experiments.  The technology used for cheating has a similar demonic presence and that is not even the worst factor.

Doctorow pulls out his trump card when he explains how outdated technology laws from the 20th century still had standing today when it is more than obvious they need to be repealed:

What’s worse, 20th-century law puts its thumb on the scales for these 21st-century demons. The Computer Fraud and Abuse Act (1986) makes it a crime, with jail-time, to violate a company’s terms of service. Logging into a website under a fake ID to see if it behaves differently depending on who it is talking to is thus a potential felony, provided that doing so is banned in the small-print clickthrough agreement when you sign up.

 

Then there’s section 1201 of the Digital Millen­nium Copyright Act (1998), which makes it a felony to bypass the software controls access to a copy­righted work. Since all software is copyrightable, and since every smart gadget contains software, this allows manufacturers to threaten jail-terms for anyone who modifies their tractors to accept third-party carburetors (just add a software-based check to ensure that the part came from John Deere and not a rival), or changes their phone to accept an independent app store, or downloads some code to let them choose generic insulin for their implanted insulin pump.

Follow Doctorow’s advice, read, test, learn, and just combat ignorance.

Whitney Grace, November 16, 2017

Elsevier Makes a Brave Play to Steal Wikipedias Users

October 9, 2017

Is Wikipedia about to be unseated in the world of academic publishing? Elsevier thinks they can give the crowdsourced, yet flawed, info hub a serious run for its money. Money, being the key word, according to a recent TechDirt article, “Elsevier Launching Rival to Wikipedia by Extracting Scientific Definitions Automatically from Author’s Texts.”

According to the piece:

Elsevier is hoping to keep researchers on its platform with the launch of a free layer of content called ScienceDirect Topics, offering an initial 80,000 pages of material relating to the life sciences, biomedical sciences and neuroscience. Each offers a quick definition of a key term or topic, details of related terms and relevant excerpts from Elsevier books.

Seems like it makes sense, right? Elsevier has all this academic information at their fingertips, so why send users elsewhere on the web for other information. This extraction system, frankly, sounds pretty amazing. However, TechDirt has a beef with it.

It’s typical of Elsevier’s unbridled ambition that instead of supporting a digital commons like Wikipedia, it wants to compete with it by creating its own redundant versions of the same information, which are proprietary. Even worse, it is drawing that information from books written by academics who have given Elsevier a license.

It’s a valid argument, whether or not Elsevier is taking advantage of its academic sources by edging into Wikipedia’s territory. However, we have a hunch their lawyers will make sure everything is on the up and up. A bigger question is whether Elsevier will make this a free site or have a paywall. They are in business to make money, so we’d guess paywall. And if that’s the case, they’d better have a spectacular setup to draw customers from Wikipedia.

Patrick Roland, October 9, 2017

Next Page »

  • Archives

  • Recent Posts

  • Meta