Reproducibility: Academics and Smart Software Share a Quirk

January 15, 2023

I can understand why a human fakes data in a journal article or a grant document. Tenure and government money perhaps. I think I understand why smart software exhibits this same flaw. Humans put their thumbs (intentionally or inadvertently) put their thumbs on the button setting thresholds and computational sequences.

The key point is, “Which flaw producer is cheaper and faster: Human or code?” My hunch is that smart software wins because in the long run it cannot sue for discrimination, take vacations, and play table tennis at work. The downstream consequence may be that some humans get sicker or die. Let’s ask a hypothetical smart software engineer this question, “Do you care if your model and system causes harm?” I theorize that at least one of the software engineer wizards I know would say, “Not my problem.” The other would say, “Call 1-8-0-0-Y-O-U-W-I-S-H and file a complaint.”

Wowza.

The Reproducibility Issues That Haunt Health-Care AI” states:

a data scientist at Harvard Medical School in Boston, Massachusetts, acquired the ten best-performing algorithms and challenged them on a subset of the data used in the original competition. On these data, the algorithms topped out at 60–70% accuracy, Yu says. In some cases, they were effectively coin tosses1. “Almost all of these award-winning models failed miserably,” he [Kun-Hsing Yu, Harvard]  says. “That was kind of surprising to us.”

Wowza wowza.

Will smart software get better? Sure. More data. More better. Think of the start ups. Think of the upsides. Think positively.

I want to point out that smart software may raise an interesting issue: Are flaws inherent because of the humans who created the models and selected the data? Or, are the flaws inherent in the algorithmic procedures buried deep in the smart software?

A palpable desire exists and hopes to find and implement a technology that creates jobs, rejuices some venture activities, and allows the questionable idea that technology to solve problems and does not create new ones.

What’s the quirk humans and smart software share? Being wrong.

Stephen E Arnold, January 15, 2023

Common Sense: A Refreshing Change in Tech Write Ups

December 13, 2022

I want to give a happy quack to this article: “Forget about Algorithms and Models — Learn How to Solve Problems First.” The common sense write up suggests that big data cowboys and cowgirls make sure of their problem solving skills before doing the algorithm and model Lego drill. To make this point clear: Put foundations in place before erecting a structure which may fail in interesting ways.

The write up says:

For programmers and data scientists, this means spending time understanding the problem and finding high-level solutions before starting to code.

But in an era of do your own research and thumbtyping will common sense prevail?

Not often.

The article provides a list a specific steps to follow as part of the foundation for the digital confection. Worth reading; however, the write up tries to be upbeat.

A positive attitude is a plus. Too bad common sense is not particularly abundant in certain fascinating individual and corporate actions; to wit:

  • Doing the FBX talkathons
  • Installing spyware without legal okays
  • Writing marketing copy that asserts a cyber security system will protect a licensee.

You may have your own examples. Common sense? Not abundant in my opinion. That’s why a book like How to Solve It: Modern Heuristics is unlikely to be on many nightstands of some algorithm and data analysts. Do I know this for a fact? Nope, just common sense. Thumbtypers, remember?

Stephen E Arnold, December 13, 2022

The Freakonomics Approach to Decision Making

November 18, 2022

It is predictable an economist like Steven Levitt would apply statistics to the process of making life’s big choices, but one may be surprised at the simplistic solution he has deduced. Levitt, an economist at the University of Chicago, hosts the “Freakonomics” podcast. Freethink explains how a “‘Freakonomics’ Study Offers Simple Strategy for Making Tough Decisions.” The study had each participant make a binary choice, to make a change or not, with a coin toss and report back. Levitt found a trend in the results. Writer Stephen Johnson reports:

“Most surprising were the results on well-being. At both the two and six-month marks, most people who chose change reported feeling happier, better off, and that they had made the correct decision and would make it again. ‘The data from my experiment suggests we would all be better off if we did more quitting,’ Levitt said in a press release. ‘A good rule of thumb in decision making is, whenever you cannot decide what you should do, choose the action that represents a change, rather than continuing the status quo.’ The study had some limitations. One is that its participants weren’t selected randomly. Rather, they opted in to the study after visiting FreakonomicsExperiments.com, which they likely heard about from the podcast or various social media channels associated with it. Another limitation is that participants whose decision didn’t play out well might have been less likely to report back on their status after two and six months. So, the study might be over-representing positive outcomes. Still, the study does suggest that people who are on the margin of a tough decision — that is, people who really can’t decide which option is best — are probably better off going with change.”

Perhaps. Johnson concludes with an old trick for checking your gut instinct that also involves a coin flip? Go ahead and toss that coin, then see which side you find yourself hoping it will land on. Will either of these methods really point to the best decision? Is Mr. Musk using them to inform decision making at Twitter? Are the results reproducible?

Cynthia Murrell, November 18, 2022

LinkedIn: The Logic of the Greater Good

September 26, 2022

I have accepted two factoids about life online:

First, the range of topics searched from my computer systems available to my research team is broad, diverse, and traverses the regular Web, the Dark Web, and what we call the “ghost Web.” As a result, recommendation systems like those in use by Facebook, Google, and Microsoft are laughable. One example is YouTube’s suggesting that one of my team would like an inappropriate beach fashion show here, a fire on a cruise ship here, humorous snooker shots here, or sounds heard after someone moved to America here illustrate the ineffectuality of Google’s smart recommendation software. These recommendations make clear that when smart software cannot identify a pattern or an intentional pattern disrupting click stream, data poisoning works like a champ. (OSINT fans take note. Data poisoning works and I am not the only person harboring this factoid.) Key factoid: Recommendation systems don’t work and the outputs can be poisoned… easily.

Second, profile centric systems like Facebook’s properties or the LinkedIn social network struggle to identify information that is relevant. Thus, we ignore the suggestions for who is hiring people with your profile and the requests to be friends. These are amusing. Here are some anonymized examples. A female in Singapore wanted to connect me with an escort when I was next in Singapore. I interpreted this as a solicitation somewhat ill suited to a 77 year old male who no longer flies to Washington, DC. Forget Singapore. What about a person who is a sales person at a cable company? Or what about a person who does land use planning in Ecuador? What about a person with 19 years experience as a Google “partner”? You get the idea. Pimps and resellers of services which could be discontinued without warning. Key factoid: Recommendations don’t match that I am retired, give lectures to law enforcement and intelligence professionals, and stay in my office in rural Kentucky, with my lovable computers, a not so lovable French bulldog, and my main squeeze for the last 53 years. (Sorry, Singapore intermediary for escorts. Sad smile)

I read a write up in the indigestion inducing New York Times. I am never sure if the stories are accurate, motivated by social bias, written by a persistent fame seeker, or just made up by a modern day Jayson Blair. For info, click here. (You will have to pay to view this exciting story about fiction presented as “real” news.

The story catching my attention today (Saturday, September 24, 2022) has the title “LinkedIn Ran Social Experiments on 20 Million Users over Five Years?” Obviously the author is not familiar with the default security and privacy settings in Windows 10 and that outstanding Windows 11. Data collection both explicit and implicit is the tension in in the warp and woof of the operating systems’ fabric.

Since Microsoft owns LinkedIn, it did not take me long to conclude that LinkedIn like its precursor Plaxo had to be approached with caution, great caution. The write up reports that some Ivory Tower types figured out that LinkedIn ran and probably still runs tests to determine what can get more users, more clicks, and more advertising dollars for the Softies. An academic stalking horse is usually a good idea.

I did spot several comments in the write up which struck me as amusing. Let’s look at a three:

First, consider this statement:

LinkedIn, which is owned by Microsoft, did not directly answer a question about how the company had considered the potential long term consequences of its experiments on users’ employment and economic status.

No kidding. A big tech company being looked at for its allegedly monopolistic behaviors not directly answering a New York Times’ reporters questions. Earth shaking. But the killer gag for me is wanting to know if Microsoft LinkedIn “consider the potential long term consequences of its experiments.” Ho ho ho. Long term at a high tech outfit is measured in 12 week chunks. Sure, there may be a five year plan, but it probably still includes references to Microsoft’s network card business, the outlook for Windows Phone and Nokia, and getting the menus and icons in Office 365 to be the same across MSFT applications, and pitching the security of Microsoft Azure and Exchange as bulletproof. (Remember. There is a weapon called the Snipex Alligator, but it is not needed to blast holes through some of Microsoft’s vaunted security systems I have heard.)

Second, what about this passage from the write up:

Professor Aral of MIT said the deeper significance of the study was that it showed the importance of powerful social networking algorithms — not just in amplifying problems like misinformation but also as fundamental indications or economic conditions like employment and unemployment.

I think a few people understand that corrosive, disintermediating impact of social media information delivered quickly can have an effect. Examples range from flash mob riots to teens killing themselves because social media just does such a bang up job of helping adolescents deal with inputs from strangers and algorithms which highlight the thrill of blue screening oneself. The excitement of asking people who won’t help one find a job is probably less of a downer but failing to land an interview via LinkedIn might spark binge watching of “Friends.”

Third, I loved this passage:

“… If you want to get more jobs, you should be on LinkedIn more.

Yeah, that’s what I call psychological triggering: Be on LinkedIn more. Now. Log on. Just don’t bother to ask me to add you my network of people whom I don’t know because “Stephen E Arnold” on LinkedIn is managed by different members of my team.

Net net: Which is better? The New York Times or Microsoft LinkedIn. You have 10 minutes to craft an answer which you can post on LinkedIn among the self promotions, weird facts, and news about business opportunities like paying some outfit to put you on a company’s Board of Advisors.

Yeah, do it.

Stephen E Arnold, September 26, 2022

Pew Data about Social Media Use: Should I Be Fearful? Answer: Me, No. You? Probably

September 26, 2022

The Pew Research outfit published more data about social media. If you want to look at the factsheet, navigate to this Pew link. I want to focus on one small, probably meaningless item. What interested me was how those in the sample get their news. If I read the snazzy graphics correctly:

  1. 82 percent of those in the sample use YouTube. (Does that make YouTube a monopoly?) Of those YouTube users, 25 percent get their “news” from the Alphabet Google YouTube DeepMind entity.
  2. 30 percent of those in the sample use TikTok, that friendly entity linked with the CCP. Of those TikTok adepts, 10 percent get their news from the Middle Kingdom’s information output and usage intake system.
  3. Other services deliver news, but it is not clear if video is the mechanism. Video interests me because of the Marshall McLuhan hot-cold notion. Video is the digital garden for couch potatoes. Reading is a bit more active, or so the fans of McLuhan would suggest.

Why am I fearful? How about these thoughts, conceived while consuming a cheese sandwich?

  1. Potent mechanisms for injecting shaped or weaponized information into consumers of video news are in the hands of two entities focused on achieving their goals. China is into having the US become subservient to the Middle Kingdom and redress the arrogance Americans have manifested over the years. The AGYD entity wants money and the ability to shape the direction in which it would prefer the users go. My view is their the approach of each entity is the same. The goals are somewhat different.
  2. Most consumers of video and news are unaware of the functionality of weaponized video information. My view is that it is pretty darned good at tearing down and cultivating certain interesting mental frameworks.
  3. Weaponization is trivial, particularly when each AGYD and TikTok can use money to incentivize the individuals and firms producing content for the respective services’ audience.

Net net: Once one pushes into double digit content dependence, a tipping point is something that can cause what appears to be a stable structure to collapse. Can digital information break the camel’s back? For sure. Am I fearful? Nah. Others? Probably not and that increases my concern.

Stephen E Arnold, September 26, 2022

False Expertise: Just Share and Feel Empowered in Intellect

September 15, 2022

I read “Share on Social Media Makes Us Overconfident in Our Knowledge.” The write up states:

Social media sharers believe that they are knowledgeable about the content they share, even if they have not read it or have only glanced at a headline. Sharing can create this rise in confidence because by putting information online, sharers publicly commit to an expert identity. Doing so shapes their sense of self, helping them to feel just as knowledgeable as their post makes them seem.

If the source were a hippy dippy online marketing outfit, I would have ignored the write up. But the research comes from a cow town university. I believe the write up. Would those cowpokes steer me wrong, pilgrim?

I wonder if the researchers will take time out after a Cowboy Kent Rollins cook out to explore the correlation between the boundless expertise of the Silicon Valley “real news” crowd and this group’s dependence on Twitter and similar output channels?

That would make an interesting study because some of the messaging is wild and crazy like a college professor lost in a college bar on dollar beer night.

Stephen E Arnold, September 15, 2022

Site Rot Quantified

July 20, 2022

There’s weird page rot. That was a feature of MySpace and GeoCities. Then there was link rot. That was a feature of my original Web site when I retired. I just stopped remediating dead links. I did not want to do the work myself and I allowed the majority of my team to find their future elsewhere. Ergo, dead links. Too bad, Google.

Now there is site rot.

10% of the Top One Million Sites Are Dead” explains the process of figuring out this number. There are rah rahs for tools and scripts. Good stuff, but my interest is a single number:

892,013

Several early morning thoughts (July 16, 2022):

  • The idea that a million is not a million illustrates the inherent ageing and concomitant deterioration of Internet “things”; namely, Web sites. Why are sites not sites as defined in the write up? Money, laziness, inconsistencies engineered into the information superhighway, or some other reason?
  • Locating sites on the Wayback Machine or whatever it is now called is an exercise in frustration. With sites rotting and Wayback delivering zero content, the data void is significant.
  • The moniker “million” when the count is smaller is another example of the close-enough-for-horse-shoes approach which is popular among some high-tech outfits.

Just remember. I don’t care, and I wonder how many others share my mind set. Good enough.

Stephen E Arnold, July 20, 2022

A Modern Believe It or Not: Phones, Autos, and Safety

June 24, 2022

Auto insurance firm Jerry recently put out a study purporting to prove Android users are safer drivers than those who use iPhones. It almost looks like a desperate, shadow PR move from Google; is the company so insecure it feels compelled to reshape data to “prove” its quantum supremacy? If so, The Next Web thwarts its efforts in the analysis, “Sorry Android Users, You’re Actually NOT the Safest Drivers.” Writer Cate Lawrence examines Jerry’s research then proceeds to poke holes in its conclusions. She writes:

“In its research, Jerry analyzed data collected from 20,000 drivers during 13 million kilometers of driving over 14 days. The data generated an overall driving score and sub-scores for acceleration, speed, braking, turning, and distraction. Then it grouped the results by smartphone operating system and various demographic characteristics. Specifically, the research found that Android users scored an overall 75, trumping iPhone users’ score of 69 in terms of safe driving overall. Sure, they scored higher, but there’s not much of a difference between 69 and 75. And even less between 82 and 84 for accelerating, or 78 and 80 for braking. Overall, I’m not sure these are significant enough differences to instigate any kind of action or triumph. Look, I get it. You number crunch, and you want to make a big assertion to prove a hypothesis, or whatever. … But these numbers are more nice than assertive. The only one that really interested me was distracted driving. This category had the biggest difference, with Android users scoring 74 over iPhone users’ 68, seven points higher. I would have liked some insights on this.”

For example, she suggests, perhaps the iPhone’s apps are more distracting or its users more absorbed in selecting audio material. Alas, the Jerry report is more about pushing its main assertion than in exploring insights.

The study also looked at disparities by educational levels and credit ratings, reporting Android users on the low end of both scales outperformed iPhone users at all levels. Though it failed to explore reasons that may be, Lawrence suggested a couple: Those with less education and with lower credit scores are likely to have lower income levels, and Android phones tend to be more affordable than iPhones. Perhaps lower-income folks have more driving experience, or they are more careful because they cannot afford a ticket. We simply do not know, and neither does Jerry. Instead, the study asserts it comes down to differences in personality between Android and iPhone users. Though it can point to a couple of sources that could be seen to back it up, we agree with the write-up that the connection is a “bit of a stretch.” Sorry Google, your PR arm will have to try harder. Or you could just focus on making a better OS.

Cynthia Murrell, June 24, 2022

Three Facts: Staffing, Old Media, and Zuck Math

April 19, 2022

I spotted three factoids in my newsfeed this morning (April 13, 2022, 0700 am US Eastern). Interesting items reveal the shift taking place in the swirling worlds of digital information.

The first factoid comes from “Cybersecurity at a Crossroads.” I noted:

Security talent shortages are especially pronounced in Germany – 53% of organizations in that country reported that struggles with recruiting and retention resulted in multiple project delays over the past 12 months, compared to 43% across other countries.

The second factoid is derived from two separate articles. The first items comes from the real news source CNBC and its write up “CNN+ Struggles to Lure Viewers in Its Early Days, Drawing Fewer Than 10,000 Daily Users.” The fact may be slushy but it is interesting because 10,000 seems modest in terms of the alleged $300 million spent for the for fee service. The second factoid comes from the paywalled “The TikTok War Didn’t Cause the TikTok Boom.” Here’s the fact: The service had attracted about 1.6 billion users in a recent month. Let’s assume these data are close enough for horseshoes. It’s clear that there is a audience pull difference between old media and new media.

The third factoid is from “Meta Plans to Take Nearly 50% of Creator’s Earnings in ‘Horizon Worlds’.” Zuckbook (formerly Facebook) will take 50 percent of money earned by a person who sells digital artifacts inside the virtual world “Horizon Worlds.” Seems fair to the Zuckbook team I assume.

Observations:

  • Cyber security is morphing into cyber insecurity with no easy fix at this time.
  • Old media lacks the magnetism of the China-linked new media from TikTok.
  • Apple’s changes to app behavior appear to cause Zuckbook to charge fees once associated with a money lender in Florence in the 11th century.

Stephen E Arnold, April 19, 2022

Teams Tracking: Are You Working at Triple Peak?

April 14, 2022

I installed a new version of Microsoft Office. I had to spend some time disabling the Microsoft Cloud, Outlook, and Teams, plus a number of other odds and ends. Who in my office uses Publisher? Sorry, not me. In fact, I knew only one client who used Publisher and that was years ago. We converted that lucky person to an easier to use and more stable product.

We have tried to participate in Teams meetings. Unfortunately the system crashes on my Mac Mini, my Intel workstation, and my AMD workstation. I know the problem is obviously the fault of Apple, Intel, and AMD, but it would be nice if the Teams software would allow me to participate in a meeting. The workaround in my office is to use Zoom. It plays nice with my machines, my mostly secure set up, and the clumsy finger of my 77 year old self.

I provide the context so that you will understand my reaction to “Microsoft Discovers Triple Peak Work Day for Its Remote Employees.” As you may know, Microsoft has been adding features to Teams since the pandemic lit a fire under what was once a software service reserved for financial meetings and some companies that wanted everyone no matter what to be in a digital face to face meeting. Those were super. I did some work for an early video conferencing player. I think it was called Databeam. Yep, perfect for kids who wanted to take a virtual class, not a presentation about the turbine problems at Lockheed Martin.

Microsoft’s featuritis has embraced surveillance. I won’t run down the tools available to an “administrator” with appropriate access to a Teams’ set up for a company. I want to highlight the fact that Microsoft shared with ExtremeTech some information I find fascinating; to wit:

… when employees were in the office, it found “knowledge workers” usually had two periods of peak productivity: before lunch and after lunch. However, with everyone working from home there’s now a third period: late at night, right before bedtime.

My workday has for years begun about 6 am. I chug along until lunch. I then chug along until dinner. Then I chug along until I go to sleep at 10 pm. I like to think that my peak times are from 6 am to 9 am, from 10 am to noon, from 1 30 pm to 3 pm, and from 330 to 6 pm. I have been working for more than 50 years, and I am happy to admit that I am an old fashioned Type A person. Obviously Microsoft does not have many people like me in its sample. The morning, as I recall from my Booz, Allen & Hamilton days, the productive in the morning crowd was a large cohort, thousands in fact. But not in the MSFT sample. These are lazy dogs its seems.

Let’s imagine your are a Type A manager. You have some employees who work from home or from a remote location like a client’s office in Transnistia which you may know as the Pridnestrovian Moldavian Republic. How do you know your remotes are working at their peak times? You monitor the wily creatures: Before lunch, after lunch, and before bed or maybe to a disco in downtown Tiraspol.

How does this finding connect with Teams? With everyone plugged in from morning to night, the Type A manager can look at meeting attendance, participation, side talks, and other detritus sucked up by Teams’ log files. Match up the work with the times. Check to see if there are three ringing bells for each employee. Bingo. Another HR metric to use to reward or marginalize a human personnel asset.

I will just use Zoom and forget about people who do not work when I do.

Stephen E Arnold, April 14, 2022

Next Page »

  • Archives

  • Recent Posts

  • Meta