Machine Learning: Cheating Is a Feature?
August 9, 2022
I read “MIT Boffins Make AI Chips 1 Million Times Faster Than the Synapses in the Human Brain. Plus: Why ML Research Is Difficult to Produce – and Army Lab Extends AI Contract with Palantir.” I dismissed the first item as some of the quantum supremacy stuff output by high school science club types. I ignored the Palantir Technologies’ item because the US Army has to make a distributed common ground system work and leave resolution to the next team rotation. Good or bad, Palantir has the ball. But the middle item in the club sandwich article contains a statement I found particularly interesting.
If you have followed out comments about smart software, we have taken a pragmatic view of getting “AI/ML” systems to work in the 80 to 95 percent confidence range in a consistent way even when new “content objects” are fed into the zeros and ones. To get off on the right foot, human subject matter experts assembled training data which reflected the content the system would be processing in the real world. The way smart software is expected to work is that it learns… on its own… sort of. It is very time consuming and very expensive to create hand crafted training sets and then “update” the system with the affected module. What if the prior content had to be reprocessed? Well, not too many have the funds, time, resources, and patience for that.
Thus, today’s AI/ML forward leaning cost conscious wizards want to use synthetic data, minimize the human SMEs’ cost and time, and do everything auto-magically. Sounds good. Yes, and the ideas make great PowerPoint decks too.
The sentence in the article which caught may attention is this one:
Data leakage occurs when the data used to train an algorithm can leak into its testing; when its performance is assessed the model seems better than it actually is because it has already, in effect, seen the answers to the questions. Sometimes machine learning methods seem more effective than they are because they aren’t tested in more robust settings.
Here’s the link to “Leakage and the Reproducibility Crisis in ML-Based Science in which more details appear. Wowza if these experts are correct. Who goes swimming without a functioning snorkel? Maybe the Google?
Stephen E Arnold, August 8, 2022
Google Management Insights: About Personnel Matters No Less
June 16, 2022
Google is an interesting company. Should we survive Palantir Technologies’ estimate of a 30 percent plus chance of a nuclear war, we can turn to Alphabet Google YouTube to provide management guidance. Keep in mind that the Google has faced some challenges in the human resource, people asset department in the past. Notable examples range from frisky attorneys to high profile terminations of individuals like Dr. Timnit Gebru. The lawyer thing was frisky; the Timnit thing was numbers about bias.
“Google’s CEO Says If Your Return to the Office Plan Doesn’t Include These 3 Things You’re Doing It Wrong. It’s All About What You Value” provides information about the human resource functionality of a very large online advertising bar room door. Selling, setting prices, auctioning, etc. flip flop as part of the design of the digital saloon. “Pony up them ad collars, partner or else” is ringing in my ears.
The conjunction of human resources and “value” is fascinating. How does one value one Timnit?
What are these management insights:
First, you must have purpose. The write up provides this explanatory quote:
A set of our workforce will be fully remote, but most of our workforce will be coming in three days a week. But I think we can be more purposeful about the time they’re in, making sure group meetings, collaboration, creative brainstorming, or community building happens then.
Okay, purpose seems to be more organized. Okay, in the pre Covid era why did Google require multiple messaging apps? What about those social media plays going way back to Orkut?
Second, you must be flexible. Again the helpful expository statements appear in the write up:
At Google, that means giving people choices. Some employees will be back in the office full time. Others will adopt a hybrid approach where they work in the office three days a week, and from home the rest of the time. In other cases, employees might choose to relocate and work fully remotely for a period of time.
Flexibility also suggests being able to say one thing and then changing that thing. How will Googlers working in locations with lower costs of living? Maybe pay them less? Move them from one position to another in order to grow or not impede their more productive in office colleagues? Perhaps shifting a full timer to a contractor basis? That’s a good idea too. Flexibility is the key. For the worker, sorry, we’re talking management not finding a life partner.
Third, you must do something with choice. Let’s look at the cited article to figure out choice:
The sense of creating community, fostering creativity in the workplace collaboration all makes you a better company. I view giving flexibility to people in the same way, to be very clear. I do think we strongly believe in in-person connections, but I think we can achieve that in a more purposeful way, and give employees more agency and flexibility.
Okay, decide, Googler. No, not the employee, the team leader. If Googlers had choice, some of those who pushed back and paraded around the Google parking lot, would be getting better personnel evaluation scores.
Stepping back, don’t these quotes sound like baloney? They do to me. And I won’t mention the Glass affair, the overdosed VP on his yacht, or the legal baby thing.
Wow. Not quite up to MIT – Epstein grade verbiage, but darned close. And what about “value”? Sort of clear, isn’t it, Dr. Gebru.
Stephen E Arnold, June 16, 2022
The UK National Health Service: The Search for a Silver Bullet
June 13, 2022
Modern health care is a bit of muddle. The UK’s National Health Service has licensed, tested, tire kicked, and tried every angle to manage its myriad activities.
According to the odd orange newspaper (the Financial Times), the often befuddled NHS may be ready to embrace the PowerPoint assertions of a US company. “Palantir Gears Up to Expand Its Reach into UK’s NHS” reports:
Over the next few months, Palantir will bid for the five-year £360mn contract for the proposed Federated Data Platform (FDP), a new data tool to connect and integrate patient and other data sources from across the health system, so real-time decisions can be made effectively by clinicians and bureaucrats.
How similar is delivering health care to analyzing information to win a battle or figure out what an adversary is likely to do?
I am not sure. I do know that many intelware companies (this is my term for firms providing specialized software and services to law enforcement, crime analysts, and intelligence professionals) find that commercial clients can become squeamish under these conditions:
- Question from potential customer: “Who are your customers?” Intelware vendor: “Sorry, that information is classified.”
- Question from potential customer: “Can you provide a specific example of how your system delivered fungible results?” Intelware vendor: “We are not permitted to disclose either the use or effect of our system.”
- Question from potential customer: “How much consulting and engineering are needed before we can provide access to the system?” Intelware vendor: “That depends.” Customer asks a follow up question: “Can you be more specific?” Intelware vendors: “That information is classified.”
You can see how the commercial outfits not engaged in fighting crimes against children, drug smuggling, terrorist actions, termination of adversaries, etc. can be a tough sell.
But one of the big issues is the question, “Is our data available to government entities in our country or elsewhere without our knowledge or permission?”
Every licensee wants to here assurances that data are private, encrypted, protected by 20 somethings in Slough, or whatever is required to close the deal.
But there is the suspicion that when a company does quite a bit of work for certain government agencies in one or more countries, stuff happens. Data mining, insider actions, or loss of data control due to bad actors behavior.
It will be interesting to see if this deal closes and how it plays out. Based on NHS’s track record with Google-type outfits and Smartlogic-type innovators, I have a hunch that the outcome will be a case study of modern business processes.
Palantir needs many big wins to regain some stock market momentum. At least the Financial Times did not reference Palantir’s estimate of a 30 percent chance of nuclear war. Undoubtedly such a terrible event would stretch NHS’s capabilities regardless of technology vendor underpinning the outfit.
Stephen E Arnold, June 13, 2022
NSO Group: Here We Go Again
June 1, 2022
That Israeli outfit NSO Group has nailed the art of publicity. Positive PR? Nope. Not so positive? Yep. But as a wit allegedly said, “Any publicity is good publicity?”
Maybe.
“NSO’s Cash Dilemma: Miss Debt Repayment or Sell to Risky Customers” tries to explain some of NSO Group’s alleged activities. [This Financial Times’ article resides behind a paywall.] The write up states:
Hulio [one of NSO Group’s senior managers] said there was one option to bring in some cash quickly enough to pay salaries and service debt: reassemble a defunct internal committee and approve sales to customers flagged as “elevated risk” during due diligence.
Why is this allegation of money pressures sparking consideration of sales to nation states which may present some challenges to NSO Group, its managers and staff, and its investors?
My thought is that money must be followed.
A pursuit of money sparked some actions at other search and content processing centric companies. I mentioned this idea in my recent essay “Autonomy Business Details: Are These Relevant to Search- and Content Processing Type Outfits Today?”
The decision to generate revenues seems to open the door for many ideas. Some of these are okay; for example, selling more licenses to governments of NATO countries. A few may have been less well received; for example, relaxing the criteria used to determine what countries could license Israeli surveillance innovations.
US sanctions and the PR cyclone have created a number of business challenges for NSO Group. The path forward according to the Financial Times’ article looks like this:
In recent months, Hulio has come up with a new plan dubbed the “phoenix plan” by company insiders. The idea is to split NSO’s greatest assets from its greatest liabilities — this meant separating the code behind Pegasus and company engineers who are highly paid graduates of Israel’s elite military intelligence units, from the clients that have drawn the ire of the US and human rights groups. Hulio and a group of creditors hope that by spinning out a new entity that houses the code and engineers, it can sidestep the commerce department’s blacklist, especially if a new owner were a top US defence contractor.
What’s the outlook for NSO Group? Three possibilities strike me:
- Other companies will fill the gap. Just as Cellebrite has to deal with an upstart iPhone penetration solution, NSO Group will find that its methods provide a springboard to other innovators.
- NSO Group gets folded into a government agency. One can be sure it will not be a part of a nation state with negative thoughts about Israel.
- NSO Group folds its tent, and certain senior managers and engineers set up another company and move on.
I want to mention that the reason there is a glass ceiling for revenues from intelware and policeware is that there are a finite number of customers for the number of products and services on offer. Once that glass ceiling bumps the head of senior managers and stakeholders, then what I see as “drastic” actions kick in. Are Palantir’s comments about nuclear war and example of this?
I am certain about one thing: NSO Group is one of the most recognized brands of intelware in the world.
Stephen E Arnold, June 1, 2022
Autonomy Business Details: Are These Relevant to Search- and Content Processing Type Outfits Today?
May 31, 2022
I read “Judge Details Lynch’s $700k Signoff via iPhone Text in Full Autonomy Judgement.” The main idea is that Autonomy — an early entrant in the smart software for search and content processing — engaged in some business practices which a British judge finds suggestive. How suggestive? I am not sure, but the idea of using resellers and transactions to amp up revenues is interesting.
Another search and content processing outfit called Fast Search & Transfer (which Microsoft acquired more than a decade ago) found itself subject to some scrutiny for financial fancy dancing. One of the firm’s founders was found guilty and may have spent some time in the custody of a government. Maybe the fellow was cross country skiing and shooting a rifle at snow bunnies.
The relevance of the cited story and the reference to skis and weapons reminds me that the financial reports of high-flying search and content processing companies have to be scrutinized. I mention this because some of the more interesting search and content processing centric companies are publicly traded. Palantir Technologies comes to mind because I have seen a couple of semi-optimistic write ups about the company.
If I were a more youthful 77 year old, I would muster the energy to:
- Investigate the US government and UK government contracts for term, sunset dates, and contracting officers (what’s the background of these individuals)
- Research the question, “What’s bundled into the basic commercial and the basic government deal?”
- Explore the question, “How is cost of sales reacting to the economic climate since Palantir went public?”
- Try to determine answers to these questions: “What’s the ratio of sales people to programmers? The ratio of full time equivalents to contractors? How has the ratio changed since the firm went public?”
- Interview some people at LE and intel conferences to get a sense of the chatter related to this question: “Is Palantir bundling Amazon cloud services or doe the licensee have a choice?” and “Has there been talk of Palantir providing a “system in a box” to licensees with this requirement?
Why think about these types of questions? Oh, I am just curious about search and content processing outfits.
Stephen E Arnold, May 31, 2022
The Business Intelligence Blind Spot: Everyone Needs These Systems
May 30, 2022
I recall that a booth called “Business Blind Spots” identified a number of behaviors which contribute to business missteps. Staff, preconceived notions, market receptivity, etc. were among the points I recall.
I want to toss one more blind spot into the raging fire of burned cash, torched reputations, and incinerated opportunities. I call this bling spot, “Everybody needs these systems.” Plug in your own “systems”; for example, software that manages several cloud accounts which are guaranteed to blow through budget assumptions with no easy way to control the rising expenses.
I read “Palantir Stock: Getting Desperate.” I think the write up has been riding the well-worn fire trail to a burning coal mine.
Palantir Technologies is when the charities, the razzle dazzle, and the jargon are stripped away, is a search and retrieval company. The idea is that a person looking for information about a bad actor, for instance, can plug in the name and see results.
Now this seems like a function which is readily available from many vendors. The twist for Palantir is that it positioned its search as one that would meet the needs of intelligence officers. The US government entity embracing Palantir’s software influenced the add-ons; for example, the ability to ingest certain types of content that only government agencies could acquire.
In order to make sales, the marketing engine of Palantir came up with the same type of “latest and greatest” verbiage that characterizes intelware (that’s software built around the specific needs of intelligence analysts). One example is importing proprietary file types. Another is keeping track of where a dataset came from, who fiddled with it, and what an authorized user did with the data when in search mode.
Over time, companies which serve government agencies have to choose one of three paths:
- Path 1 is to just do commercial work. Forget the intelligence market. A company which has moved in this direction is one you may not know anything about. It is LifeRaft. Look them up. Now the company does market and ad intelligence for commercial companies, ad agencies, and probably some non profit outfits.
- Path 2 is to just focus on government sales. An example of this type of outfit is BAE Systems which has software able to do Palantir type functions. I am not sure BAE Systems returns phone calls from a bank or real estate agency wanting some Detica goodness.
- Path 3 is to do both. The best example of this is Voyager Labs which does the LifeRaft type work and the intelligence and law enforcement work of outfits like Palantir.
Which is the right path?
From my point of view, a company selling intelware should stick to government clients, maintain a low profile, and keep systems and methods secret. LifeRaft told me, “Don’t even mention our firm at the 2022 National Cyber Crime Conference.” Why? Doing work for certain government agencies gives some commercial firms and their go-go decision makers the heebie jeebies. The fear comes from folks who are interacting with investigators, intelligence operatives, and analysts could say something that will create big time thunderstorms for the commercial company. Some businesses are not exactly paragons of behavior. This means that the purchase cycle is drawn out, excuses are made, concerns about confidentiality raised, and weirdness about the amount of training, customizing, and optimizing the intelware system requires. The result? Some pretty crazy attempts to sell the product and the resulting disconnect from promises of reality from the commercial sector and the inevitable gap. This type of “gap” created some interesting situations in the decade or so.
What about government sales? Unless a company is selling hardware, software, spare parts, training, and services governments a fickle. Sure, an intelware outfit like Palantir will get initial contracts. But the government agencies have roving eyes and will keep licensing, looking for the perfect solution to intel needs. What happens is that the software only vendor runs out of customers. Once a number of big agencies sign up, the US General Services Administration or the Defense Services Administration will start angling for a deal. Cut the fees or lose the contracts. This is bad news because expensive software takes time to sell to government customers who want a demo or a year of free or discounted use in order to figure out if the system actually works. The problem is that There are not that many government agencies in the free world to support the intelware companies hungry for allocated budget dollars. Stated another way, the intelware company has to get some contracts, make the software work, and forget about the hockey stick financial projections. The intelware vendors chase US allies, but there are vendors in those countries, and it may make more sense to license Trendalyze or Verint, not the Silicon Valley type outfit. Bad financial news? Yep.
Path three is to sell to anyone who wants the system. This is very, very difficult because the intelware system has to be fiddled with in order to meet the specific requirements of an organization. Chasing bad actors is one thing; figuring out what type of beverage a college student wants is another thing. Hanging over the commercial sales call is the concern about the government work, the government customers, and the government processes, which — once started — are tough to turn off.
This means that companies crafted for intelware users find that government sales slow down, commercial sales cycles take a long time and often end up at a dead end, and non government organizations don’t want or can’t pay big bucks for what is search software.
The market itself is changing. If you want to analyze tweets, hire a marketing agency and get rid of them once they have completed a project. Clean, tidy, easy. If a client has some Google grade programmers, download Maltego, license the $100 Hunchly, and spend some time looking at tools on GitHub. (Thank you, Microsoft, but do you know what’s on that service? I thought so.)
The cited article makes this point:
…the company must expand internationally. What better way to get new sales than to start fires and be the person to sell the smoke detectors? That is what Palantir’s software does, assess and analyze data for threats. It is a loose analogy but fitting. But why is Palantir in such desperate need of expansion to new governments and industries? It is because the only thing keeping the stock going is the revenue growth rate which has been so strong. The company has incurred losses every year of operation. It expects operating expenses to increase.
And what about international sales? Three points:
- There are vendors offering comparable or better systems so buying non-US may make economic and political sense
- The cost of closing deals internationally is — the last time I checked — two to three times the cost of selling from Chicago to US based customers
- The number of purchasers is not as large as one thinks? The US is the living embodiment of Parkinson’s Law and the Peter Principle. Other countries are not much better and they have less disposable cash.
Net net: The word desperate may be appropriate for Palantir Technologies. I don’t have a good set of options for the company: Too much hype, too much development cost, too much customizing and tuning and training, and too much nuke talk. Not helpful.
Stephen E Arnold, May 30, 2022
On Mitigating Open-Source Vulnerabilities
May 16, 2022
Open-source software has saved countless developers from reinventing the proverbial wheel so they can instead spend their time creating new ways to use existing code. That’s great! Except for one thing: Now that open-source components make up about 90% of most applications, they pose tempting opportunities for hackers. Perhaps the juiciest targets lie in the military and intelligence communities. US counter-terrorism ops rely heavily on the likes of Palantir Technologies, a heavy user of and contributor to open-source software. Another example is the F-35 stealth fighter, which operates using millions of lines of code. A team of writers at War on the Rocks explores “Dependency Issues: Solving the World’s Open-Source Software Security Problem.” Solve it? Completely? Right, and there really is a tooth fairy. The article relates:
“The problem is that the open-source software supply chain can introduce unknown, possibly intentional, security weaknesses. One previous analysis of all publicly reported software supply chain compromises revealed that the majority of malicious attacks targeted open-source software. In other words, headline-grabbing software supply-chain attacks on proprietary software, like SolarWinds, actually constitute the minority of cases. As a result, stopping attacks is now difficult because of the immense complexity of the modern software dependency tree: components that depend on other components that depend on other components ad infinitum. Knowing what vulnerabilities are in your software is a full-time and nearly impossible job for software developers.”
So true. Still, writers John Speed Meyers, Zack Newman, Tom Pike, and Jacqueline Kazil sound optimistic as they continue:
“Fortunately, there is hope. We recommend three steps that software producers and government regulators can take to make open-source software more secure. First, producers and consumers should embrace software transparency, creating an auditable ecosystem where software is not simply mysterious blobs passed over a network connection. Second, software builders and consumers ought to adopt software integrity and analysis tools to enable informed supply chain risk management. Third, government reforms can help reduce the number and impact of open-source software compromises.”
The article describes each part of this plan in detail. It also does a good job explaining how we got so dependent on open-source software and describes ways hackers are able to leverage it. The writers submits that, by following these suggestions, entities both public and private can safely continue to benefit from open-source collaboration. If the ecosystem is made even a bit safer, we suppose that is better than nothing. After all, ditching open-source altogether seems nigh impossible at this point.
Cynthia Murrell, May 16, 2022
Voyager Labs Exposed: Another NSO Group?
May 10, 2022
I read “Voyager Labs: L’Arma spuntata dell’intelienza artificiale.” I was expecting some high-flying smart software. What the article delivers is some juicy detail about intelware, conferences where quite non-public stories are told, and an alleged tie up between those fine folks at Palantir Technologies and the shadowy Israeli company. One caveat: One has to be able to read Italian or have a way to work around the limitations of online translation systems. (Good luck with finding a free to use system. I just asked my local Pizza Hut delivery person, who speaks and reads Italian like a Roma fan.)
Here are some allegedly spot on factoids from the write up:
- One of the directors of the company has a remarkably unusual career at a US government agency. The individual presided over specialized interrogation activities and allowing a person with a bomb to enter a government facility. There were a handful of deaths.
- The Voyager Labs’ cloud services are allegedly “managed globally by Palantir’s Gotham platform.
- Voyager’s Labs’ content was described at an intelligence conference owned and managed by an American in this way: “usable and previously unattainable information by analyzing and understanding huge amounts of open, deep and obscure Web data.”
- Allegations about the use of Voyager Labs’ system to influence an Italian election.
- Voyager Labs identifies for licensees people with red, orange, and green icons. Green is good; red is bad; orange is in the middle?
Interesting stuff. But the zinger is the assertion that Voyager Labs’ smart software can output either dumb or aberrant results. The whiz kids at Gartner Group concluded in 2017 that Voyager Labs was a “cool vendor.” That’s good to know. Gartner likes intelware that sort of works. Cool.
Interesting profile and there are more than 100 footnotes. I assume that the founder of Voyager Labs, the conference organizer, and assorted clients were not will to participate in an interview. This is an understandable position, particularly when an Israeli outfit could be the next in the NSO Group spotlight.
Stephen E Arnold, May 10, 2022
Google: Nosing into US Government Consulting
April 4, 2022
I spotted an item on Reddit called “Google x Palantir.” Let’s assume there’s a smidgen of truth in the post. The factoid is in a comment about Google’s naming Stephen Elliott as its head of artificial intelligence solutions for the Google public sector unit. (What happened to the wizard once involved in this type of work? Oh, well.)
The interesting item for me is that Mr. Elliott will have a particular focus on “leveraging the Palantir Foundry platform.” I thought that outfits like Praetorian Digital (now Lexipol) handled this type of specialist consulting and engineering.
What strikes me as intriguing about this announcement is that Palantir Foundry will work on the Google Cloud. Amazon is likely to be an interested party in this type of Google initiative.
Amazon has sucked up a significant number of product-centric searches. Now the Google wants to get into the “make Palantir work” business.
Plus, Google will have an opportunity to demonstrate its people management expertise, its ability to attract and retain a diverse employee group, and its ability to put some pressure on the Amazon brachial nerve.
How will Microsoft respond?
The forthcoming Netflix mockumentary “Mr. Elliot Goes to Washington” will fill someone’s hunger for a reality thriller.
And what if the Reddit post is off base. Hey, mockumentaries can be winners. Remember “This Is Spinal Tap”?
Stephen E Arnold, April 4, 2022
Anduril Victorious with SOCOM Contract
February 25, 2022
Tech startups, and the venture capitalists that back them, have been trying valiantly to break the chains of traditional government procurements. Pointing to a recent nearly billion-dollar deal, Breaking Defense ponders, “Anduril Nets Biggest DoD Contract to Date: Signifier or Outlier for Defense Start-Ups?” Anduril is based in Irvine, California, and was founded in 2017. The surveillance and military tech company beat out 11 others competing for the lucrative contract with Special Operations Command (SOCOM). Reporter Andrew Eversden writes:
“Anduril will serve as a systems integrator partner on SOCOM’s counter-unmanned systems efforts. The contract is worth a maximum of $967,599,957 over the next the decade. Under the contract, SOCOM will be able to purchase Anduril’s systems through traditional means, in addition to buying Anduril’s products as a service, meaning the command can configure the system ‘based on mission profiles and ensuring SOCOM can rapidly adapt to new and evolving threat profiles.’ According to the company press release, the company will ‘deliver, advance, and sustain CUxS capabilities for special operations forces wherever they operate.’ It will provide counter-drone capability through its Lattice AI platform, which is designed to autonomously identify and classify threats. The system will be deployed both domestically and overseas, the Jan. 20 announcement stated. Anduril has made major strides in the last year positioning itself to win major defense contracts and augment its technology portfolio. Last year, it acquired Area-I, a tube-launched unmanned aerial system maker. Last summer, the company won a five-year, $99 million production other transaction agreement with the Pentagon’s Defense Innovation Unit for its counter-drone tech. In September, it bought Copious Imaging, whose technology added another layer of threat detection to Anduril’s air defense portfolio.”
We also note the firm had the honor of collaborating with Palantir on the Army’s Tactical Intelligence Targeting Access Node (TITAN) prototype last year. Tech executives and investors have expressed frustration at the challenges of doing business with our military, but this latest contract may be a signal that startups and other non-giant companies can make their way in the federal marketplace after all. On the other hand, we are told, SOCOM has long been the DoD division most likely to embrace innovative, non-traditional partners. If this contract goes well, perhaps SOCOM’s forward-thinking perspective will spread to other agencies. No pressure, Anduril.
Cynthia Murrell, February 25, 2022
-
- Subscribe to Beyond Search
Feature archive
News archive
-
