Palantir Pushes Beyond What Any Other System Can Do It Seems
August 13, 2021
I believe everything I read online. Don’t you. I spotted this interesting article: “Palantir: Revolutionizing Big Data Analytics.” The write up shows a Covid dashboard and focuses on what’s called “data integration.” Putting information in an index or series of indexes so a user or software can run a query across that which has been placed in said indexes is sometimes called “federation”. Without entering a rabbit hole, let’s accept the “data integration” idea and ignore the buzzwords like “cross function collaborations.”
The Palantir system has a four step “process flow.” These steps include:
- Aggregating data
- Transforming data
- Securing data
- Empowering data.
I track with the first three steps, which have been required by policeware and intelware systems for decades.
The baffler is “empowering” data. I think this means that Palantir data are more valuable, potent, or muscular than data in a system for which I was a consultant many years ago. That was the i2 Analysts Notebook from the late 1990s.
That’s neither here nor there because Palantir did the Silicon Valley thing and found inspiration in that pioneering i2 system, which is now owned by IBM.
But here’s the statement in the write up that left me scratching my head:
Palantir is different from traditional business intelligence solutions like Tableau, Alteryx, or Cloudera, as it’s able to answer questions that a regular model isn’t able to. Questions such as “What steps should be taken if there’s another global pandemic”, or “How to increase margins in the most effective way”.
The companies cited in the passage are not intelware or policeware centric. Second, Palantir seems to be able to process natural language queries, extract on point facts and data from the aggregated and transformed data, and deliver answers.
As far as I know, NLP system do not reliably field ad hoc questions about general business issues or warfighting/intelligence issues. If systems did, there would not be the grousing about training, complexity, and disused intelware due to complexity and instability.
I don’t want to suggest that Palantir cannot deliver NLP which works. I would like to gently suggest that this just may not work in a way which would be useful in certain situations.
I understand the reasons “traditional” intelware fails. Managing data and logic together is tricky and made more challenging and expensive because real time streams can be ingested into some intelware systems. Specialists exist to deal with the real time challenge, and I am not sure Palantir has the robustness of Trendalyze, for example.
The data integrity issue is a big deal. Palantir makes it possible to know who input data. But the integrity issue is larger than than a single person. There are vendors who assemble data sets. Automated data sets work okay too, but when a stream is lost from an authorized intercept, the data set takes a hit. Plus, there is just bad data; for example, variable mechanisms for counting Covid deaths. Has Palantir whipped this garbage in problem? Maybe.
One weakness of Palantir’s competitors is described this way:
The inability to define key business metrics transparently in a common data foundation
This is an ambiguous statement. Most managers don’t know what they need or want. A case in point is a cyber security vendor offering phishing protection to clients. What happens if phishing techniques rely on auto generated emails with smart software crafting the pitch and the inclusion of valid links to the recipient’s company’s Web site. How is an employee to recognize these malformed email? We know phishing systems are not working because of the notable breaches in the US and elsewhere in the last six months of 2021. Senior managers want answers, and hopefully the answers are “good” or at least don’t lead to a diplomatic crisis or a severe business impact. Has Palantir cracked the problem of people who say, “I know what I want when I see it.” In my experience, quite a few CxOs rely on this method. Unfortunately this is not 1690 in Rhode Island where the vigilant are on the look out for irritated Native Americans. Recognizing that eye ball glimmering in a bush is not something intelware systems are able to do in a reliable, economical, speedy way.
Finally, the Palantir competitors “lack flexibility due to rigid data assets.” I remember the sales pitch of MarkLogic, a vendor of slicing-and-dicing content systems. The idea is that XML was almost magical. Input parameters and one gets output like a book made up of relevant content from the objects in the database. XML is a useful tool, but based on my experience with intelware systems, most of them use structured files, open source software, and the same popular algorithms taught in CompSci 401 around the world. The flexibility issue is a big one because now intelware must make sense of audio, video, pictures, gifs, database files, proprietary files from legacy systems, consumer file types like Word, and numeric streams. The phrase “rigid data assets” does quite capture the nuances of the data chaos facing most organizations.
Net net: This is an interesting write up, but I think it needs evidence, and substantive information. Palantir certainly has magnetism, but I still ask myself:
Why is Palantir funding SPACs and allegedly requiring these firms to agree to license the Palantir system?
This is a mystery to me. Because if Palantir whipped NLP, for instance, or the data chaos problem, the company would the hottest thing since i2 Analysts Notebook.
Stephen E Arnold, August 13, 2021
Comments
One Response to “Palantir Pushes Beyond What Any Other System Can Do It Seems”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
You should check out Whitney Webb’s articles on Palantir and the people behind it.