CyberOSINT banner


Palantir Technologies Challenges US Government Procurement

I was a wee lad when I read Don Quixote. I know that students in Spain and some other countries study the text of the 17th century novel closely. I did not. I remember flipping through a Classics’ comic book, reading the chapter summaries in Cliff’s Notes, and looking at the pictures in the edition in my high school’s library. Close enough for horse shoes. (I got an A on the test. Heh heh heh.)

Here’s what I recall the Don and his sidekick. A cultured fellow read a lot of fantasy fiction, mixed it up the real world, and went off on adventures or sallies. The protagonist (see I remember something from Ms. Sperling’s literature class in 1960) rode a horse and charged into the countryside to kill windmills. I remember there were lots and lots of adventures, not too much sex – drugs – rock and roll, and many convoluted literary tropes.

I still like the windmills. A Google search showed me an image which is very similar to the one in the comic book I used as my definitive version of the great novel. Here it is:

Image result for don quixote windmills

What does a guy riding a horse with a lance toward a windmill have to do with search and content processing? Well, I read “Palantir Lambastes Army Over $206 Million Contract Bidding.” I assume the information in the write up is spot on.

Palantir Technologies, a unicorn which is the current fixation of a Buzzfeed journalist, is going to sue the US Army over a “to be” contract for work. The issue is an all source information system procurement known as DCGS or sometimes DI2E. The acronyms are irrelevant. What is important is that the US Army has been plugging away with a cadre of established government contractors for a decade. Depending on whom one asks, DCGS is the greatest thing since sliced bread or it is a flop.

However, Palantir believes that its augmented intelligence system is a better DCGS / DI2E. than the actual DCGS / DI2E.

The US Army may not agree and appears be on the path to awarding the contract for DCGS work to other vendors.

According to the write up:

Palantir claims the Army’s solicitation is “unlawful, irrational, arbitrary and capricious,” according to the letter of intent Palantir sent to the U.S. Army and the Department of Justice, which was obtained by Bloomberg. The letter is a legal courtesy, which states Palantir will file a formal protest in the U.S. Court of Federal Claims next week and requests the Army delay awarding the first phase of the contract until litigation is resolved. The contract is slated to be awarded by the end of 2016.

The contract is worth a couple of hundred million, but the follow on work is likely to hit nine figures. Palantir has some investors who want more growth. The best way to get it, if the write up is accurate, is on the backs of legal eagles.

I don’t know anything about the US Army and next to nothing about Palantir, but I have some experience watching vendors protest the US government’s procurement process. My thought is that when bidders sue the government:

  • Costs go up. Lawyers are very busy, often for a year or more. In lawyer land, billing is really good.
  • Delays occur. The government unit snagged in the contracting hassle have to juggle more balls; for example, tasks have to be completed. When the vendors are not able to begin work, delays occur. This may not be a problem in lawyer land, but in the real world, downstream dependencies can be a hitch in the git along.
  • Old scores may be hummed. Palantir settled a legal dust up with IBM which owns i2 Analysts Notebook. The Analysts Notebook is the very same software system whose file structure Palantir wanted to understand. i2 was not too keen on making its details available. (Note: I was a consultant to i2 for a number of years, and this was input number one to me from one of the founders). IBM has a pretty good institutional memory without consulting Watson.)

And Don Quixote? I wonder if the Palantirians, some of whom fancy themselves Hobbits, are going to be able to shape the real world to their vision. The trajectory of this legal dust up will be interesting to watch as it flames across the sky toward Spain and Don Quixote’s fictional library. Flame out or direct hit? The US Army and US government procurement policies are able to absorb charging horses and possibly a lance poke or two.

Stephen E Arnold, June 22, 2016


Exclusive Interview: Danny Rogers, Terbium Labs

Editor’s note: The full text of the exclusive interview with Dr. Daniel J. Rogers, co-founder of Terbium Labs, is available on the Xenky Cyberwizards Speak Web service at The interview was conducted on August 4, 2015.

Significant innovations in information access, despite the hyperbole of marketing and sales professionals, are relatively infrequent. In an exclusive interview, Danny Rogers, one of the founders of Terbium Labs, has developed a way to flip on the lights to make it easy to locate information hidden in the Dark Web.

Web search has been a one-trick pony since the days of Excite, HotBot, and Lycos. For most people, a mobile device takes cues from the user’s location and click streams and displays answers. Access to digital information requires more than parlor tricks and pay-to-play advertising. A handful of companies are moving beyond commoditized search, and they are opening important new markets such as secret and high value data theft. Terbium Labs can “illuminate the Dark Web.”

In an exclusive interview, Dr. Danny Rogers, one of the founders of Terbium Labs with Michael Moore, explained the company’s ability to change how data breaches are located. He said:

Typically, breaches are discovered by third parties such as journalists or law enforcement. In fact, according to Verizon’s 2014 Data Breach Investigations Report, that was the case in 85% of data breaches. Furthermore, discovery, because it is by accident, often takes months, or may not happen at all when limited personnel resources are already heavily taxed. Estimates put the average breach discovery time between 200 and 230 days, an exceedingly long time for an organization’s data to be out of their control. We hope to change that. By using Matchlight, we bring the breach discovery time down to between 30 seconds and 15 minutes from the time stolen data is posted to the web, alerting our clients immediately and automatically. By dramatically reducing the breach discovery time and bringing that discovery into the organization, we’re able to reduce damages and open up more effective remediation options.

Terbium’s approach, it turns out, can be applied to traditional research into content domains to which most systems are effectively blind. At this time, a very small number of companies are able to index content that is not available to traditional content processing systems. Terbium acquires content from Web sites which require specialized software to access. Terbium’s system then processes the content, converting it into the equivalent of an old-fashioned fingerprint. Real-time pattern matching makes it possible for the company’s system to locate a client’s content, either in textual form, software binaries, or other digital representations.

One of the most significant information access innovations uses systems and methods developed by physicists to deal with the flood of data resulting from research into the behaviors of difficult-to-differentiate sub atomic particles.

One part of the process is for Terbium to acquire (crawl) content and convert it into encrypted 14 byte strings of zeros and ones. A client such as a bank then uses the Terbium content encryption and conversion process to produce representations of the confidential data, computer code, or other data. Terbium’s system, in effect, looks for matching digital fingerprints. The task of locating confidential or proprietary data via traditional means is expensive and often a hit and miss affair.

Terbium Labs changes the rules of the game and in the process has created a way to provide its licensees with anti-fraud and anti-theft measures which are unique. In addition, Terbium’s digital fingerprints make it possible to find, analyze, and make sense of digital information not previously available. The system has applications for the Clear Web, which millions of people access every minute, to the hidden content residing on the so called Dark Web.


Terbium Labs, a start up located in Baltimore, Maryland, has developed technology that makes use of advanced mathematics—what I call numerical recipes—to perform analyses for the purpose of finding connections. The firm’s approach is one that deals with strings of zeros and ones, not the actual words and numbers in a stream of information. By matching these numerical tokens with content such as a data file of classified documents or a record of bank account numbers, Terbium does what strikes many, including myself, as a remarkable achievement.

Terbium’s technology can identify highly probable instances of improper use of classified or confidential information. Terbium can pinpoint where the compromised data reside on either the Clear Web, another network, or on the Dark Web. Terbium then alerts the organization about the compromised data and work with the victim of Internet fraud to resolve the matter in a satisfactory manner.

Terbium’s breakthrough has attracted considerable attention in the cyber security sector, and applications of the firm’s approach are beginning to surface for disciplines from competitive intelligence to health care.

Rogers explained:

We spent a significant amount of time working on both the private data fingerprinting protocol and the infrastructure required to privately index the dark web. We pull in billions of hashes daily, and the systems and technology required to do that in a stable and efficient way are extremely difficult to build. Right now we have over a quarter trillion data fingerprints in our index, and that number is growing by the billions every day.

The idea for the company emerged from a conversation with a colleague who wanted to find out immediately if a high profile client list was ever leaded to the Internet. But, said Rogers, “This individual could not reveal to Terbium the list itself.”

How can an organization locate secret information if that information cannot be provided to a system able to search for the confidential information?

The solution Terbium’s founders developed relies on novel use of encryption techniques, tokenization, Clear and Dark Web content acquisition and processing, and real time pattern matching methods. The interlocking innovations have been patented (US8,997,256), and Terbium is one of the few, perhaps the only company in the world, able to crack open Dark Web content within regulatory and national security constraints.

Rogers said:

I think I have to say that the adversaries are winning right now. Despite billions being spent on information security, breaches are happening every single day. Currently, the best the industry can do is be reactive. The adversaries have the perpetual advantage of surprise and are constantly coming up with new ways to gain access to sensitive data. Additionally, the legal system has a long way to go to catch up with technology. It really is a free-for-all out there, which limits the ability of governments to respond. So right now, the attackers seem to be winning, though we see Terbium and Matchlight as part of the response that turns that tide.

Terbium’s product is Matchlight. According to Rogers:

Matchlight is the world’s first truly private, truly automated data intelligence system. It uses our data fingerprinting technology to build and maintain a private index of the dark web and other sites where stolen information is most often leaked or traded. While the space on the internet that traffics in that sort of activity isn’t intractably large, it’s certainly larger than any human analyst can keep up with. We use large-scale automation and big data technologies to provide early indicators of breach in order to make those analysts’ jobs more efficient. We also employ a unique data fingerprinting technology that allows us to monitor our clients’ information without ever having to see or store their originating data, meaning we don’t increase their attack surface and they don’t have to trust us with their information.

For more information about Terbium, navigate to the company’s Web site. The full text of the interview appears on Stephen E Arnold’s Xenky cyberOSINT Web site at

Stephen E Arnold, August 11, 2015

Latest News

Weakly Watson: Real Journalists May Be Remaindered

That IBM Watson is a versatile confection. I read “IBM’s Watson Tries Its Hand at Editing a Magazine.” Not only does software have a hand, the software... Read more »

June 25, 2016 | | Comment

Enterprise Search Vendors: A Partial List

I spoke with a confused and unbudgeted worker bee at a giant outfit this weekend. The stellar professional was involved in figuring out what to do about enterprise... Read more »

June 24, 2016 | | Comment

Forbes, News Coverage, and Google Love

Short honk: US news coverage has “faves.” I assume that the capitalist tool avoids bias in its admirable reporting about business. Navigate to “Television... Read more »

June 24, 2016 | | Comment

Self Driving Bicycle Prank Points to Wider Arena

Technology companies are no stranger to making April Fools’ pranks in the form of media releases. This year, The Inspiration Room shared an article highlighting... Read more »

June 24, 2016 | | Comment

Stepes: Human Translation at Your Fingertips

Though today’s machine translation is a convenient way to quickly get the gist of a foreign-language passage, it has its limitations; professionals still turn... Read more »

June 24, 2016 | | Comment

Palantir Technologies: Now Beer Pong and Human Augmented Intelligence?

I went months, nay years, without reading very much about Palantir Technologies. Now the unicorn seems to be prancing through my newsfeeds frequently. I read “Palantir’s... Read more »

June 23, 2016 | | Comment

Factoid: Need a Fast Computer? Buzz China.

I read “China Makes New Supercomputing Gains.” (You may have to pay real money, not the Ethereum stuff to read the article.) The main idea is that China has... Read more »

June 23, 2016 | | Comment

Peak Unicorn: Hooves of Doom

I loved the phrase “peak unicorn.” The co9mbination of mixed metaphors and a mythical horned equine is delicious. Navigate to “The Unicorn Godmother Dishes... Read more »

June 23, 2016 | | Comment

Who Will Connect the Internet of Things to Business

Remember when Nest Labs had all the hype a few years ago? An article from BGR reminds us how the tides have turned: Even Google views its Nest acquisition as a disappointment.... Read more »

June 23, 2016 | | Comment

ZyLab Places eDiscovery in the Cloud

Through their Press Room site, ZyLab announces, “Zylab Introduces eDiscovery as a Service.” Billed as a cost-saving alternative to in-house solutions, the new... Read more »

June 23, 2016 | | Comment