Picking and Poking Palantir Technologies: A New Blood Sport?

April 25, 2018

My reaction to “Palantir Has Figured Out How to Make Money by Using Algorithms to Ascribe Guilt to People, Now They’re Looking for New Customers” is a a sign and a groan.

I don’t work for Palantir Technologies, although I have been a consultant to one of its major competitors. I do lecture about next generation information systems at law enforcement and intelligence centric conferences in the US and elsewhere. I also wrote a book called “CyberOSINT: Next Generation Information Access.” That study has spawned a number of “experts” who are recycling some of my views and research. A couple of government agencies have shortened by word “cyberosint” into the “cyint.” In a manner of speaking, I have an information base which can be used to put the actions of companies which offer services similar to those available from Palantir in perspective.

The article in Boing Boing falls into the category of “yikes” analysis. Suddenly, it seems, the idea that cook book mathematical procedures can be used to make sense of a wide range of data. Let me assure you that this is not a new development, and Palantir is definitely not the first of the companies developing applications for law enforcement and intelligence professionals to land customers in financial and law firms.

baseball card part 5

A Palantir bubble gum card shows details about a person of interest and links to underlying data from which the key facts have been selected. Note that this is from an older version of Palantir Gotham. Source: Google Images, 2015

Decades ago, a friend of mine (Ev Brenner, now deceased) was one of the pioneers using technology and cook book math to make sense of oil and gas exploration data. How long ago? Think 50 years.

The focus of “Palantir Has Figured Out…” is that:

Palantir seems to be the kind of company that is always willing to sell magic beans to anyone who puts out an RFP for them. They have promised that with enough surveillance and enough secret, unaccountable parsing of surveillance data, they can find “bad guys” and stop them before they even commit a bad action.

Okay, that sounds good in the context of the article, but Palantir is just one vendor responding to the need for next generation information access tools from many commercial sectors.

Read more

French Report: Demographics of Bad Actors

April 20, 2018

If this report is to be believed, a demographic segment may come under increased scrutiny, online and offline. The Local lays out the results of a recent study from the director of publications at the French Institute of International Relations, Marc Hecker, in the write-up, “Aged 26, Poor, and Already a Criminal: Who Is the Typical French Jihadist?” The sample size was not large—137 people who had been convicted of Islamist- or jihad-related terror offences in France, most of whom were French nationals. The write-up shares a number of specific findings, including this bit of interest to IT folks: Though jihadists do use the internet extensively for networking and coordination, most become radicalized through extensive in-person contact; they have not been enticed simply by material found online. The article also reports:

“Out of the 137 cases, 131 were men and six were women. The average age of those at the time they were charged was 26, with 90 percent of them coming from large broken families and 40 percent coming from poor backgrounds. Of the 137 jihadists looked at by the study, some 74 percent were born Muslim while the remaining 26 percent converted during their lifetime. Although in general the study found there was a low level of religious knowledge among the individuals. Some 47 percent of the 68 French jihadists whose education records were available left school with no qualifications while 26 percent passed their baccalaureate and 11 percent graduated from university. Some 36 percent were unemployed at the time of arrest while another 22 percent were in low-paid unstable jobs (emploi précaire). More than half of those charged with Islamist terror offences were in a couple (57 percent). IFRI says the study shows “that these individuals are distinguished by a lower level of education and professional integration, a higher degree of poverty, a greater involvement in crime and a closer relationship to North and sub-Saharan Africa than the average population of France.”

One more key statistic—about 40 percent of respondents had already been convicted of at least one crime, while another 8 had been reported to police with no conviction. So, those already inclined toward criminality may be more likely to see violence as a viable tool for change; imagine that. The study also found that the process of radicalization takes months or, in 30 percent of cases, several years. A preview of Hecker’s study can be found here.

Cynthia Murrell, April 20, 2018

Quote to Note: Statistics May Spoil Like Bananas

April 13, 2018

I noticed this synopsis for a talk by Andrew Gelman, a wizard who teaches at Columbia University. You can find the summary in “Do Statistical methods Have an Expiration Date?” Here’s the quote I noted:

The statistical methods which revolutionized science in the 1930s-1950s no longer seem to work in the 21st century. How can this be? It turns out that when effects are small and highly variable, the classical approach of black-box inference from randomized experiments or observational studies no longer works as advertised.

What happens when these methods are bolted into next generation data analytics systems which humans use to make decisions? My great uncle (Vladimir.I. Arnold and his co worker Andrey Kolmogorov could calculate an answer I assume?)

Stephen E Arnold, April 13, 2018

Cambridge Analytica: The April 3, 2018, DarkCyber Report Is Now Available

April 3, 2018

DarkCyber for April 3, 2018, is now available. The new program can be viewed at www.arnoldit.com/wordpress and on Vimeo at https://vimeo.com/262710424.

This week’s program focuses on the Facebook, GSR, Cambridge Analytica data controversy. The 12 minute video addresses the role of GSR and the Cambridge professor who developed a personality profile app. The DarkCyber program outlines how raw social data is converted into actionable, psychographic “triggers.” By connecting individuals, groups, and super-groups with “hot buttons” and contentious political issues, behaviors can be influenced, often in an almost undetectable way.

The DarkCyber research team has assembled information from open source coverage of Cambridge Analytica and has created a generalized “workflow” for the Facebook-type data set. The outputs of the workflow are “triggers” which can be converted into shaped messages which are intended to influence behaviors of individuals, groiups, and super-groups.

The program explains how psychographic analyses differ from the more well known demographic analyses of Facebook data. The link analysis or social graph approach is illustrated in such a way that anyone can grasp the potential of this data outputs. The program includes a recommendation for software which anyone with basic programming skills can use to generate “graphs” of relationships, centers of influence, and individuals who are likely to take cues from these centers of influence.

DarkCyber’s next special feature focuses on the Grayshift GrayKey iPhone unlocking product. The air date will appear in Beyond Search.

Kenny Toth, April 3, 2018

DarkCyber Explores the Cambridge Analytica Matter

March 29, 2018

Short honk: The April 3, 2018, DarkCyber devotes the program to the Cambridge Analytica Matter. What makes this program different is the DarkCyber approach. The DarkCyber researchers examined open source information for factoids about how Cambridge Analytica created their “actionable” information for political clients. If you want to see a social media survey question can generate “triggers” to cause action via an image, a tweet, or blog post — tune in on April 3, 2018. Plus the program provides a link so you can download an application which can be used to generate “centers of influence”. Who knows? You could become the next big thing in content analysis and weaponizing information.

Make a note. On Tuesday, April 3, 2018, You will be able to view the video at www.arnoldit.com/wordpress or on Vimeo.

Kenny Toth, March 29, 2018

Cambridge Analytica and Fellow Travelers

March 26, 2018

I read Medium’s “Russian Analyst: Cambridge Analytica, Palantir and Quid Helped Trump Win 2016 Election.” Three points straight away:

  1. The write up may be a nifty piece of disinformation
  2. The ultimate source of the “factoids” in the write up may be a foreign country with interests orthogonal to those of the US
  3. The story I saw is dated July 2017, but dates – like other metadata – can be fluid unless in a specialized system which prevents after the fact tampering.

Against this background of what may be hefty problems, let me highlight several of the points in the write up I found interesting.

More than one analytics provider. The linkage of Cambridge Analytica, Palantir Technologies, and Quid is not a surprise. Multiple tools, each selected for its particular utility, are a best practice in some intelligence analytics operations.

A Russian source. The data in the write up appear to arrive via a blog by a Russian familiar with the vendors, the 2016 election, and how analytic tools can yield actionable information.

Attributing “insights.” Palantir allegedly output data which suggested that Mr. Trump could win “swing” states. Quid’s output suggested, “Focus on the Midwest.” Cambridge Analytica suggested, “Use Twitter and Facebook.”

If you are okay with the source and have an interest in what might be applications of each of the identified companies’ systems, definitely read the article.

On April 3, 2018, my April 3, 2018, DarkCyber video program focuses on my research team’s reconstruction of a possible workflow. And, yes, the video accommodates inputs from multiple sources. We will announce the location of the Cambridge Analytica, GSR, and Facebook “reconstruction” in Beyond Search.

Stephen E Arnold, March 26, 2018

What Happens When Intelligence Centric Companies Serve the Commercial and Political Sectors?

March 18, 2018

Here’s a partial answer:






Years ago, certain types of companies with specific LE and intel capabilities maintained low profiles and, in general, focused on sales to government entities.

How times have changed!

In the DarkCyber video news program for March 27, 2018, I report on the Madison Avenue type marketing campaigns. These will create more opportunities for a Cambridge Analytica “activity.”

Net net: Sometimes discretion is useful.

Stephen E Arnold, March 18, 2018

Crime Prediction: Not a New Intelligence Analysis Function

March 16, 2018

We noted “New Orleans Ends Its Palantir Predictive Policing Program.” The interest in this Palantir Technologies’ project surprised us from our log cabin with a view of the mine drainage run off pond. The predictive angle is neither new nor particularly stealthy. Many years ago when I worked for one of the outfits developing intelligence analysis systems, the “predictive” function was a routine function.

Here’s how it works:

  • Identify an entity of interest (person, event, organization, etc.)
  • Search for other items including the entity
  • Generate near matches. (We called this “fuzzification” because we wanted hits which were “near” the entity in which we had an interest. Plus, the process worked reasonably well in reverse too.)
  • Punch the analyze function.

Once one repeats the process several times, the system dutifully generates reports which make it easy to spot:

  • Exact matches; for example, a “name” has a telephone number and a dossier
  • Close matches; for example, a partial name or organization is associated with the telephone number of the identity
  • Predicted matches; for example, based on available “knowns”, the system can generate a list of highly likely matches.

The particular systems with which I am familiar allow the analyst, investigator, or intelligence professional to explore the relationships among these pieces of information. Timeline functions make it trivial to plot when events took place and retrieve from the analytics module highly likely locations for future actions. If an “organization” held a meeting with several “entities” at a particular location, the geographic component can plot the actual meetings and highlight suggestions for future meetings. In short, prediction functions work in a manner similar to Excel’s filling in items in a number series.

heat map with histogram

What would you predict as a “hot spot” based on this map? The red areas, the yellow areas, the orange areas, or the areas without an overlay? Prediction is facilitated with some outputs from intelligence analysis software. (Source: Palantir via Google Image search)

Read more

Schmidt Admits It Is Hard to Discern Between Fact and Fiction

March 15, 2018

One basic research essential is learning how to tell the difference between fact and fiction.  It used to be easier to control and verify news because information dissemination was limited to physical mediums.  The Internet blew everything out of the water and made it more difficult to discern fact and fiction.  Humans can be taught tricks, but AI still has a lot to learn.  The Daily Mail reports that, “Alphabet Chairman Eric Schmidt Admits It Is ‘Very Difficult’ For Google’s Algorithm To Separate Fact From Fiction In Its Search Results.”

Millions of articles and other content is posted daily online.  Google’s job is to sift through it and delivery the most accurate results.  When opposing viewpoints are shared, Google’s algorithm has difficulty figuring out the truth.  Eric Schmidt says that can be fixed with tweaking.  He viewed fact vs. fiction problems as bugs that need repair and with some work they can be fixed.  The article highlights some of the more infamous examples of Google’s failing such as the AutoComplete feature and how conspiracy theories can be regarded as fact.

Search results displaying only hard truth will be as elusive as accurate sentiment analytics.

Schmidt added:

That is a core problem of humans that they tend to learn from each other and their friends are like them.  And so until we decide collectively that occasionally somebody not like you should be inserted into your database, which is sort of a social values thing, I think we are going to have this problem.’

Or we can just wait until we make artificial intelligence smarter.

Whitney Grace, March 15, 2018

Come on Google, Stop Delivering Offensive Content

March 14, 2018

Sentiment analytics is notoriously hard to program and leads to more chuckles than accurate results.  Throughout the year, Google, Facebook, and other big names have dealt with their own embarrassing sentiment analytics fiascos and they still continue.  The Verge shares, “Google’s Top Search Results Promote Offensive Content, Again” in an unsurprising headline.

One recent example took an offensive meme from the swathe subreddit when “gender fluid” was queried and made it the first thing displayed.  Yes, it is funny, but stuff like this keeps happening without any sign of stopping:

The slip-up comes just a month after Google briefly gave its “top stories” stamp of approval to two 4chan threads identifying the wrong suspect in the recent Las Vegas mass shooting tragedy. This latest search result problem appears to be related to the company’s snippet feature. Featured snippets are designed to answer queries instantly, and they’ve often provided bad answers in the past. Google’s Home device, for example, used a featured snippet to answer the question ‘are women evil?’ with the horrendously bad answer ‘every woman has some degree of prostitute in her.’

The ranking algorithm was developed to pull the most popular stories and deliver them regardless of their accuracy.  Third parties and inaccurate sources can manipulate the ranking algorithm for their own benefit or human.  Google is considered the de facto source of information.  There is a responsibility of purveying the truth, but there will always be people who take advantage of the news outlets.

Whitney Grace, March 14, 2018

Next Page »

  • Archives

  • Recent Posts

  • Meta