Phrase of the Day: Collateral Damage
June 14, 2018
The phrase “collateral damage” means, according to the Cambridge Dictionary:
during a war, the unintentional deaths and injuries of people who are not soldiers, and damage that is caused to their homes, hospitals, schools, etc.
Cambridge University itself may be touched by blowback from the antics of one of its professors and a company which shares the name of the town on the River Cam. Twitch the mantle blue, of course.
The Cambridge Analytica/Facebook data scandal has rightly been scrutinized by everyone from individual users to entire government bodies. As could be expected when the players are this large, what people are finding links together unlikely suspects and victims in this data breach. One such surprise popped up this week when we read a Gizmodo report, “Facebook ‘Looking Into’ Palantir’s Access to User Data.”
According to the story:
“The inquiry was led by Damian Collins, chair of Parliament’s Digital, Culture, Media, and Sport Committee. According to CNBC, Collins asked if Palantir was part of Facebook’s “review work…. While it’s unclear if it gained access to the Facebook user data that Cambridge Analytica harvested, Palantir’s connection to the social network extends beyond any potential collaboration with Cambridge Analytica. Peter Thiel, a Facebook board member, is a Palantir co-founder.”
We aren’t sure what the big data powerhouse Palantir knew or didn’t know, but so far the company has been outside the blast zone.
Take for example, the recent news that Cambridge Analytica’s data seems to be out of business or in business under a different name.
Keep that ceramic plate on. The dominoes may continue to fall.
Patrick Roland, May 13, 2018
Search Is a Problem: Still a Clumsy Song and Dance Routine
May 24, 2018
Enterprise search has been around for decades. Hundreds of consultants have asserted patterns, models, methods, and MBA infused strategies to “fix” enterprise search.
Why?
Wherever there is an organization with one or more enterprise search systems, I have found these characteristics:
- Unhappy users
- Unhappy senior manager
- Unhappy bean counters
- Unhappy vendors
- Usually happy consultants if they are paid.
I am biased, old, and hard nosed. After writing the first three editions of the Enterprise Search Report, the New Landscape of Search, adding a word or two to that astounding guru Martin White’s book about Successful Enterprise Search Management, talking with dozens of PhD candidates whose dissertatioins about search and retrieval would change the world, and meeting with vendors large and small for decades—I am amused by the arm waving enterprise search engenders.
Don’t get me wrong. There are very good information access systems. But these vendors license solutions which usually focus on solving a specific problem. Case in point: Blackdot, Terbium Labs, and Verint, and many others.
From the point of view of flailing content management experts, “enterprise search” means finding information in a usually flawed, Rube Goldberg construct called a CMS or content management system.
Against this wallpaper with my scrawled biases, I read “Diagnosing Enterprise Search Failures.” The pivot point for the story is another report that almost two thirds of enterprise search users are not satisfied with the retrieval system.
Like a reprise of a vaudeville act from the 1920s at a rap concert, the music and the footwork are stale, out of touch, and worn.
Enterprise search had its decade in the sun. The period between 1995 and 2005 was the golden age of search. Then the sun imploded. Over-promising and under-delivering made it clear to those licensing enterprise search systems that finding information was not a solution to digital information woes.
In fact, an enterprise search system exacerbated the problems employees encountered when trying to locate specific information. Fast Search & Transfer, Convera, Delphes, Entopia (remember that outfit), and other aggressively marketed companies found out that companies would license technology and then balk at the on going costs.
One by one the big names in enterprise search went out of business or found themselves owned by larger firms with a belief that their managers could make search a winner.
How did that work out? Chase down someone at Lexmark and ask about their experience with ISYS Search Software. Repeat the process at Dassault Systèmes? Do the same thing for products ranging from Artificial Linguistics to Vivisimo.
The result is that the universe of companies offering search solutions has changed since 2008. The legal dust up between HP and Autonomy continues. Search did not make HP happy.
Surveys are fine, but the data reveal nothing new. Enterprise search is not a solution to information problems in an enterprise. Companies are embracing free or low cost solutions based on open source technology. Specialist systems which address specific information access problems are thriving. One may not think of Diffeo and Palantir Technologies as enterprise search systems, but they are information access solutions and not designed to solve a panoply of retrieval and information management issues.
The reason enterprise search fails to please users boils down to the disconnect between what the user wants and what an enterprise wide system can deliver. The vendors promise more than technology can provide.
Checklists, MBA rah rah, and misplaced confidence in technology will not solve these specific challenges:
- The cost of maintaining, upgrading, and tuning an enterprise search system to the needs of specific users is significant
- Users have a keen desire to rely on the software to do the thinking for them. When a system requires the user to think or formulate a query or perform downstream analysis, the search system becomes a problem
- Procurement teams often lack the discipline and clout to lay out tight requirements and select a vendor to do that job. The pattern is to create a wish list, sign a deal, and leave the baggage of failure behind.
- The systems provided do not match what the marketers demo, suggest, or assert the software will actually do in an affordable, reliable, understandable manner.
As a former rental at a reasonably competent management consulting firm, a method for figuring out how to solve a problem has one objective: Sell billable work. I understand that.
Do not confuse a consultant’s report with solving the problem of enterprise search. If enterprise search worked, there would be little appetite for methodologies to figure out failure.
Why such hostility to enterprise search? I think clueless large and medium sized companies want to buy a silver bullet. Even better, the bullet must kill the content vampire with a single, low cost, easy to use, accurate shot.
That’s not going to happen… ever.
The problem is that individuals looking for information need tools to solve quite specific business tasks. In enterprise search, there are numerous points of failure; for example:
- Management support is weak
- Organizational infighting triggers departments to get their own search solution
- The technology does not work
- Results do not meet user needs
- Funding is insufficient
- Technical staff find that fixes are not easy or possible
- Content known to be in the system cannot be found
- Vendors change direction from search to customer support and leave search customers dangling
- The people involved are focused on their careers, lunch, or finding a new job, not the nitty gritty of designing a solution for a specific group of workers with an information need.
And there are other issues related to over-promising and under-delivering. I wrote about this years ago and talked about falling off the cliff of high expectations. Enterprise search users inevitably crash into the reality of the system. Thus, the significant percentage of dissatisfaction with enterprise search.
I know of no enterprise search system which delivers on these points. Furthermore, as venture funding flows into Coveo and LucidWorks, as IBM falls farther and farther behind its revenue goals for Watson search (OmniFind, Vivisimo, et al), and as Microsoft buys more and more search start ups in the hopes of finding a silver bullet to its search mess—It is clear that stakeholders, customers, and users are going to become increasingly annoyed at the problem of enterprise search.
Why did Google bow out of enterprise search? Why has Elastic emerged as the go-to solution for many enterprise search applications? Why are companies like Funnelback, Sophia, Exorbyte, and dozens of others scrambling?
Enterprise search looked like a solution to some important problems. Today not so much. Open source search software is fine. However, how many of the open source vendors are going to be able to generate a return for their investors with what amounts to free software.
Enterprise search is the wrong label for today’s solutions. Even proprietary systems in hock for $100 million have longer odds than a nag entered in the Kentucky Derby.
Therefore, thrashing.
Stephen E Arnold, May 24, 2018
The Guardian Reveals That Big Tech Yields Useless Products
May 21, 2018
I don’t want to be a Luddite, but it seems to me that autonomous drones carrying interesting payloads are often darned helpful. The value of the big tech which gets these puppies aloft and in theater can be questioned when the person asking the questions is sitting at a laptop in an urban center thinking big thoughts. I would suggest that if that urban beastie were under fire in a slightly less refined environment and needed a bit of air cover to make an escape from a free fire zone, the big tech in the form of a MQ-1L Predator attack drone with AGM 114 K2 Hellfire Missiles would be somewhat useful. Mostly.
My hunch is that the British newspaper disseminating real news in “Ignore the Hype Over Big Tech. Its Products Are Mostly Useless” has a keen desire to poke sticks in the eye of Amazon, Apple, Facebook, Google, Netflix, and probably other outfits. Why not toss in Palantir Technologies too? Oh, don’t forget Darktrace.
The main point of the write up struck me as:
In fact, might Duplex be a grim portal into a future in which high-flyers get digital “assistants” to do their chores, while poorly paid people have to meekly talk to computers, in constant fear that they are about to be automated into joblessness?
I like this notion of reworking a two tier society so those in the lower tier could perform jobs like booking a haircut.
The target here is Google and its demo of its artificial intelligence wizardry. Some of this magic is performed in the UK by DeepMind.
Google is now a couple of decades into its search based businesses.
Several questions occur to me:
- Where was the Guardian and other “real” news outfits in the pre IPO phase of Google when Yahoo took Google to task for finding inspiration in GoTo.com, Overture.com, and Yahoo’s “pay to play” models?
- Where have newspapers been for the last two decades as companies finding revenue streams different from print and classified display advertising on paper?
- With the insight of a skilled editorial staff and the ability to generate information, why haven’t those professionals been able to communicate the risk of taking “free lunches” every day for two decades?
Net net: The train has left the station. The “woke” folks are already at the airport with their digital assistants in their pocket and the beach on their mind.
Stephen E Arnold, May 21, 2018
Facebook: Collateral Damage?
May 17, 2018
The Cambridge Analytica/Facebook data scandal has rightly been scrutinized by everyone from individual users to entire government bodies. As could be expected when the players are this large, what people are finding links together unlikely suspects and victims in this data breach. One such surprise popped up this week when we read a Gizmodo report, “Facebook ‘Looking Into’ Palantir’s Access to User Data.”
According to the story:
“The inquiry was led by Damian Collins, chair of Parliament’s Digital, Culture, Media, and Sport Committee. According to CNBC, Collins asked if Palantir was part of Facebook’s “review work.”
“While it’s unclear if it gained access to the Facebook user data that Cambridge Analytica harvested, Palantir’s connection to the social network extends beyond any potential collaboration with Cambridge Analytica. Peter Thiel, a Facebook board member, is a Palantir co-founder.”
We aren’t sure what the big data powerhouse Palantir knew or didn’t know, but if they are found to have violated laws it could get ugly. And the ugliness doesn’t seem to know any depths in this case. Take for example, the recent news that Cambridge Analytica’s data could be up for sale since the company declared bankruptcy after the data breach news tanked the company. Buckle up, because we don’t think the dominoes are done falling yet.
Patrick Roland, May 17, 2018
LucidWorks Has a Search App for That. What?
April 27, 2018
Is there life in enterprise search after many years of hype, razzle dazzle, and over the top marketing?
Maybe?
Lucidworks announced that they have a brand new search tool for enterprise business systems, says Global Newswire in the article, “Lucidworks Launches AI-Powered Site Search App For Enterprise.” The new application is dubbed Lucidworks Site Search and it is an easy configurable, embeddable site-based application.
Lucidworks Site Search uses workflows that optimize natural language processing and machine learning for users to personalize their search results. The application uses rich faceting and filtering to drill down for the most accurate results. Users will be able to access content and insights quicker than older applications.
The Lucidworks CEO said,
“‘Developing a website’s search with both a powerful backend and an elegant UI can be an arduous process. We’ve created Site Search to empower more teams to get site search apps done and out the door,’ explains Lucidworks CEO Will Hayes. ‘By increasing the usability through an applications-based approach, we’re able to bring Lucidworks’ operationalized AI to more customers.’”
We enjoy terms like “operationalize.” Do we understand these MBA inspired noun to verb arabesques? Not really.
Key word search is a useful utility. The new Lucidworks Site Search scans through every document, allows quick configuration, and has an attractive user interface. Elasticsearch does this as well.
We believe the future belongs to vendors with a more comprehensive next generation information access system. In short, more like Palantir Gotham or BAE NetReveal and less like the mainframe centric IBM Stairs approach.
Whitney Grace, April 27, 2018
CyberOSINT: Next Generation Information Access Explains the Tech Behind the Facebook, GSR, Cambridge Analytica Matter
April 5, 2018
In 2015, I published CyberOSINT: Next Generation Information Access. This is a quick reminder that the profiles of the vendors who have created software systems and tools for law enforcement and intelligence professionals remains timely.
The 200 page book provides examples, screenshots, and explanations of the tools which are available to analyze social media information. The book is the most comprehensive run down of the open source, commercial, and cloud based systems which can make sense of social media data, lawful intercept data, and general text and imagery content.
Companies described in this collection of “tools” include:
- Cyveillance (now LookingGlass)
- Decisive Analytics
- IBM i2 (Analysts Notebook)
- Geofeedia
- Leidos
- Palantir Gotham
- and more than a dozen developers of commercial and open source, high impact cyberOSINT tool vendors.
The book is available for $49. Additional information is available on my Xenky.com Web site. You can buy the PDF book online at this link gum.co/cyberosint.
Get the CyberOSINT monograph. It’s the standard reference for practical and effective analysis, text analytics, and next generation solutions.
Stephen E Arnold, April 5, 2018
Cambridge Analytica and Fellow Travelers
March 26, 2018
I read Medium’s “Russian Analyst: Cambridge Analytica, Palantir and Quid Helped Trump Win 2016 Election.” Three points straight away:
- The write up may be a nifty piece of disinformation
- The ultimate source of the “factoids” in the write up may be a foreign country with interests orthogonal to those of the US
- The story I saw is dated July 2017, but dates – like other metadata – can be fluid unless in a specialized system which prevents after the fact tampering.
Against this background of what may be hefty problems, let me highlight several of the points in the write up I found interesting.
More than one analytics provider. The linkage of Cambridge Analytica, Palantir Technologies, and Quid is not a surprise. Multiple tools, each selected for its particular utility, are a best practice in some intelligence analytics operations.
A Russian source. The data in the write up appear to arrive via a blog by a Russian familiar with the vendors, the 2016 election, and how analytic tools can yield actionable information.
Attributing “insights.” Palantir allegedly output data which suggested that Mr. Trump could win “swing” states. Quid’s output suggested, “Focus on the Midwest.” Cambridge Analytica suggested, “Use Twitter and Facebook.”
If you are okay with the source and have an interest in what might be applications of each of the identified companies’ systems, definitely read the article.
On April 3, 2018, my April 3, 2018, DarkCyber video program focuses on my research team’s reconstruction of a possible workflow. And, yes, the video accommodates inputs from multiple sources. We will announce the location of the Cambridge Analytica, GSR, and Facebook “reconstruction” in Beyond Search.
Stephen E Arnold, March 26, 2018
Crime Prediction: Not a New Intelligence Analysis Function
March 16, 2018
We noted “New Orleans Ends Its Palantir Predictive Policing Program.” The interest in this Palantir Technologies’ project surprised us from our log cabin with a view of the mine drainage run off pond. The predictive angle is neither new nor particularly stealthy. Many years ago when I worked for one of the outfits developing intelligence analysis systems, the “predictive” function was a routine function.
Here’s how it works:
- Identify an entity of interest (person, event, organization, etc.)
- Search for other items including the entity
- Generate near matches. (We called this “fuzzification” because we wanted hits which were “near” the entity in which we had an interest. Plus, the process worked reasonably well in reverse too.)
- Punch the analyze function.
Once one repeats the process several times, the system dutifully generates reports which make it easy to spot:
- Exact matches; for example, a “name” has a telephone number and a dossier
- Close matches; for example, a partial name or organization is associated with the telephone number of the identity
- Predicted matches; for example, based on available “knowns”, the system can generate a list of highly likely matches.
The particular systems with which I am familiar allow the analyst, investigator, or intelligence professional to explore the relationships among these pieces of information. Timeline functions make it trivial to plot when events took place and retrieve from the analytics module highly likely locations for future actions. If an “organization” held a meeting with several “entities” at a particular location, the geographic component can plot the actual meetings and highlight suggestions for future meetings. In short, prediction functions work in a manner similar to Excel’s filling in items in a number series.
What would you predict as a “hot spot” based on this map? The red areas, the yellow areas, the orange areas, or the areas without an overlay? Prediction is facilitated with some outputs from intelligence analysis software. (Source: Palantir via Google Image search)
Is Change Coming to High Tech Lobbying in Washington, DC?
March 14, 2018
The received wisdom in Washington, DC is that when it comes to politics, money talks.
The idea is simple: Donate money to a politician’s campaign or a politician’s favorite “cause” and get your email and phone calls answered.
The Independent explains that, “Google Outspends All Rival Washington Lobbyists For First Time In 2017.”
In 2017, Google spent $18 million to lobby Congress on a slew of issues ranging from immigration, tax reform, antitrust, and online advertising. Tech companies have big bucks and the power to take on Congress on governmental policies. Lawmakers, on the other hand, fire back with pot shots like allowing Russian operatives to share content and how their software and other technology allows tech companies to abuse their power.
Google’s Washington operation proposed legislation that would require Web companies to collaborate on a public database of political as that run on their platforms. The idea is that the database would prevent foreign nations from exploiting online platforms. Other companies like Amazon and Facebook have ramped up their lobbying spending too.
Despite the power tech companies wield, their roles in society are changing and there is some fear associated with it:
“‘These are companies that are touching so many parts of the economy, they are touching so many parts of our geography. So it’s inevitable that they are going to engage in a host of political and policy issues,’ said Julie Samuels, the executive director of Tech: NYC, a group that represents New York-based tech firms. Samuels added that Silicon Valley has also had to adjust to a new political order, under a Republican administration. ‘Many tech companies had only been real players during the Obama administration. They had a lot to learn.’”
Now the received wisdom may have to modified. Beyond Search noted that Palantir has landed a chunk of a US government contract to create a DCGS which meets the needs of the US Army.
We think that Google will continue to support lobbying, but it will seek more deals like its tie up with the US government’s push for artificial intelligence. What may emerge is a new approach to influencing procurement decisions and legislation in Washington.
Whitney Grace, March 14, 2018
OpenText Wants to Be the Big Dog in Cyber Security
February 4, 2018
My wife and I rescued a French bull dog. We also have a boxer, which is three times the size of the rescued canine. The rescued canine thinks he is a bull mastiff. We believe that the French bull dog has a perception problem.
Here’s a quote from “OpenText Enfuse 2018 To Showcase The Future of Cybersecurity and Digital Investigations”:
OpenText’s industry leading digital investigation, forensic security and data risk management solutions are defining the future of cybersecurity, digital investigations and e-Discovery, and serve to extend the security capabilities of OpenText’s leading information management platform.”
I noticed this statement at the bottom of the “real” news story:
Certain statements in this press release may contain words considered forward-looking statements or information under applicable securities laws.
I think our French bull dog might say something like this when he tries to impose his will on Max, our large, strong, aggressive boxer.
In the cyber marketplace, will IBM i2 roll over and play dead? Will Palantir Technologies whimper and scamper back to Philz Coffee? Will the UAE vendor DarkMatter get into the pizza business? Will the Google and In-Q-Tel funded Recorded Future decide that real estate development is where the action is?
Forward looking? Yeah, no kidding.
Stephen E Arnold, February 4, 2018
-
- Subscribe to Beyond Search
Feature archive
News archive
-
