Intelligence Industry Becoming Privatized and Concentrated
March 10, 2017
Monopolies aren’t just for telecoms and zipper manufacturers. The Nation reveals a much scarier example in its article, “5 Corporations Now Dominate Our Privatized Intelligence Industry.” Reporter Tim Shorrock outlines the latest merger that brings us to this point, one between Pentagon & NSA contractor Leidos Holdings and a division of Lockheed Martin called Information Systems and Global Solutions. Shorrock writes:
The sheer size of the new entity makes Leidos one of the most powerful companies in the intelligence-contracting industry, which is worth about $50 billion today. According to a comprehensive study I’ve just completed on public and private employment in intelligence, Leidos is now the largest of five corporations that together employ nearly 80 percent of the private-sector employees contracted to work for US spy and surveillance agencies.
Yes, that’s 80 percent. For the first time since spy agencies began outsourcing their core analytic and operational work in the late 1990s, the bulk of the contracted work goes to a handful of companies: Leidos, Booz Allen Hamilton, CSRA, SAIC, and CACI International. This concentration of ‘pure plays’—a Wall Street term for companies that makes one product for a single market—marks a fundamental shift in an industry that was once a highly diverse mix of large military contractors, small and medium technology companies, and tiny ‘Beltway Bandits’ surrounding Washington, D.C.
I should mention that our beloved leader, Stephen E Arnold, used to work as a gopher for one of these five companies, Booz Allen Hamilton. Shorrock details the reasons such concentrated power is a problem in the intelligence industry, and shares the profile he has made on each company. He also elaborates on the methods he used to analyze the shadowy workforce they employ. (You’ll be unsurprised to learn it can be difficult to gather data on intelligence workers.) See the article for those details, and for Shorrock’s discussion of negligence by the media and by Congress on this matter. We can agree that most folks don’t seem to be aware of this trend, or of its potential repercussions.
Cynthia Murrell, March 10, 2016
Cambridge Analytica: Buzz, Buzz, Buzz
March 9, 2017
The idea that software can make sense of information is a powerful one. Many companies tout the capabilities of their business processes, analytical tools, and staff to look at data and get a sense of the future. The vast majority of these firms have tools and methods which provide useful information.
What happens when a person who did not take a course in analytics learns about the strengths and limitations of these systems?
Answer: You get some excitement.
I read “Big Data’s Power Is Terrifying. That Could Be Good News for Democracy.” The main idea is that companies with nifty analytic systems and methods can control life is magnetic. Lots of folks want to believe that a company’s analyses can have a significant impact on elections, public opinion, and maybe the stock market.
The write up asserts:
Online information already lends itself to manipulation and political abuse, and the age of big data has scarcely begun. In combination with advances in cognitive linguistics and neuroscience, this data could become a powerful tool for changing the electoral decisions we make. Our capacity to resist manipulation is limited.
My view is that one must not confuse the explanations from marketing mavens, alarmists, and those who want to believe that Star Trek is “real” with what today’s systems can do. Firms like Cambridge Analytica and others generate reports. In fact, companies have been using software to figure out what’s what for many years.
What’s interesting is that folks learn about these systems and pick up the worn ball and carry it down field while screaming, “Touchdown.”
Sorry. The systems don’t warrant that type of excitement. Reality is less exciting. Probabilities are useful, not reality. But why not carry the ball. It is easier than learning what analytics firms do.
Stephen E Arnold, March 9, 2017
Enterprise Search in the Cloud: Which Service Provider?
March 9, 2017
In the wake of Amazon’s glitch, a number of publications rushed to report on the who, what, where, and why. ZDNet took a different approach in “Which Cloud Will Give You the Biggest Bang for the Buck?” The write up recycled in the best tradition of “real” journalism a report from a vendor named Cloud Spectator. I won’t ask too many questions about sample size, methodology, the meaning assigned to “value,” and statistical validity. I will assume that the information is not Facebook news.
The guts of the write up is this chart, which is impossible to read in this blog post, but the original is reasonably legible:
What this chart reveals about hosting is that the 1&1 system is the big dog. I would point out that the naming of the service is “1+1” in the chart; the “real” name of the company is “1&1”, a real joy to search using free Web search systems.
Okay, 1+1 was on my radar as a very low cost provider of Web page hosting and other services. Now the company remains a low cost provider and has added a range of new services. Cloud Spectator finds the company A Number One. I was tempted to type ANo1, another keen string to plug into a Web search system.
What interested me was the cluster of outfits which the Cloud Spectator survey pegged as small dogs; for example, Amazon Web Services, the very same outfit that nuked some major Web sites. (Send in a two pizza team, Mr. Bezos.)
Close to Amazon’s lower third ranking was Microsoft Azure. Somehow that seems par for the new Microsoft. Google and the financially challenged Rackspace were in the middle of the pack. (What happened to Rackspace’s love affair with Robert Scobel, recently removed from the Gilmore Gang.)
But the major news for me was that IBM, yep, the owner of the famed and much admired Watson thing, was darn near last. IBM nosed out DimensionDate for the “Also Participated” badge.
Net net: Maybe 1&1 should get more attention. Perhaps the company will change its name to minimize the likelihood of misspellings. Alternatively 1&1 can hire Recode to endlessly repeat that one spells embarrassed with two r’s and two esses.
When it comes to search in the cloud, the question becomes, “How does one deploy an enterprise class search and content processing on the 1&1 system?” Good question.
Stephen E Arnold, March 9, 2017
Quote to Note: Google and Its Red Herrings
March 9, 2017
I read “Google Makes Such a Big Deal Out of Everything But Its Search Business.” Wake up call. Google is going on 20 years of fun and excitement in online and content processing. It’s been running the same game plan for maybe 14, 15 years. It is good that CNBC is sort of thinking about the GOOG.
In the write up, there is a quote which I noted. The rest of the write up is pretty forgettable. Here’s the statement allegedly made by Peter Thiel, founder of Palantir Technologies and adviser to the new US administration. Here’s the comment:
If you have a monopoly, you will tell people you are in a super-competitive business. And if you are in a super-competitive business, you will tell people that you have a monopoly of sorts. So for example, if you have a search company in Silicon Valley that I will not name, if you were to go around to CEOs saying, ‘We have a bigger share of the market and higher profit margins than Microsoft ever had in the 1990s,’ you wouldn’t do that…You don’t even talk about search. You say, ‘We are a technology company with an enormous space called technology, and we’re competing with Apple on smartphones, and we’re competing on self-driving cars, and there’s competition in everything we’re doing except this one thing called search, and we never talk about that.'”
Stephen E Arnold, March 9, 2017
Dark Web Explosives Buyer Busted Through FBI Infiltration
March 9, 2017
Here is the story of another successful Dark Web bust. Motherboard reports, “Undercover FBI Agent Busts Alleged Explosives Buyer on the Dark Web.” The 50-year-old suspect was based in Houston, and reporter Joseph Cox examined the related documents from the Southern District of Texas court. We are not surprised to learn that the FBI found this suspect through its infiltration of AlphaBay.; Cox writes:
The arrest was largely due to the work of an undercover agent who posed as an explosives seller on the dark web marketplace AlphaBay, showing that, even in the age of easy-to-use anonymization technology, old-school policing tactics are still highly effective at catching suspects.
According to the complaint, on August 21, an FBI Online Covert Employee (OCE)—essentially an undercover agent—located outside Houston logged into an AlphaBay vendor account they were running and opened an unsolicited private message from a user called boatmanstv. ‘looking for wireless transmitter with detonator,’ the message read. ‘Everything I need to set of a 5 gallon can of gas from a good distance away [sic].’ The pair started a rapport, and boatmanstv went into some detail about what he wanted to do with the explosives.
One thing led to another, and the buyer and “seller” agreed to an exchange after communicating for a couple of weeks. (Dark Web sting operations require patience. Lots of patience.) It became clear that Boatmanstv had some very specific plans in mind for a very specific target, and that he’d made plenty of purchases from AlphaBay before. The FBI was able to connect the suspect’s email account to other accounts, and finally to his place of business. He was arrested shortly after receiving and opening the FBI’s package, so it would appear there is one fewer violent criminal on the streets of Houston.
It is clear that the FBI, and other intelligence organizations, are infiltrating the Dark Web more and more. Let the illicit buyer be wary.
Cynthia Murrell, March 9, 2016
Short Honks: 8 March 2017
March 8, 2017
We have a number of items which reveal great thought and actions in our world of digital information and other whizzy technological doings.
An Imponderable
First up, a quote to note from the New York Times, an outfit which is reinventing itself to be digital. I wonder if anyone recalls Jeff Pemberton and his Times Online notion from the 1980s. My hunch: Nah. Here’s the quote from the March 7, 2017 dead tree edition, ScienceTimes section, page D3 under the heading “Activists Rush to Find Dark Data under Threat”:
If they [the US government] are going to delete something, how will we even know it is deleted if we did not know it was there?
Yes, another expert in step with the antics of FirstGov.gov now USA.gov. Where are those data? The online version of the story at this link may charge you to view the content. Yes, the digital Gray Lady.
Hewlett Packard: Into Commodities
I don’t know too much about the ins and outs of a big time outfit like Hewlett Packard. I did note that Hewlett Packard is going to buy Nimble Storage for $1 billion. The write up states:
Some analysts, however, wonder if HPE overpaid. “This take-out price seems a little stretched for an asset that was not turning a profit,” Barclays analyst Mark Moskowitz wrote in a note to clients. “Plus, Nimble had been losing competitive momentum as the storage incumbents caught up on flash- and hybrid-based solutions.”
Yep, buying a money losing business which is pitching a commoditized storage method. What happens if HPE pairs its pricey Autonomy technology with Nimble Storage? Interesting hybrid to analyze with HPE’s predictive analytics tools.
Farewell Socl (Pronounced Social)
I don’t think there is a counseling service for disappointed Socl users. You know and use Socl, don’t you? Microsoft has decided to kill its social community. Microsoft is sure its “supportive community of like minded people” will forgive the Softies for this anti-social action. We noted this comment in the Verge:
[Microsoft] launched its own social network more than four years ago.
How time flies when you are fighting Facebook.
Google Buys a Community
The Google is going to be social. I noted that the containerizing outfit has purchased Kaggle. The write up in TechCrunch reported:
[Kaggle] is basically the de facto home for running data science — and machine learning — competitions.
Get those talented coders early and be social about it. You know. Friendly, courteous, team oriented. Take that, Facebook.
Stephen E Arnold, March 8, 2017
Index Is Important. Yes, Indexing.
March 8, 2017
I read “Ontologies: Practical Applications.” The main idea in the write up is that indexing is important. Now indexing is labeled in different ways today; for example, metadata, entity extraction, concepts, etc. I agree that indexing is important, but the challenge is that most people are happy with tags, keywords, or systems which return a result that has made a high percentage of users happy. Maybe semi-happy. Who really knows? Asking about search and content processing system satisfaction returns the same grim news year after year; that is, most users (roughly two thirds) are not thrilled with the tools available to locate information. Not much progress in 50 years it seems.
The write up informs me:
Ontologies are a critical component of the enterprise information architecture. Organizations must be capable of rapidly gathering and interpreting data that provides them with insights, which in turn will give their organization an operational advantage. This is accomplished by developing ontologies that conceptualize the domain clearly, and allows transfer of knowledge between systems.
This seems to mean a classification system which makes sense to those who work in an organization. The challenge which we have encountered over the last half century is that the content and data flowing into an organization changes often rapidly over time. At any one point in time, the information today is not available. The organization sucks in what’s needed and hopes the information access system indexes the new content right away and makes it findable and usable in other software.
That’s the hope anyway.
The reality is that a gap exists between what’s accessible to a person in an organization and what information is being acquired and used by others in the organization. Search fails for most system users because what’s needed now is not indexed or if indexed, the information is not findable.
An ontology is a fancy way of saying that a consultant and software can cook up a classification system and use those terms to index content. Nifty idea, but what about that gap?
This is the killer for most indexing outfits. They make a sale because people are dissatisfied with the current methods of information access. An ontology or some other jazzed up indexing component is sold as the next big thing.
When an ontology, taxonomy, or other solution does not solve the problem, the company grouses about search and cotenant processing again.
Is there a fix? Who knows. But after 50 years in the information access sector, I know that jargon is not an effective way to solve very real problems. Money, know how, and old school methods are needed to make certain technologies deliver useful applications.
Ontologies. Great. Silver bullet. Nah. Practical applications? Nifty concept. Reality is different.
Stephen E Arnold, March 8, 2017
ScyllaDB Version 3.1 Available
March 8, 2017
According to Scylla, their latest release is currently the fastest NoSQL database. We learn about the update from SiliconAngle’s article, “ScyllaDB Revamps NoSQL Database in 1.3 Release.” To support their claim, the company points to a performance benchmark test executed by the Yahoo Cloud Serving Benchmark project. That group compared ScyllaDB to the open source Cassandra database, and found Scylla to be 4.6 times faster than a standard Cassandra cluster.
Writer Mike Wheatley elaborates on the product:
ScyllaDB’s biggest differentiator is that it’s compatible with the Apache Cassandra database APIs. As such, the creators claims that ScyllaDB can be used as a drop-in replacement for Cassandra itself, offering users the benefit of improved performance and scale that comes from the integration with a light key/value store.
The company says the new release is geared towards development teams that have struggled with Big Data projects, and claims a number of performance advantages over more traditional development approach, including:
*10X throughput of baseline Cassandra – more than 1,000,000 CQL operations per second per node
*Sub 1msec 99% latency
*10X per-node storage capacity over Cassandra
*Self-tuning database: zero configuration needed to max out hardware
*Unparalleled high availability, native multi-datacenter awareness
*Drop-in replacement for Cassandra – no additional scripts or code required”
Wheatley cites Scylla’s CTO when he points to better integration with graph databases and improved support for Thrift, Date Tiered Compaction Strategy, Large Partitions, Docker, and CQL tracing. I notice the company is hiring as of this writing. Don’t let the Tel Aviv location of Scylla’s headquarters stop from applying you if you don’t happen to live nearby—they note that their developers can work from anywhere in the world.
Cynthia Murrell, March 8, 2016
Alphabet Google Smart Software Cannot Define Hate
March 7, 2017
Again. More artificial intelligence news. Or is it “fake news.” Many people want smart software to generate big revenues. Salesforce has Einstein. Vendors of keyword search centric information retrieval systems suddenly have smart software. At a meeting this week, a vendor of plastic piping said his new inventory system was intelligent. Yep, plastic pipes.
I read “Alphabet’s Hate Fighting AI Doesn’t Understand Hate Yet.” That struck me as odd. I learned in “Google’s AI Learned to Be Highly Aggressive When Stressed.” I assumed that an aggressive AI would take on an online dictionary, wrest the definition of hate from the Web site, and stuff the bits into the voracious multi petabyte storage system available to Deep Mind.
The issue of hate is relevant to hate speech. I think this is a gentle way of saying that unless text has some cue like a known entity which outputs nasty grams or a list of words likely to be used to convey hate, the smart software is performing like most smart software; that is, somewhere in the 40 to 65 percent accuracy range. Toss in human help, assorted dictionaries, a curated set of hateful content objects, and patient tuning and smiles all around.
The write up sidesteps my views and offered:
Google and its sister Alphabet company Jigsaw announced Perspective, a tool that uses machine learning to police the internet against hate speech.
And then noted:
Computer scientists and others on the internet have found the system unable to identify a wide swath of hateful comments, while categorizing innocuous word combinations like “hate is bad” and “garbage truck” as overwhelmingly toxic.
We know about Google’s track record in releasing early versions of software which maybe will sort of work a little bit someday.
The Googlers are busy doing what Autonomy Software did in the 1990s and other vendors of smart software have done in the subsequent quarter century: Teach the software by spoon feeding information into the system.
The write up points out:
Like all machine-learning algorithms, the more data the Perspective API has, the better it will work. The Alphabet subsidiary worked with partners like Wikipedia and The New York Times to gather hundreds of thousands of comments, and then crowd sourced 10 answers for each comment on whether they were toxic or not. The effort was intended to kick start the deep neural network that makes the backbone of the Perspective API.
I love the “all.” The truth of the matter is that self learning software just doesn’t work as well as the more narrowly defined artificial intelligence systems. Buy a book and Amazon looks in its database to see what other book buyers with a statistically generated profile seem to have bought. Bang. A smart recommendation. Skip the fact that books already purchased and stored in an Amazon database appear again and again on my list of recommended books. Smart but stupid, and that’s reasonably good implementation of smart software.
The write up works through examples of hate speech. Consult the source document for the lists. The write up works overtime to paint the lily gold and put some stage make up on what seems to be a somewhat dowdy implementation of Deep Mind / Google’s artificial intelligence.
Hey, I don’t want to drag another cat into the kitchen, but why not ask Watson what hate means. My hunch is that either the Google / Deep Mind engineers or the IBM Watson engineers will have a laugh over that idea. The smart software, on the other hand, might try to knock some sense into the competitor’s system.
Stephen E Arnold, March 7, 2017
HonkinNews for March 7, 2017 Now Available
March 7, 2017
The March 7, 2017, HonkinNews explains how to make bad data better. Curious about this type of digital magic? We report that one expert believes one just adds more data. The result is that a data lake becomes crystal clear. Believe it or not. Fake news and fake research may be more widespread than some believe. One well respected academic reports that often rejected research will get published. The author just needs to be persistent. Google may have a competitor which believes it can become the king of search. The proud owners of Qwant, a search system based on Pertimm technology, perceive the GOOG as vulnerable. With more than 95 percent of Web search traffic from France and Germany flowing to Google, we are not sure if Qwant realizes that Quaero tried this play and failed to get a first down. Did you know that one search wizard believes that Google’s PageRank system is like a soccer game? We sure didn’t. We red carded this explanation. Hewlett Packard Enterprise seems to be emulating IBM’s financial trajectory. Indicators are down. And Yahoo is making headlines once again. The person responsible for the security lapses and the failure of the company to manage the situation is revealed. Tune in to find out who won the purple plastic ring at the carnival. The video is at this link.
Ken Toth, March 7, 2017
-
- Subscribe to Beyond Search
Feature archive
News archive
-
