Department of Defense: Learning from Social Media Posts

May 25, 2019

A solicitation request dated May 13, 2019, “A–Global Social Media Archive, 350 billion digital data records” is an interesting public message. Analysis of social media allegedly has been a task within other projects handled by firms specializing in content analytic. These data mining efforts are, based on DarkCyber’s understanding of open source information from specialist vendors, are nothing new. The solicitation offers some interesting insights which may warrant some consideration.

First, the scope of the task is 350 billion digital records. It is not clear what a “digital record” constitutes, but the 350 billion number represents about two or three months of Facebook posts. It is not clear if the content comes from one service like Twitter or is drawn from a range of messaging and content sources.

Second, the content pool must include 60 languages. The most used languages on the public Internet are English, Chinese, and Spanish. The other 57 languages contribute a small volume of content, and this fact may create a challenge for the vendors responding to the solicitation. The document states:

Data includes messages from at least 200 million unique users in at least 100 countries, with no single country accounting for more than 30% of users.

Third, the text content and the metadata must be included in the content bundle.

The exclusion of photographs and videos is interesting. These are important content mechanisms. Are commercial enterprises operating without connections to nation states operating large-scale content aggregation systems likely to be able to comply? Worth watching to find out who lands this project.

Stephen E Arnold, May 25, 2019

Google: Whom Does One Believe?

May 24, 2019

Apparently, what is good for China is situational for Google. The Intercept declares, “Google’s Censored Search Would Help China ‘Be More Open,’ Said Ex-CEO Eric Schmidt.” Writer Ryan Gallagher points to a recent BBC News interview with Eric Schmidt in which the former Google CEO, and current board member, seemed to defend his company’s choice to build a censored search platform just for China. To be fair, Schmidt’s opinion is a bit more nuanced than the above headline suggests: the theory seems to be that, by cozying up to China, Google might influence that country to embrace internet freedom. Seems reasonable? Gallagher writes:

“During Schmidt’s tenure as CEO, in 2006, Google launched a search engine in China but pulled out of the country in 2010, due to concerns about Chinese government interference. At that time, [co-founder Sergey] Brin said the decision to stop operating search in the country was mainly about ‘opposing censorship and speaking out for the freedom of political dissent.’ Schmidt revealed in his BBC interview that he had argued against Brin — believing that the company should remain in China, despite the censorship requirements. He said he felt that it was better ‘to stay in China and help change China to be more open.’

We also noted this passage:

“Brin has previously said that he felt the same way for a period of time — that Google could help China embrace greater internet freedom. But he watched as the company, over a number of years, faced increasing censorship requests from the Chinese government. ‘Things started going downhill, especially after the Olympics [in Beijing in 2008],’ Brin said in a 2010 interview. ‘And there’s been a lot more blocking going on since then. … [S]o the situation really took a turn for the worse.’”

At least Google’s workers understand that cooperating with China’s censorship efforts will do nothing to dampen China’s enthusiasm for censorship. Once word of the search platform tailor-made for China, code named Dragonfly, got out, workers protested. Many of them were unhappy to learn they had unwittingly supported censorship through their work, and called for more transparency and employee input. In that BBC interview, Schmidt says Google is “no longer pursuing Dragonfly,” but could not rule out a return to the project in the future.

Meanwhile, censorship has only gotten worse in China since Brin’s 2010 concerns; according to the Human Rights Watch, a 2016 cybersecurity law has brought internet control to “new heights.”

Now Google has complied with the US government’s directive about Huawei. Compliance and an apparent leadership position. Google’s diplomacy may be tested, and the firm’s leadership will have opportunities to craft other statements.

But whom does one believe when it comes to reading tea leaves about Google and its intentions?

Cynthia Murrell, May 24, 2019

Forcing China to Fill the Google Gap

May 20, 2019

I read the Reuters’ exclusive “Google Suspends Some Business with Huawei after Trump Blacklist – source.” The news story presents some information which on the surface is interesting. Google allegedly has “suspended business with Huawei.” There is a caveat; namely, “except those publicly available via open source licensing.” Huawei mobile phone users can chug along for now. Reuters quotes an unnamed source in the rich tradition of “real news.” The source allegedly said:

“Huawei will only be able to use the public version of Android and will not be able to get access to proprietary apps and services from Google.

The number of blog posts and “real” news stories about this Google move is intriguing. Most of these follow the standard impact on business, what about the users, and whither Android lines.

My thought is that innovation often is a result of adversity. If I narrow my focus to topics related to intelligence analysis and a bit of the lawlful intercept activity, this development could have some unintended consequences. Put aside fears of more industrial espionage, hassling of Google and other US firms as a retaliatory measure, and the grousing of US companies faced with losing Huawei and its suppliers as customers.

Chinese engineers may turn their attention from reasonably effective facial recognition and surveillance systems to the job of moving beyond Google’s technology, creating parallels for some US technologies, and innovating in ways to lock out prying eyes from certain types of data transmissions.

The thrill of making life difficult for Huawei and demonstrating that Google is on board with US trade policies may be short lived. Maybe China moves some of its more interesting technology into the “Google gap”? Perhaps China steps up the for functionalities no longer easily available? What if China finds a way to shut certain mechanisms for monitoring information shoved around by Huawei and other Chinese vendors’ equipment?

The unintended consequence is that the US and possibly some of its allies will be forced to become more technologically innovative.

The big question for me is: “What if this is the turning point for Chinese technology?” China could force the US to become makers of buggy whips and seat covers for the new communication vehicles which may come down the information superhighway.

That’s a big and unintended consequence to consider in my opinion.

Stephen E Arnold, May 20, 2019

BBC Explains the End of the Open Internet After It Ended

May 18, 2019

A 3,500 word story from the BBC explains the end of the open Internet. The main idea is that the US approach of sending anything to anyone is not what China, Russia, and other countries will accept. “The Global Internet Is Disintegrating. What Comes Next?” is not news. The essay is a pinch of intelligence agency analysis (a small pinch I might add), some business school semantics, and the routine quotes from experts.

The “what comes next” is mostly ignored. The reason is that the actions taken by a number of countries over the last decade represent the construction of a series of walled gardens. Blocking access is old hat in Iran. China and Russia have stepped up their efforts with political hand waving. Russia has laws which make the US companies either roll over or shut down. How about LinkedIn in Russia?

China is doing the system administrator squeeze. The twist is that Chinese high technology companies are lending a helping hand. Last time I was in China it took only a few minutes for my mobile phone to become a less than helpful gizmo. Five years earlier it took a couple of days to achieve the near useless state.

The BBC explains:

A separate internet for some, Facebook-mediated sovereignty for others: whether the information borders are drawn up by individual countries, coalitions, or global internet platforms, one thing is clear – the open internet that its early creators dreamed of is already gone.

With the business school jargon “digital deciders” wafting through the article, the question “what comes next” is not answered. The reason is that the reality is unpalatable to many in what China and Russia think of as the West.

The actions of countries attempting to prevent unfettered flows of information are designed to protect the government and commercial sector from the difficulties that arise when using US technology without an old fashioned speed limiter. Smaller countries are not keen on having Facebook and Twitter users coordinate protests and disrupt what these countries’ governments see as “normal” processes.

The so called digital deciders have already decided. The future is in place, and what needs to be described and understood include:

  • The actions of China and Russia are designed to control US influence. The future is a shift from control to more aggressive actions.
  • The alignment of nation states will be a decision by those countries to sign on for either the China approach or the Russia approach. In short, new blocs are now taking shape.
  • The behaviors of US high technology companies are designed to increase the power of these firms. Therefore, the companies will find themselves sued and hassled because some thinkers in China and Russia believe it is their duty to step in and reign in the actions of unregulated US firms.

The future of the Internet is, in my opinion, a battle ground. Bad things can happen in such a place even if it is digital.

Stephen E Arnold, May 18, 2019

digital deciders

A Disconnect: Department of Defense and the Silicon Valley Scooter Riders

May 2, 2019

I have a short conversation with a real reporter about why Silicon Valley top dogs cannot keep their puppies in line. There are walk outs, sit ins, and dust ups about what senior managers expect and what their code monkeys do.

Defense One published an interesting summary of this jar of pickled pigs feet, the ghastly green and pale pink delights once popular in rural Kentucky.

The Pentagon Is Flubbing Its Pitch to Silicon Valley” runs down the gap between the Silicon Valley jefes and the Department of Defense. The summary is quite useful, and it includes hot links to several high-value books.

But the highlight of the article was this observation:

Surveys have shown us for decades that tech executives are quite politically liberal, albeit rather libertarian on regulatory issues. But it’s becoming clearer how much more progressive their workers are than the bosses.

One key point is tucked behind this statement. The leaders of Silicon Valley companies are used to getting their own way. People used to getting their own way expect those whom they pay to do what they are told. Entitlement? Arrogance? Upbringing?

Probably a pinch of each stirred into an ego centric soup.

The chasm is widening and there is no easy way to cross it. The split within the large firms has been identified by Defense One. Now what will the newly privacy infused Facebook, the online ad giant Google, and the Amazon bulldozer do?

No answers from Harrod’s Creek, of course, because scooters versus smart weapons is a tough call for some to make.

Stephen E Arnold, May 2, 2019

New Department of Defense Web Site

April 26, 2019

DarkCyber noted a number of articles about the new Department of Defense Web site. The Executive Gov Web site ran a short item. If you view the article, you will notice one omission: The url for the site. Here it is: .

If you poke around on the Web site, you will find that it provides some information which was either difficult to locate, not indexed on, or simply unavailable. Be alert, however. Some of the headings can disappoint. A click on “Marketplace” explains an event which will be on either April 25th or April 26th. The Marketplace features some tweets which may be of interest to some site visitors. The news section contains news about the new Web site and six events which took place earlier this year. A click on the AFA event reveals that the next one will be held in 2020, but there is no information about how to learn about the upcoming event.

US government Web sites often start strong and then fade. With a continuous flow of content, this could become a useful source of information.

Stephen E Arnold, April 26, 2019

JEDI: A Down to Earth Battle Between Digital Super Powers

April 20, 2019

This may be good news for China and Russia. Nextgov predicts, “Without JEDI, Pentagon’s Artificial Intelligence Efforts May Be Hindered.” The Pentagon requires an enterprise cloud computing solution for its ambitious AI plans—once it gets past one little snag, that is. They had a plan, called the Joint Enterprise Defense Infrastructure contract, but it is now on hold pending litigation. Reporter Frank Konkel writes:

“Through the JEDI contract, the Pentagon aims to put a commercial company in charge of hosting and distributing mission-critical workloads and classified data to warfighters around the globe in a single cloud computing environment. That environment would also process large swaths of military and defense data and serve as the computing and analytics workhorse for artificial intelligence applications.

Motley Fool reports in “An Unexpected Scandal Threatens To Cripple Amazon”:

the Department of Defense (DoD) cleared itself of wrongdoing following an internal investigation into the forthcoming award of the $10 billion cloud computing Joint Enterprise Defense Initiative (JEDI) program. Yet the Pentagon’s self-exoneration was not comprehensive, as Bloomberg noted that: “The investigation uncovered evidence of unethical conduct that will be referred to the DoD inspector general for a separate review.”

Nations like China will not oblige us by putting their AI plans on hold while we catch up. The DoD could try using a hardware stack instead, but that would severely constrict their plans, according to Lt. Gen. Jack Shanahan, head of the Joint Artificial Intelligence Center.

Is DarkCyber surprised? A better question, “What was the business ethos of DH Shaw when Mr. Bezos honed his financial and business skills at that Wall Street firm?”

DarkCyber does not know.

Cynthia Murrell, April 20, 2019

Follow Intelligence? Watch the National Geospatial-Intelligence Agency

April 12, 2019

I read “Is Geospatial Intel the New Framework for Civilization? The NGA’s New Director Speaks His Mind.” The article contains several points which DarkCyber has identified as important. Are you into geo-fencing? If not, you may want to learn a bit more about this function.

Stephen E Arnold, April 12, 2019

Jason and Darpa-Nauts

April 12, 2019

If you are not up to speed on the Jason Group, Wikipedia, for now, has a write up about the organization of academics who provide input and other support to the US government. Yes, you can become a member. The trick is to identify people who are Jasons, lobby a couple, and wait until you are voted in. A Nobel prize is a useful award. High level contacts at Mitre can be a plus too.

So who cares about Jasons aside from some Washington insiders? Fewer DoD types than in the 1960s and 1970s.

According to Government Executive:

Pentagon officials are killing JASON in all but name. Last month, they sent the bad news to MITRE, the nonprofit corporation that runs the program. “The Pentagon Is Killing a Key Independent-Research Program” reported:

They [government COTRs] don’t have to “terminate” the contract to kill the program, since it was set to expire at the end of March. By changing the contract from IDIQ to a single contract, other agencies will no longer be able to commission studies, essentially killing the program without  technically terminating it.

This is a nice way of saying, “So long for now.”

What’s the conclusion Government Executive drew from this announcement:

Bottom line: the Pentagon is spending more on new science and tech initiatives but will be spending less on independent academic research into how those initiatives will fare.

DarkCyber believes that other research avenues are more likely to deliver the type of outputs that the DoD and its units require. Good news or bad news? Consulting firms are likely to benefit. Some academics will have to chase RFQs with more diligence.

Stephen E Arnold, April 12, 2019

Natural Language Processing: Will It Make Government Lingo Understandable

April 11, 2019

I noted a FBO posting for a request for information for natural language processing services. You can find the RFI at this link on the FBO Web site. Here’s a passage I found interesting:

OPM seeks information on how to use artificial intelligence, particularly natural language processing (NLP), to gain insights into statutory and regulatory text to support policy analysis. The NLP capabilities should include topic modeling; text categorization; text clustering; information extraction; named entity resolution; relationship extraction; sentiment analysis; and summarization. The NLP project may include statistical techniques that can provide a general understanding of the statutory and regulatory text as a whole. In addition, OPM seeks to learn more about chatbots and transactional bots that are easy to implement and customize with the goal of extending bot-building capabilities to non-IT employees. (Maximum 4 pages requested.)

The goal is to obtain information about a system that performs the functions associated with an investigative software system; for example, Palantir Technologies, IBM i2, or one of the numerous companies operating from Herzliya, north of Tel Aviv.

I am curious about the service provider who assisted in the preparation of this RFI. The time window is responding is measured in days. With advanced text analysis systems abounding in US government agencies from the Department of Justice to the Department of Defense and beyond, I wonder why additional requests for information are required.

Ah, procurement. A process in love with buzzwords so an NLP system can make things more clear. Sounds like a plan.

Stephen E Arnold, April 11, 2019

« Previous PageNext Page »

  • Archives

  • Recent Posts

  • Meta