Secrecy News Talks Declassification

April 15, 2014

Declassified records are an interesting element to the public. But there is more to declassification than simply putting them out there for the public to find. Findability and search play a role also. Secrecy News focuses on the topic in their blog entry, “Putting Declassified Records to Good Use.”

The article says:

“The final, climactic step in the declassification of government records is not the formal removal of classification markings or even the transfer of the declassified documents to public archives. The culmination of the declassification process is when the records are finally examined by an interested reader and their contents are absorbed into the body of public knowledge.”

Secrecy News is an FAS project on government secrecy. They provide documentary resources on secrecy, intelligence, and national security. Interested readers can subscribe for regular updates. Secrecy is a hot topic due to the Snowden case, but this blog has been in business for years, and offers a steady flow of information, even if not completely original in scope.

Emily Rae Aldridge, April 15, 2014

Sponsored by, developer of Augmentext

Government Tackles Acquisition Inefficiencies

April 6, 2014

Given evidence like the vile backlog on veterans’ benefits and the still-operating paperwork bunker in Pennsylvania, one could be forgiven for suspecting that no one in government is even trying to bring our bureaucracy into this century. You may be surprised to know there is plan in place for at least part of the problem, as evidenced by the Integrated Award Environment: the Path Forward from the U.S. General Services Administration (GSA). That document, which looks suspiciously like a Power Point presentation converted to PDF, outlines the GSA’s recommendations for improving the federal government’s acquisition procedures.

Anyone interested in the details should check out the document, but the list of “our principles” summarizes the organization’s targets:

  • Open (source code, data, APIs)
  • Data as an asset
  • Continuous improvement
  • Effective user experience
  • Measurable transactions
  • Security is foundational
  • Build value over maintaining status quo

The paper expounds on each of these points, defining the implications of each goal, a point or two on maintaining balance, and questions workers should ask themselves going forward. For example, the section on “Open” notes that users must balance the stability of, say, Oracle with the agility of open source solutions and security with openness. For the data-enthused among us, the section on “Data as an asset” reads:

“Accurate, timely, complete, and authoritative”


*Significant effort to manage data quality; implementers must have data-oriented SLAs

*Change control of the data needs to be transparent

*Will follow the data->information->knowledge chain Implies


*Our flexibility has to account for the strong change management of our data Balance

Ask ourselves:

*“How do we ensure that we are providing timely and accurate data?”

*“How are we enabling decision-making through use of our data?”

So, next time you’re tempted to think our government is doomed to be stuck in the 20th century, remember that some folks within the bureaucracy are on the case. Soon, it may be time for them to party like it’s 1999.

Cynthia Murrell, April 06, 2014

Sponsored by, developer of Augmentext

New York Public Library Posts Maps

April 5, 2014

The New York Public Library has a massive collection of beautiful maps, but instead of keeping them locked in an archive Motherboard reports, “The New York Public Library Releases 20,000 Beautiful High Resolution Maps.”

All of the 20,000 maps are available via open access. What is even more amazing is that the NYPL decided to release the maps under the Creative Commons CCO 1.0 Universal Public Domain Dedication. If you are unfamiliar with a Creative Commons license, it means that users are free to download content and do whatever they want with it.

“Combined with its existing historical GIS program, the NYPL wants its users to engage with the maps, and allows them to warp (fitting together based on corresponding anchor points) and overlay the historic maps with modern geoweb services like Google and Open Street Map. Users can export WMS, KML files, and high-quality TIFFs. The historic map appears side by side with the modern maps, and users are invited to mark corresponding points on each, so you can overlay the historic map over the current day’s.”

Google Maps using old maps to explore the world of the past. It is yet another amazing use of modern technology and makes one wonder what people of yesterday would have thought about exploring their world via a small box.

Whitney Grace, April 5, 2014

Darpa Prods Big Data Experts

March 29, 2014

I read “Darpa Calls for Advanced Big Data Ideas.” If the write up is accurate, Darpa is not on board with the marketing innovations about Big Data, whatever the term means. Darpa wants more. According to the TechRadar story:

According to V3, DARPA director Arati Prabhakar told a briefing on emerging threats with the House Armed Services Committee’s Subcommittee on Intelligence that it is looking to come up with some advanced big data ideas. She said that DARPA is creating a new set of cyber security capabilities that will ensure that networked information is trustworthy.

Address “big data” may be easier if those talking about it would define the term and the context in which the phrase is being used. Those who chant “Big Data,” including Darpa, are just empowering the sales people, the self appointed experts, and the failed middle school teachers who write “reports” for mid tier consulting firms.

Stephen E Arnold, March 29, 2014

US Government Content Processing: A Case Study

March 24, 2014

I know that the article “Sinkhole of Bureaucracy” is an example of a single case example. Nevertheless, the write up tickled my funny bone. With fancy technology,, and the hyper modern content processing systems used in many Federal agencies, reality is stranger than science fiction.

This passage snagged my attention:

inside the caverns of an old Pennsylvania limestone mine, there are 600 employees of the Office of Personnel Management. Their task is nothing top-secret. It is to process the retirement papers of the government’s own workers. But that system has a spectacular flaw. It still must be done entirely by hand, and almost entirely on paper.

One of President Obama’s advisors is quote as describing the manual operation as “that crazy cave.”

And the fix? The article asserts:

That failure imposes costs on federal retirees, who have to wait months for their full benefit checks. And it has imposed costs on the taxpayer: The Obama administration has now made the mine run faster, but mainly by paying for more fingers and feet. The staff working in the mine has increased by at least 200 people in the past five years. And the cost of processing each claim has increased from $82 to $108, as total spending on the retirement system reached $55.8 million.

One of the contractors operating the system is Iron Mountain. You may recall that this outfit has a search system and caught my attention when Iron Mountain sold the quite old Stratify (formerly Purple Yogi automatic indexing system to Autonomy).

My observations:

  1. Many systems have a human component that managers ignore, do not know about, or lack the management horsepower to address. When search systems or content processing systems generate floods of red ink, human processes are often the culprit
  2. The notion that modern technology has permeated organizations is false. The cost friction in many companies is directly related to small decisions that grow like a snowball rolling down a hill. When these processes reach the bottom, the mess is no longer amusing.
  3. Moving significant information from paper to a digital form and then using those data in a meaningful way to answer questions is quite difficult.

Do managers want to tackle these problems? In my experience, keeping up appearances and cost cutting are more important than old fashioned problem solving. In a recent LinkedIn post I pointed out that automatic indexing systems often require human input. Forgetting about those costs produces problems that are expensive to fix. Simple indexing won’t bail out the folks in the cave.

Stephen E Arnold, March 24, 2014

Stephen E Arnold, March 24, 2014

Tech Knows Best Asserts a Google Employee

March 21, 2014

I highly recommend that you read Jacques Ellul’s The Technological Bluff (La Bluff technologique) (1988) available from Amazon and at this link as of March 21, 2014.

I read “Occupy Founder Calls on Obama to Appoint Eric Schmidt ‘CEO of America’.” According to the write up, Justine Tunney, a Google engineer (who must be really smart by definition, right?) “is demanding that the tech industry take over the US government.”

Like Swift’s “A Modest Proposal,” I find the idea interesting. No doubt Mr.Schmidt is flattered by one of the Google elite’s idea. According to the write up:

Yasha Levine, a reporter for Silicon Valley publication Pando Daily, noted the seeming discrepancy between Tunney’s former anarchist beliefs and her current role at Google. Since her arrival at the firm, he writes, “she has become an astroturfer par excellence for the company, including showing up in a comment section to bash my reporting on Google’s vast for-profit surveillance operation.”

Amusing in a way. Now back to Ellul. His informed monograph points out that solving a problem via technology and technologists may not deliver the solution anticipated. I am confident that Justine Tunney is familiar with Ellul’s viewpoint and rejects it.

An old French theologian-philosopher is definitely not Google material. I would suggest that the alleged recommendations  to retire all government employees with full pensions, transfer administrative authority to the tech industry, and appoint Eric Schmidt CEO of America are rational within the Googley world.

For an old person in rural Kentucky, the ideas seem to ignore Jacques Ellul’s insights and remind me of the crazy lists that IDC type writers cook up to seem informed.

Around the cast iron stove in Harrod’s Creek, the ideas might be greeted with considerable skepticism. If you work at an outfit that wants to defeat death and build a phone inside a human body via self assembly nanotech, Justine Tunney’s ideas make perfect sense. At least that’s my assumption.

Stephen E Arnold, March 21, 2014

Google: A Small Discontinuity

March 20, 2014

I read the BBC summary of Larry Page’s TED interview. You can find the BBC write up in this story: “Ted 2014: Larry Page on Google’s Robotic Future.” The statement in the allegedly accurate article I noted is [emphasis added by me]:

Mr Page was also asked about the Edward Snowden revelations, following a surprise appearance from the whistle-blower at Ted. “It is disappointing that the government secretly did this stuff and didn’t tell us about it,” said Mr Page. “It is not possible to have a democracy if we have to protect our users from the government. The government has done itself a tremendous disservice and we need to have a debate about it,” he added.

Clear enough, if accurate.

Next, I noted “US Tech Giants Knew of NSA Data Collection, Agency’s Top Lawyer Insists.” Again I don’t know if the information in the Guardian article is accurate. Nevertheless, I noted this passage:

Asked during a Wednesday hearing of the US government’s institutional privacy watchdog if collection under the law, known as Section 702 or the Fisa Amendments Act, occurred with the “full knowledge and assistance of any company from which information is obtained,” De replied: “Yes.” [Rajesh De is the NSA general counsel]

The Guardian story then tosses in:

Google, Microsoft, Facebook and AOL – claimed they did not know about a surveillance practice described as giving NSA vast access to their customers’ data. Some, like Apple, said they had “never heard” the term Prism.

A small thing. A discontinuity. Probably just a misunderstanding. It is more fun to think about Google smart watches, Google balloons, and improving “search.” Precision, recall, accuracy—hmmm.

Stephen E Arnold, March 290, 2014

Digital Government Makes People Unhappy

March 17, 2014

A GCN headline states that “Report Finds US Citizens Unhappy With Digital Government.” All we can say to this is we are not surprised. The Accenture report titled: “Digital Government: Pathways To Delivering Public Services For The Future” says that the US ranks sixth in government using social media and digital services to communicate with people.

US citizens are apparently uncomfortable with using mobile and cloud technology to communicate with the government.

The government launched 140 free apps in both English and Spanish that deal with government services, but 43 percent of the US does not to use them. As for the cloud, US citizens fear that security is not tight enough and their privacy rights are not protected. The report does offer three priorities that the US population wants the government to focus on:

“According to U.S. citizens, the top three priorities for improving future public services are to provide cost-efficient, sustainable services, to deliver a clear and stable long-term vision and to better understand better the priorities of citizens and communities.”

What exactly does that mean? It does not even add up to three! It sounds like a whole bunch of jib jab or a company’s bland mission statement. The US is never satisfied.

Whitney Grace, March 17, 2014
Sponsored by, developer of Augmentext Listing May Be Enough

March 8, 2014

I am all for slipshod work, particularly when delivered by government contractors. Hey, the emphasis is on scope changes and engineering change orders, not on delivering what the wild and crazy statement of work requires.

I was delighted to read the Hacker News thread at about broken links and missing data sets on at The thread contains a number of interesting comments. These may be evidence that substandard attention to detail suggests digital eczema. Just Bing it.

Examples range from corrected links that fail to odd ball outputs. See, for example, There are some gems in the comments; for instance,

In the early days of, some effort was expended to minimize the number of dead links on US government servers. In the present incarnation as, there are some interesting changes.

My view is that the dead links are a lesser problem than content that is no longer available and to which the links have been removed. If I were younger, I would suggest that you, gentle reader, look for information about MIC, RAC, and ZPIC contract awards. But I will not.

Stephen E Arnold, March 8, 2014

DARPA Hints That Search Fails

February 11, 2014

One of my two or three readers sent me a link to “DARPA-BAA-14-21: Memex.” The item is interesting because it reaches back to the idea of Vannevar Bush, sidesteps the use of the word “Memex” by a search vendor once operating in the United Kingdom, and provides pretty clear proof that DARPA is not happy with search. You can dig into the details at

US government content has some interesting characteristics. One of the most interesting is that items like DARPA-BAA-14-21 appear without context. For example, there is not a hint, nary a whisper of In-Q-Tel’s investments in search and content processing. Years ago, I heard at an intel conference that In-Q-Tel funds promising companies but few of these deliver operational payoffs. You can see a list of In-Q-Tel investments at Some of these companies deliver darned interesting demonstration systems. Others have offered solutions that were eventually abandoned. Others are  like Fourth of July fireworks; that is, the financial support and walk arounds provide the type of show that some decision makers perceive as progress and purposeful action.

The net net is that this DARPA item underscores that information retrieval system is not appropriate for the future needs of DARPA. For me, this is one indication that my assertion about the troubled state of information retrieval.

Perhaps the funding, the TREC tests, and the DARPA solicitation will yield a payoff for operational personnel. “Perhaps” is a bit soft even if the devalued dollars are real. Our research offers some interesting facts that finding information today is more difficult than it was five years ago.

Stephen E Arnold, February 11, 2014

Next Page »