Palantir: Now an Enterprise App Developer

September 30, 2014

I read “Hush Hush Data Firm Palantir Snags ICE Case Tracking Deal.” Palantir may be moving from supporting intelligence agencies to the market sector dominated by government contractors like SRA, Booz Allen Hamilton, and CACI.

The article states:

Immigration and Customs Enforcement has awarded secretive data-mining firm Palantir a $42 million contract to redo the investigation agency’s failed case filing system.

The challenge will be to make a case management system work in a manner that satisfies the statement of work. Other case management efforts have crashed and burned.

Palantir appears to be working with a tough mandate: On time and on budget delivery. As you may know, the notion of on time and on budget is only valid until the first scope change rolls down the timeline.

Are flaws in case management systems unusual. Nah. The article reveals:

The Justice Department inspector general last week released a report on the FBI’s new case management system, Sentinel, assailing its searching and indexing features for slowing the investigations of special agents and the productivity levels of evidence technicians.

Why are case management systems problematic? I can identify a number of reasons, but it will be more entertaining if I wait for news about the Palantir project’s path.

Stephen E Arnold, October 17, 2014

NSA Catalog Available

September 27, 2014

Short honk: I you want a copy of National Security Agency 2014 Technology Catalog: Technology Transfer Program, you can download it for now from this link. If found pages 26 to 40 fascinating. Will IDC issue its own version of this document, using its surfing technical demonstrated by Dave Schubmehl with my content? I will keep my eye open.

Stephen E Arnold, September 27, 2014

Federal Agencies Perpetually Battle Connectivity Loss

September 24, 2014

This may be stating the obvious, but ComputerWorld declares that “IT Outages Are an Ongoing Problem for the U.S. Government.” The article cites a recent report sponsored by Symantec and performed by MeriTalk, which runs a network for government IT workers. Though the issues that originally plagued were their own spectacular kettle of fish, our federal government’s other computer networks are no paragons of efficiency. Writer Patrick Thibodeau tells us:

“Specifically, the survey found that 70% of federal agencies have experienced downtime of 30 minutes of more in a recent one-month period. Of that number, 42% of the outages were blamed on network or server problems and 29% on Internet connectivity loss….

“The report is interesting because it surveys two distinct government groups, 152 federal ‘field workers,’ or people who work outside the office, and 150 IT professionals. Because the field workers are outside the office, some of the outages may be the result of local connectivity problems at either a hotel, home or other remote site. But, overall, the main reason for loss of access to data was a government outage.”

The write-up goes on to note that most workers can go on with their tasks via some other method, like by telephone (48%), through their personal devices (33%), or with some other workaround like Google Apps (24%). Imagine how much more efficient government workers could be if they were not frequently required to get creative just to do basic parts of their jobs.

Cynthia Murrell, September 24, 2014

Sponsored by, developer of Augmentext

Quote to Note: Google Tells Europe to Embrace Disruption

September 23, 2014

The quote to note is from an EC Web page in an article called “Boosting Digital Europe.”

Europe needs to accept and embrace disruption. The old ways of doing things need to face competition that forces them to innovate. Uber, for example, is shaking up the taxi market — for the good. It offers riders convenience and cheaper fares. Understandably, the incumbent taxi industry is unhappy.

Context: French farmers setting fires. German political tensions increasing. Scotland’s voting to leave the United Kingdom. Portugal, Ireland, Italy, Greece, and Spain. The PIIGS. Well, you get the idea.

And, of course, Google continues to wrestle with the EC’s decidedly un-Googley privacy concerns.

Google wants more disruption. That should play well in Austria and Switzerland, two nation states really into radical change.

Stephen E Arnold, September 23, 2014

Good-Bye Court Documents

September 22, 2014

The Internet makes it easier to access information, including documents from the government. While accessing government documents might cost a few cents, it is amazing that the information can be accessed within a few mouse clicks. BoingBoing, run by the infamous Cory Doctorow, notes that five important US courts are removing their documents from the Internet in “As Office Of US Courts Withdraws Records For Five Top Benches, Can We Make Them Open?”

The court documents are housed on the PACER system, most notable for charging users ten cents a page to access information. Doctorow advocates for free information and stopping governments from spying on its citizens. It is not surprising that he supports reopening these documents, along with the Free Law Project, Internet Archive, and Public.Resource.Org.

The plea reads:

“Our judiciary is based on the idea that we conduct justice public, not in star chambers and smoke-filled back rooms. Our system of justice is based on access to the workings of our courts, and when you hide those workings behind a pay wall, you have imposed a poll tax on access to justice. Aaron [Swartz] and many others believed very deeply in this principle and we will continue to fight for access to justice, equal protection, and due process. These are not radical ideas and the Administrative Office of the U.S. Courts should join us in our commitment.”

Swartz is known for working against Internet censorship bills, so joining Doctorow and the others will get the right backers to make these documents available again. You can fight city hall and win, especially if you are a technology enthusiast with legal aid.

Whitney Grace, September 22, 2014
Sponsored by, developer of Augmentext

Russian Content: Tough to Search If Russia Is Not on the Internet

September 20, 2014

Forget running queries on if Russia disconnects from the Internet. Sure, there may be workarounds, but these might invite some additional scrutiny. Why am I suggesting that some Russian content becomes unsearchable. Well, I believed the story “Russia to Be Disconnected from the Internet.” Isn’t Pravda a go to source for accurate, objective information?

The story asserts:

This is not a question of disconnecting Russia from the international network, yet, Russian operators will need to set up their equipment in a way to be able to disconnect the Russian Internet from the global network quickly in case of emergency, the newspaper wrote. As for the state of emergency, it goes about both military actions and large-scale riots in the country. In addition, the government reportedly discusses a possibility to empower the state with the function to administer domains. Currently this is a function of a public organization – the Coordination Center for the National Domain of the Internet. The purpose of the possible measure is not to isolate Russia from the outside world, but to protect the country, should the USA, for example, decide to disconnect Russia from the system of IP-addresses. It will be possible to avoid this threat, if Russia has a local regulator to distribute IP-addresses inside the country, rather than the ICANN, controlled by the United States government. This requires operators to set up “mirrors” that will be able to receive user requests and forward them to specific domain names.

Interesting. Who is being kept in the information closet? I suppose it depends on one’s point of view. Need an update for Sphinx Search? There will be a solution because some folks will plan ahead.

Stephen E Arnold, September 20, 2014

Feds Warned to Sweat the Small Stuff When Considering Big Data Solutions

September 15, 2014

Say, here’s a thought: After spending billions for big-data software, federal managers are being advised to do their research before investing in solutions. We learn about this nugget of wisdom from Executive Gov in their piece, “Report: Fed Managers Should Ask Data Questions, Determine Quality/Impact Before Investing in Tech.” Writer Abba Forrester sums up the Federal Times report:

“Rutrell Yasin writes that the above managers should follow three steps as they seek to compress the high volume of data their agencies encounter in daily tasks and to derive value from them. According to Shawn Kingsberry, chief information officer for the Recovery Accountability and Transparency Board, federal managers should first determine the questions they need to ask of data then create a profile for the customer or target audience.

“Next, they should locate the data and their sources then correspond with those sources to determine quality of data, the report said. ‘Managers need to know if the data is in a federal system of records that gives the agency terms of use or is it public data,’ writes Yasin.

“Finally, they should consider the potential impact of the data, the insights and resulting technology investments on the agency.”

For any managers new to data management, the article notes they should choose a platform that includes data analysis tools and compiles data from multiple sources into one repository. It also advises agencies to employ a dedicated chief data officer and data scientists/ architects. Good suggestions, all. Apparently, agencies need to be told that a cursory or haphazard approach to data is almost certain to require more time, effort, and expense down the line.

Cynthia Murrell, September 15, 2014

Sponsored by, developer of Augmentext

Is Search Too Hard?

August 26, 2014

I find the readers who send me links to the UK Daily Mail stories helpful. Are these referrers easily fooled?

The story in question has a Google friendly headline:

‘It’s all been a big lie!’ Obama administration lawyer now admits ‘missing’ Lois Lerner emails WERE backed up but claims it’s too hard to search for them”

The US government is a busy beaver when it comes to search. You can explore at your leisure or seek information on myriad dot Gov Web sites without my inputs.

Here’s a passage from the write up. You determine if it is on the money:

‘The Department of Justice attorney told the Judicial Watch attorney on Friday.’ Fit ton said during a Monday afternoon Fox News broadcast, ‘that it turns out the federal government backs up all computer records in case something terrible happens in Washington and there is a catastrophe, so the government can continue operating.’ The catch, he added, is that the DOJ attorney also claimed ‘it would be too hard to go and get Lois Lerner’s emails from that backup system.’

This search and retrieval stuff seems to be difficult. Perhaps these folks should turn to a real expert like Dave Schubmehl, the Arnold surfer for real insight?

Stephen E Arnold, August

Short Honk: Surveillance Database Report

August 26, 2014

I wanted to document a report that ICREACH exists. For information, see The Intercept’s report. No further comment from Beyond Search.

Stephen E Arnold, August 26, 2014

Let Google Do It: Outsource the Government

August 1, 2014

I saw a discussion thread describing a proposed action to allow Google to become the National Technical Information Service. Although I served on the Board of NTIS years ago, I don’t think too much about the operation. Apparently Google does. Apparently there are some folks who ignore the repository aspect of NTIS. Google is search and pretty darned good the argument goes.

The idea is presented in S 2206, but the officials elected by the people are occupied with a number of weighty issues.

I did not this item about the efficacy of US government management. Take a look at “Poorly Managed Construction Cost $840 Million, Watchdog Finds.” A billion here and a billion there may not be a big deal as the economy improves according to some pundits.

What Google does not do, perhaps will? What about the Library of Congress, various government document repositories, and, of course, the funding entities themselves?

Let Google do it? Why not?

Stephen E Arnold, August 4, 2014

Next Page »