Recorded Future: Google and Cyber OSINT
February 2, 2015
I find the complaints about Google’s inability to handle time amusing. On the surface, Google seems to demote, ignore, or just not understand the concept of time. For the vast majority of Google service users, Google is no substitute for the users’ investment of time and effort into dating items. But for the wide, wide Google audience, ads, not time, are more important.
Does Google really get an F in time? The answer is, “Nope.”
In CyberOSINT: Next Generation Information Access I explain that Google’s time sense is well developed and of considerable importance to next generation solutions the company hopes to offer. Why the craw fishing? Well, Apple could just buy Google and make the bitter taste of the Apple Board of Directors’ experience a thing of the past.
Now to temporal matters in the here and now.
CyberOSINT relies on automated collection, analysis, and report generation. In order to make sense of data and information crunched by an NGIA system, time is a really key metatag item. To figure out time, a system has to understand:
- The date and time stamp
- Versioning (previous, current, and future document, data items, and fact iterations)
- Times and dates contained in a structured data table
- Times and dates embedded in content objects themselves; for example, a reference to “last week” or in some cases, optical character recognition of the data on a surveillance tape image.
For the average query, this type of time detail is overkill. The “time and date” of an event, therefore, requires disambiguation, determination and tagging of specific time types, and then capturing the date and time data with markers for document or data versions.
A simplification of Recorded Future’s handling of unstructured data. The system can also handle structured data and a range of other data management content types. Image copyright Recorded Future 2014.
Sounds like a lot of computational and technical work.
In CyberOSINT, I describe Google’s and In-Q-Tel’s investments in Recorded Future, one of the data forward NGIA companies. Recorded Future has wizards who developed the Spotfire system which is now part of the Tibco service. There are Xooglers like Jason Hines. There are assorted wizards from Sweden, countries the most US high school software cannot locate on a map, and assorted veterans of high technology start ups.
An NGIA system delivers actionable information to a human or to another system. Conversely a licensee can build and integrate new solutions on top of the Recorded Future technology. One of the company’s key inventions is numerical recipes that deal effectively with the notion of “time.” Recorded Future uses the name “Tempora” as shorthand for the advanced technology that makes time along with predictive algorithms part of the Recorded Future solution.
What does Recorded Future provide? It functions as a prediction solution. Predictions are based on probability. Recorded Future determines based on processing of historical and real time data flows what high probability events may occur. The predictions are anchored in terms of time. In fact, time may be the key differentiator between the first generation Autonomy system that kicked off the NGIA revolution and where NGIA companies are moving as quickly as they can.
Because Google owns a chunk of Recorded Future, it does not take much investigation to learn that Recorded Future can make use of Google functions. Recorded Future also makes use of some Amazon data. There are other inputs as well, including social media, the licensees proprietary content, and data from third party providers. The value of Recorded Future is that it takes the mid 1990s timeline approach of IBM i2 and moves the utility and functionality to the 21st century.
You can work through the visualization features which push the needle beyond the Spotfire outputs in the CyberOSINT chapter devoted to Recorded Future. Most of the information has not previously been gathered and presented in terms of law enforcement and intelligence functionality. I placed Autonomy in CyberOSINT because it was the trigger point. I devoted a full chapter to Recorded Future because it points the way to the generation of temporal outputs that can be used for on the fly decision making by a user with access only to a mobile device or as inputs directly into an intelligent, autonomous system.,
In short, Google recognized years ago that the temporal factor was important. Like In-Q-Tel, Google wanted to make sure it had access to the time capabilities that are inappropriate for users of an ad supported search system.
How can you learn more about Recorded Future? The easiest way is to work through the information in my monograph and review the Recorded Future Web site.
What does this mean for critics of Google who enjoy pointing out that Google is clueless about time? First, it means the “experts” don’t know beans about time functionality. Second, it means that Google is already active in next generation functionality. Third, it suggests that Google is big, has some problems, retains its ability to identify and acquire a stake in a company that is defining where NGIA systems will be moving at a quickening pace.
Stephen E Arnold, February 2, 2015
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
