Nowcasting: Lots of Behind the Scenes Human Work Necessary
September 10, 2014
Some outfits surf on the work of others. A good example is the Schubmehl-Arnold tie up. Get some color and details here.
Other outfits have plenty of big thinkers and rely on nameless specialists to perform behind the scenes work.
A good example of this approach is revealed in “Predicting the Present with Bayesian Structural Time Series.” The scholarly write up explains a procedure to perform “nowcasting.” The idea is that one can use real time information to help predict other now happenings.
Instead of doing the wild and crazy Palantir/Recorded Future forward predicting, these Googlers focus on the now.
I am okay with whatever outputs predictive systems generate. What’s important about this paper is that the authors document when humans have to get involved in the processes constructed from numerical recipes known to many advanced math and statistics whizzes.
Here are several I noted:
- The modeler has to “choose components for the modeling trend.” No problem, but it is tedious and important work. Get this step wrong and the outputs can be misleading.
- Selecting sampling algorithms, page 6. Get this wrong and the outputs can be misleading.
- Simplify by making assumptions, page 7. “Another strategy one could pursue (but we have not) is to subjectively segment predictors into groups based on how likely the would be to enter the model.”
- Breaking with Bayesian, page 8. “Scaling by “s^2/y”* is a minor violation of the Bayesian paradigm because it means our prior is data determined.”
There are other examples. These range from selecting what outputs from Google Trends and Correlate to use to the sequence of numerical recipes implemented in the model.
My point is that Google is being upfront about the need for considerable manual work in order to make its nowcasting predictive model “work.”
Analytics deployed in organizations depend on similar human behind the scenes work. Get the wrong thresholds, put the procedures in a different order, or use bad judgment about what method to use and guess what?
The outputs are useless. As managers depend on analytics to aid their decision making and planners rely on models to predict the future, it is helpful to keep in mind that an end user may lack the expertise to figure out if the outputs are useful. If useful, how much confidence should a harried MBA put in predictive models.
Just a reminder that ISIS caught some folks by surprise, analytics vendor HP seemed to flub its predictions about Autonomy sales, and the outfits monitoring Ebola seem to be wrestling with underestimations.
Maybe enterprise search vendors can address these issues? I doubt it.
Note: my blog editor will not render mathematical typography. Check the original Google paper on page 8, line 4 for the correct representation.
Stephen E Arnold, September 10, 2014
Comments
One Response to “Nowcasting: Lots of Behind the Scenes Human Work Necessary”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
[…] the state of the art. I have discussed the approach in my analysis of Google’s nowcasting model here. Only hitch? Well, it is not right when it counts: Ebola threat, ISIS/ISIL, horse […]