Algorithms Can Deliver Skewed Results
April 18, 2012
After two days of lectures about the power of social media analytics, Stephen E Arnold raised doubts about the reliability of certain analytics outputs. He opined: “Faith in analytics may be misplaced.”
Arnold’s lecture focused on four gaps in social media analytics. He pointed out that many users were unaware of the trade offs in algorithm selection made by vendors’ programmers. Speaking at the Social Media Analytics Summit, he said:
Many companies purchase social media analytics reports without understanding that the questions answered by algorithms may not answer the customer’s actual question.
He continued:
The talk about big data leaves the impression that every item is analyzed and processed. The reality is that sampling methods, like the selection of numerical recipes can have a significant impact on what results become available.
The third gap, he added, “is that smart algorithms display persistence. With smart software, some methods predict a behavior and then look for that behavior because the brute force approach is computationally expensive and adds latency to a system.” He said:
Users assume results are near real time and comprehensive. The reality is that results are unlikely to be real time and built around mathematical methods which value efficiency and cleverness at the expense of more robust analytic methods. The characteristic is more pronounced in user friendly, click here type of systems than those which require to specify a method using SAS or SPSS syntax.”
The final gap is the distortion that affects outputs from “near term, throw forward biases.” Arnold said:
Modern systems are overly sensitive to certain short term content events. This bias is most pronounced when looking for emerging trend data. In these types of outputs the “now” data respond to spikes and users act on identified trends often without appropriate context.
The implication of these gaps is that outputs from some quite sophisticated systems can be misleading or present information as fact when that information has been shaped to a marketer’s purpose.
The Social Media Analytics conference was held in San Francisco, April 17 and 18, 2012. More information about the implications of these gaps may be found at the Augmentext.com Web site.
Donald C Anderson, April 18, 2012,
Sponsored by Pandia.com
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
