PolySpot Solutions Enable Data Scientists to Arrive at More Accurate Conclusions
February 11, 2013
We all know that big data means more information but as a recent article “Beware the Errors of ‘Big Data’” from Wired points out, there is also the potential for more false information to arise because of big data.
Antifragile, a concept explored in the article, is having the unique position and ability to be be capable of benefiting from uncertainty and complexity at the expense of others. Big data researchers or data scientists have the power to choose to stop their research and inquiries when they have the right result that they need.
We learned more about this perspective that more data does not automatically produce meaningful and correct information:
But beyond that, big data means anyone can find fake statistical relationships, since the spurious rises to the surface. This is because in large data sets, large deviations are vastly more attributable to variance (or noise) than to information (or signal). It’s a property of sampling: In real life there is no cherry-picking, but on the researcher’s computer, there is. Large deviations are likely to be bogus.
While there is certainly the potential for improper conclusions to be drawn from data due to the nature of decision-making, this could mean that data scientists need to rely further on technology to aid them in this process. Solutions like PolySpot enable departments across the enterprise to tap into the insights produced by big data and the organization wide access offers the transparency and accountability organizations are looking for.
Megan Feil, February 11, 2013
Sponsored by ArnoldIT.com, developer of Beyond Search.
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
