Information Overload Is a Filter Problem
September 23, 2008
I just clicked through some of the hundreds of posts about the Google Android phone. I was startled by the redundancy in the posts. There were some useful items buried in the flood of messages, but none of these added to my understanding of this Google initiative.
Bored and underwhelmed, I turned my attention to other information snagged by my newsreader. My eye was hooded by the video of Clay Shirky’s keynote at Web 2.0 Expo during the week of September 15. You can watch the talk here. The snippet that caught my attention was this remark:
Privacy is a way of managing information flow. The inefficiency of information flow wasn’t a bug, it was a feature.
I found this comment somewhat disturbing even though I agree with most of Mr. Shirky’s comments. Here’s what troubled me:
- Information flow in today’s volumes are largely unexpected and not fully understood. As a result, most organizations and experts don’t know how to address the issues of data flow scale in a helpful manner. Social “voting”, old fashioned key word filters, and zingy visualizations of hot spots can help as well as give users a sense of false confidence. That’s risky.
- Enterprise and consumer systems are mostly toys in that only a handful of services can operate at petabyte scale. Even mid sized businesses are struggling with terabyte flows and most tools are not very good, economical, or easy to use. I am concerned about the assumption that these systems deliver good enough solutions. I don’t think these systems do. Example: the financial crashes caused by flawed models’ ability to pinpoint significant data fed into them. A trillion dollar mistake strikes me as a relatively big problem.
- Social media is one tool, and it is [a] not understood, [b] immature, and [c] chock full of potential weaknesses. Many of these issues–such as security–will be addressed over time. For now, I think the risks in regulated companies may outweigh the benefits. Another silver bullet shifts the focus from problem solving to a quick fix.
The final issue I have is that I don’t have an answer to this question: “When I don’t know what I need to answer my question, what do I filter in and out?” Information does not behave like some other human constructs. For example a doctor who misdiagnoses a problem, prescribes the wrong treatment, and assumes her solution is the right one can injure, maybe kill, a patient. The doctor filtered information, but the decision was not optimal.
I am not yet convinced that this “social” trends in information will do much to alleviate the severe information problems that face most organizations. I am certainly not trendy, and I need to see fungible evidence that the payoffs are substantive, not just another wagon load of baloney sold to pump cash into vendors’ threadbare pockets.
Stephen Arnold, September 23, 2008
Comments
5 Responses to “Information Overload Is a Filter Problem”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
Great post. Note that I am an academic science librarian, so my view is distorted through that lens. Concerning the collapse of the financial markets, it was caused by an “irrational exuberance” that house prices would continue to increase MUCH greater than inflation, thus mortgage bankers assumed that people with low credit scores would have no problem with variable rate loans, or interest only loans. This flawed logic created the credit crisis.
Concerning the problem of filtering terabyte and petabyte amounts of data, librarians have done a reasonably good job of creating metadata that helps people figure out what 500 page book would be useful from maybe 5 lines of subject headings. What the information world needs now is better evaluation and indexing tools, in addition to the searching and finding tools.
As a science librarian, I know there is a TON of scientific data behind all of the research and journal articles. There are plenty of publishers, vendors, software companies and people who have different ways of approaching the problem. I am not sure how the search and find market will shake out in 5-10 years, but I bet Google is in the mix somewhere. Let’s just home they don’t do anything evil with all of the data they have.
There is another problem. Filtering information you post on the web. People are too lazy to find tools that will protect their phone, email or credit cards. There are tools though and some of them are free. The main issue is how to educate the user. Check out http://blog.arzoola.com/. They are trying to take a lead on educating users on Privacy 2.0 thingy.
[…] Information Overload Is a Filter Problem […]
There is machine traffic and then there is human traffic – they are not
the same – they are not even close.
It’s like the difference between the pressure waves of voice passing
through air from my mouth to your ear and the dialog we are having.
A key question in my mind is what is a human level network? Can it
emerge from the machine level network?
I believe that when crossing over from the machine to the human – on
that borderline lurks the fundamental flaw which was designed in as a
feature – you cannot and should not build the infrastructure for human
interaction as you do for machines – they are fundamentally different
stuff altogether. Humans are intelligent beings with free will – there is no good-for-all, or even reasonable-to-many, way to provision them with the right piece of information at the right time – all the time.
There is no standard user – no such thing – and users expect something different by definition. Generalizing, coming top down trying to figure out what question to answer (at the machine level there is always a predefined question (or a valid way to generate it) to answer) cannot answer a question that only I can ask – in my context! – This is the key: my questions are asked in my context – the only way to filter out what is not a valid FOR ME answer is in my context.
To cross to the human network we need to build something much more
organic, something in which a unique solution, built of many micro
elements running all over the place – collaborating while serving
individual users in their own individual way – all this must happen on
top of a contextification strata so there is a chance for a human level
dialog in any meaningful sense.
[…] Information Overload Is a Filter Problem by Stephen E. Arnold 1-Sentence Summary* "The final issue I have is that I don’t have an answer to this question: “When I don’t know what I need to answer my question, what do I filter in and out?” Indeed, this is a critical question that kick starts the thinking process. A good starting point is to map out the problem and identify as many attributes and components of the question. Mind mapping facilitates identification and organization of complex ideas into clearly defined concepts and ideas. Using mind mapping tools helps immensely to portray the problem by using keywords, short phrases, and pictures which all are interconnected. The strength of this approach is its simplicity. Through associative thinking, brainstorming, any problem will reveal links with relevant concepts and relationships. In fact, visually mapping a problem serves as an effective filter to frame the issues and validate understanding of the problem. Once this is done it is much easier to search for answers. * 1-Sentence Summary is done using Context Organizer. Download Context Organizer today. […]