Analysis of Aardvark and Crowdsourcing Answers to Questions
February 9, 2010
We received a copy of the paper “Anatomy of a Large-Scale Social Search Engine”.
The information was interesting. The idea is that individuals who have an intereset in answering questions related to their area of expertise provides an adjunct to other research methods. The paper explains the method used to determine who should answer a question and the other components of the “social search system”.
If you have not visited the Aardvark Web site, you will want to take a look at the service. The url is www.vark.com. The screen shot below shows the search box and the firm’s explanation of what happens when you ask a question of the Aardvark question answering community.
The informatoin in the paper suggests that members’ question are more lengthy than a query sent to Google. Instead of two or three words, think about a sentence with a dozen or more words. In addition, the answerer and the questioneer can enter into a conversation which further disambiguates and tugs out the needed information.
All good.
When we talked about this paper at lunch, the goslings asked a number of rhetorical questions. I want to share three of these with you, so you can think about the Aardvark paper and your own experience with question answering systems:
- The people answering questions have self selected to answer questions. When the method is moved to a more general audience—for example, Facebook or Orkut—will the metrics in the paper be congruent with the broader community’s behavior? (In the self selected community about half of those registered did not ask or answer a question according to the paper.)
- In question answering systems, how will disinformation be identified and filtered? (Some government entities, not the US in my opinion, could inject intentionally shaped information which the questioner could accpet as fact than pass along as accurate information.)
- Pre computing certain values is one way to minimize computational load; however, over time an expert may acquire additional domains of expertise. How can the system adapt and get “old” experts with “new” informatoin on the roster for certain questions?
We think the Aardvark service is interesting and the paper stimulated our thinking.
Stephen E Arnold, February 9, 2010
No one paid us to read and write about this paper. I will report this to the National Parks Service, an outfit familiar with crowds in Yellowstone and elsewhere.
Comments
One Response to “Analysis of Aardvark and Crowdsourcing Answers to Questions”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
I have been answering Aardvark questions for a couple of months now. I find it interesting to see the range of questions out there and don’t mind a couple of minutes to consider and type out a general answer. Bottom line is, it’s basically free consulting.
What I do now is post the Q&A on my blog (http://blog.spidergourmet.com), so I get to update that frequently as well. I would say it’s my return for providing free consulting, that I get to build up the content on my blog, and also open myself to the opportunity to receive prospective business, although so far that hasn’t even come close to being realized!