Is Precision and Recall Making a Comeback?
March 15, 2011
Microsoft-centric BA Insight explored these touch points of traditional information retrieval. Precision and recall have quite specific meanings to those who care about the details of figuring out which indexing method actually delivers useful results. The Web world and most organizations care not a whit about fooling around with this equation.
And recall. This is another numerical recipe that causes the procurement team’s eyes to glaze.
I was interested to read in The SharePoint and FAST Search Experts Blog’s “What is the Difference Between Precision and Recall?” This is a very basic question for determining the relevance of query search results.
Equations aside, precision is the percentage of relevant retrieved documents, and recall is the percentage of relevant documents that are retrieved. In other words, when you have a search that’s high in precision, your results list will have a large percentage of items relevant to what you typed in, but you may also be missing a lot of items in the total.
With a search that is high in recall, your results list will have more items of what you’re searching for, but will also have a lot of irrelevant items as well. The post points out that determining the usefulness of search results is actually simpler than this sounds:
“The truth is, you don’t have to calculate relevance to determine how SharePoint or FAST search implementation is performing. You can look at a much more telling KPI. Are users actually finding what they are looking for?”
The problem, in my opinion is that most enterprise search deployments lack a solid understanding of the corpus to be processed. As a result, test queries are difficult to run in a “lab-type” setting. A few random queries are close enough for horseshoes. The cost and time required to benchmark a system and then tune it for optimal precision and recall is a step usually skipped.
Kudos to BA Insight for bringing up the subject of precision and recall. My view is that the present environment for enterprise search puts more emphasis on point and click interfaces and training wheels for users who lack the time, motivation, or expertise to formulate effective queries. Even worse, the content processed by the index is usually an unexplored continent. There are more questions like “Why can’t I find that PowerPoint?” that shouts of Eureka! Just my opinion.
Stephen E Arnold, March 15, 2011
Freebie
Comments
2 Responses to “Is Precision and Recall Making a Comeback?”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
[…] Precision and Recall Making a Comeback? http://tinyurl.com/4avwrhj (Stephen […]
[…] Precision and Recall Making a Comeback? http://tinyurl.com/4avwrhj (Stephen […]