NIST Trec Results
February 18, 2013
Short honk: A person heard me give a talk in which I said, “NIST Trec results are not widely available.” I thought the comment was appropriate, but someone thought I was full of goose feathers. I want to point you to the Trec page at http://trec.nist.gov/results.html. On that page one can view the results provided the user has a valid user name and password. In my book, that means “not widely available.” Your mileage may vary. Basic information about TREC is at http://en.wikipedia.org/wiki/Text_Retrieval_Conference, which like most information about search is sort of right and sort of not right.
My hunch about the closed nature of the comprehensive data Trec results spit out is that some outfits participating use math magic to demonstrate a particular system’s performance. My inner goose thinks that the flat lining of precision and recall scores and the data about search and retrieval within certain test sets indicates that search is not making the sort of progress most vendors wish to be true.
Stephen E Arnold, February 18, 2013
Comments
One Response to “NIST Trec Results”
-
- Subscribe to Beyond Search
Feature archive
News archive
-
The TREC results in the archive are the raw submissions: ranked lists of document ids submitted for each topic, or the equivalent as appropriate for the task.
The evaluation output for every run submitted to a TREC conference is included in the appendix to that year’s proceedings. Proceedings are publicly available in the Publications section of the TREC web site.
Ellen Voorhees
TREC project manager
NIST