IBM Socrates Wins 2017 Semantic Web Challenge
January 10, 2018
We learn from the press release “Elsevier Announces the Winner of the 2017 Semantic Web Challenge,” posted at PRNewswire, that IBM has taken the top prize in the 2017 Semantic Web Challenge world cup with its AI project, Socrates. The outfit sponsoring the competition is the number one sci-tech publisher, Elsevier. We assume IBM will be happy with another Jeopardy-type win.
Knowledge graphs were the focus of this year’s challenge, and a baseline representing current progress in the field was established. The judges found that Socrates skillfully wielded natural language processing and deep learning to find and check information across multiple web sources. About this particular challenge, the write-up specifies:
This year, the SWC adjusted the annual format in order to measure and evaluate targeted and sustainable progress in this field. In 2017, competing teams were asked to perform two important knowledge engineering tasks on the web: fact extraction (knowledge graph population) [and] fact checking (knowledge graph validation). Teams were free to use any arbitrary web sources as input, and an open set of training data was provided for them to learn from. A closed dataset of facts, unknown to the teams, served as the ground truth to benchmark how well they did. The evaluation and benchmarking platform for the 2017 SWC is based on the GERBIL framework and powered by the HOBBIT project. Teams were measured on a very clear definition of precision and recall, and their performance on both tasks was tracked on a leader board. All data and systems were shared according to the FAIR principles (Findable, Accessible, Interoperable, Reusable).
The Semantic Web Challenge has been going on since 2003, organized in cooperation with the Semantic Web Science Association.
Cynthia Murrell, January 10, 2018
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
