OKE2017 Challenge – ESWC 2017

Open Knowledge Extraction (OKE) Challenge – ESWC 2017

Challenge Motivation

The Open Knowledge Extraction Challenge invites researchers and practitioners from academia as well as industry to compete to the aim of pushing further the state of the art of knowledge extraction for the Semantic Web. The Open Knowledge Extraction Challenge is accepted at ESWC 2017.

Most of the Web content consists of natural language text, e.g., websites, news, blogs, micro-posts, etc., hence a main challenge is to extract as much relevant knowledge as possible from this content, and publish it in the form of Semantic Web triples. There is huge work on knowledge extraction (KE) and knowledge discovery contributing to address this problem. In fact, results of knowledge extraction systems are usually evaluated against tasks that do not focus on specific Semantic Web goals. For example, tasks such as named entity recognition, named entity disambiguation and relation extraction are certainly of importance for the SW, but in most cases such tasks are designed without considering the output design and formalization in the form of Linked Data and OWL ontologies. This makes results of existing methods often not directly reusable for populating the SW, until a translation from linguistic semantics to formal semantics is performed.

Challenge Overview

The goal of the Open Knowledge Extraction Challenge is to test the performance of Knowledge Extraction Systems with respect to the Semantic Web. The OKE challenge has the ambition to provide a reference framework for research on “Knowledge Extraction from text for the Semantic Web” by redefining a number of tasks (typically from information and knowledge extraction) by taking into account specific Semantic Web requirements.


For more information check out the forum google groups oke2017 or send an e-mail to: OKE-contact@googlegroups.com 

Important Dates

Paper submission deadline: Friday March 10th, 2017
Challenge paper reviews: Tuesday April 5th, 2017
Paper Notifications and invitation to task: Friday April 7th, 2017
Camera ready papers (5 pages document): Sunday April 23rd, 2017
Release of training data and instructions: Friday January 13th, 2017
Release of test dataset: Friday April 7th, 2017
Deadline for submission of system answers: TBA
Release of evaluation results: TBA
Submission of camera-ready papers: Monday April 24th, 2017
Proclamation of winners: During ESWC2017 closing ceremony

Tasks and Training Data

  • Task 1: Focused NE Identification and Linking
  • Task 2: Broader NE Identification and Linking
  • Task 3: Focused Musical NE Recognition and Linking
  • Task 4: Knowledge Extraction

Prerequisites for participation

Each participant must provide a system that has to be a downloadable Docker image and is NIF-based for the input and output. For more details read the Hobbit project platform wiki.

Registration and Submission

  • All challenge papers should be exactly five (5) pages in length in PDF file format and written in English.
  • In order to prepare their manuscripts, authors should follow Springer’s Lecture Notes in Computer Science (LNCS) style. For details and templates see Springer’s Author Instructions.
  • Paper submissions will be handled electronically via the EasyChair conference management system, available at the following address: https://easychair.org/conferences/?conf=oke2017.
  • Papers must be submitted no later than Friday March 10th, 2017, 23:59 Hawaii Time.
    NOTE: Eligible to submit papers are only authors participating in the challenge.
  • Each submission will be peer-reviewed by members of the challenge program committee.  Papers will be evaluated according to their significance, originality, technical content, style, clarity, and relevance to the challenge.
  • Each participant must provide his/her solution as a docker image that abides by the technical specifications.


The organization responsibility will be shared by the following four main organizers:

  • René Speck, University of Leipzig, Germany
    Expertise: Knowledge extraction, Named Entitiy Recognition, Relation Extraction
    Website: http://aksw.org/RenSpeck
    Email: speck@informatik.uni-leipzig.de
  • Michael Röder, University of Leipzig, Germany
    Expertise: Benchmarking and Topic Modeling
    Website: http://aksw.org/MichaelRoeder
    Email: roeder@informatik.uni-leipzig.de
  • Ricardo Usbeck, University of Leipzig, Germany
    Expertise: Knowledge extraction and Question Answering
    Website: http://aksw.org/RicardoUsbeck
    Email: usbeck@informatik.uni-leipzig.de
  • Axel-Cyrille Ngonga Ngomo, Institute for Applied Informatics, Germany
    Expertise: Knowledge extraction, Machine Learning, Question Answering, Information Retrieval
    Website:  http://aksw.org/AxelNgonga
    Email: ngonga@informatik.uni-leipzig.de
  • Horacio Saggion, Universitat Pompeu Fabra, Spain
    Expertise: Natural Language Processing and Human Language Technology
    Website: http://www.taln.upf.edu/users/hsaggion
    Email: horacio.saggion@upf.edu
  • Luis Espinosa-Anke, Universitat Pompeu Fabra, Spain
    Expertise: Natural Language Processing and Information Extraction
    Website: http://www.taln.upf.edu/users/lespinosa
    Email: uis.espinosa@upf.edu
  • Sergio Oramas, Universitat Pompeu Fabra, Spain
    Expertise: Information Extraction and Music Information Retrieval
    Email: sergio.oramas@upf.edu

The following set of people comprises a tentative list of programme committee members. All the members of the list are experts from research and industry, who will review the paper submissions, independently of the organization team. From the following list, the 11 members that are marked with an asterisk have already agreed on participating in the Open Knowledge Extraction Challenge as PC members. More members will be added.

  1. Adeline Nazarenko, University Paris 13, FR
  2. Andrea Nuzzolese*, National Research Council Rome, IT
  3. Anna-Lisa Gentile*, University of Mannheim, DE
  4. Borja Navarrao, University of Alicante, ES
  5. Davide Buscaldi*, University Paris 13s, FR
  6. Diana Maynard, University of Sheffield, UK
  7. Elena Cabrio, University of Nice Sophia Antipolis, FR
  8. Elena Montiel Ponsoda, Universidad Politécnica de Madrid, ES
  9. Fabian M. Suchanek, Télécom ParisTech University, FR
  10. Francesco Barbieri*, Universitat Pompeu University, ES
  11. Isabelle Augenstein, University of Sheffield, UK
  12. Johannes Hoffar*, Max Planck Institute, DE
  13. Johanna Völker, University of Mannheim, DE
  14. José Camacho-Collados*, Sapienza University of Rome, IT
  15. Luciano Del Corro*, Max Planck Institute, DE
  16. Malvina Nissim, University of Bologna, IT
  17. Michel Buffa, University of Nice Sophia Antipolis, FR
  18. Paola Velardi, University of Roma La Sapienza, IT
  19. Philipp Cimiano*, Bielefeld University, DE
  20. Raphaël Troncy*, EURECOM, FR
  21. Sergio Consoli*, Data Science Department at Philips Research, DE
  22. Thierry Declerck, Saarland University, DE
  23. Valerio Basile, University of Groningen, NL
  24. Xavier Serra*, Universitat Pompeu Fabra, ES