Open Knowledge Extraction (OKE) Open Challenge 2017-2018

Challenge Motivation

After a successful organization of the OKE challenge by the HOBBIT project (https://project-hobbit.eu/) at the ESWC 2017, HOBBIT is proud to announce that OKE will be launched in September as an open challenge. The OKE Open Challenge will ensure continuous participation and systems evaluation.
A monetary prize of at least 250€ will be provided.
Stay tuned to get ready to participate!        

 Challenge Overview

The aim of this challenge is to test the performance of knowledge extraction systems in aspects that are relevant for the Semantic Web. These include precision, recall and runtime. The challenge will test the systems against data derived from several datasets and will comprise the following tasks:

  • Task 1 Focused Named Entity Identification and Linking
  • Task 2 Broader Named Entity Identification and Linking
  • Task 3 Focused Musical Named Entity Recognition and Linking

We herewith invite system developers to participate in the aforementioned tasks. To ensure that the system results are comparable, we will provide the HOBBIT benchmarking platform for the generation of the final results to be included into the system publications. A specification of the hardware on which the benchmarks will be ran will be released in due course.

Prizes

The winner of the challenge will get a prize of at least 250€. Further prizes are being organized and will be announced on the challenge website.

Q&A

Contact Email: OKE-contact@googlegroups.com OR speck@informatik.uni-leipzig.de

 Important Dates

  1. Training data set and small test data (to communicate with HOBBIT) release: middle of August
  2. Evaluation: release of evaluation results in every two weeks from the end of September onwards
  3. Final system: end of January 2018
  4. Proclamation of winners: February 2018

Tasks and Training Data

The challenge will test the systems against data derived from several datasets and will comprise the following tasks:

  • Task 1 Focused Named Entity Identification and Linking
  • Task 2 Broader Named Entity Identification and Linking
  • Task 3 Focused Musical Named Entity Recognition and Linking

Small test data and training datasets for all three OKE tasks:

Task 1: http://hobbitdata.informatik.uni-leipzig.de/OKE_OC/Task1/
Task 2: http://hobbitdata.informatik.uni-leipzig.de/OKE_OC/Task2/
Τask 3: http://hobbitdata.informatik.uni-leipzig.de/OKE_OC/Task3/

The documentation for the three first tasks is the same as in the OKE2017 challenge (https://project-hobbit.eu/challenges/oke2017-challenge-eswc-2017/#tasks-and-data).

Registration and Submission

TBA

Organization

  • Axel-Cyrille Ngonga Ngomo, Institute for Applied Informatics/University of Paderborn, Germany
  • René Speck, Institute for Applied Informatics, Germany
  • Sergio Oramas, Universitat Pompeu Fabra, Spain
  • Vassiliki Rentoumi, NCSR “Demokritos”, Greece
  • Michael Röder, Institute for Applied Informatics/University of Paderborn, Germany

————————————————————————————–
Open Knowledge Extraction (OKE) Open Challenge 2017-2018
————————————————————————————–
After a successful organization of the OKE challenge by the HOBBIT project (https://project-hobbit.eu/) at the ESWC 2017, HOBBIT is proud to announce that OKE will be launched in September as an open challenge.
The OKE Open Challenge will ensure continuous participation and systems evaluation.
Stay tuned to get ready to participate!

  1. A monetary prize of at least 250€ will be provided.
  2. Please express your interest in the OKE Open Challenge by adding your email in the link: https://project-hobbit.eu/open-challenges/oke-open-challenge/express-interest. You will receive further information and notifications regarding the progress of the OKE Open Challenge.

————————————————————————————-
OKE Open Challenge at a glance:
————————————————————————————-
The aim of this challenge is to test the performance of knowledge extraction systems in aspects that are relevant for the Semantic Web. These include precision, recall and runtime. The challenge will test the systems against data derived from several datasets and will comprise the following tasks:

* Task 1 Focused Named Entity Identification and Linking
* Task 2 Broader Named Entity Identification and Linking
* Task 3 Focused Musical Named Entity Recognition and Linking

We herewith invite system developers to participate in the aforementioned tasks. To ensure that the system results are comparable, we will provide the HOBBIT benchmarking platform for the generation of the final results to be included into the system publications. A specification of the hardware on which the benchmarks will be ran will be released in due course.

Small test data and training datasets for all three OKE tasks:
Task 1: http://hobbitdata.informatik.uni-leipzig.de/OKE_OC/Task1/
Task 2: http://hobbitdata.informatik.uni-leipzig.de/OKE_OC/Task2/
Τask 3: http://hobbitdata.informatik.uni-leipzig.de/OKE_OC/Task3/
The documentation for the three first tasks is the same as in the OKE2017 challenge (https://project-hobbit.eu/challenges/oke2017-challenge-eswc-2017/#tasks-and-data).

————————————————————————————–
Prizes
————————————————————————————–
The winner of the challenge will get a prize of at least 250€. Further prizes are being organized and will be announced on the challenge website.

  1. Training data set and small test data (to communicate with HOBBIT) release: middle of August
  2. Evaluation: release of evaluation results in every two weeks from the end of September onwards
  3. Final system: end of January 2018
  4. Proclamation of winners: February 2018

————————————————————————————–
Organization
————————————————————————————–
* Axel-Cyrille Ngonga Ngomo, Institute for Applied Informatics/University of Paderborn, Germany
* René Speck, Institute for Applied Informatics, Germany
* Sergio Oramas, Universitat Pompeu Fabra, Spain
* Vassiliki Rentoumi, NCSR “Demokritos”, Greece
* Michael Röder, Institute for Applied Informatics/University of Paderborn, Germany

————————————————————————————–
Further Information and Contact
————————————————————————————–
For detailed information, including datasets and submission guidelines,
please visit the challenge website: https://project-hobbit.eu/open-challenges/oke-open-challenge/
Contact Email: OKE-contact@googlegroups.com OR speck@informatik.uni-leipzig.de