Open Mighty Storage (MOCHA) Challenge 2017-2018

Challenge Motivation

After the successful organization of the MOCHA challenge at the ESWC 2017, the HOBBIT project is proud to announce that MOCHA will be launched in September as an open challenge. The MOCHA Open Challenge will run periodically with cutoff dates in February and September. Therewith, the challenge will ensure continuous participation and system evaluation.
Monetary prizes will be provided to the best winning systems at each cutoff date.
Stay tuned to get ready to participate!        

Challenge Overview

Triple stores are the backbone of most applications based on Linked Data. Hence, devising systems that achieved an acceptable performance on real datasets and real loads is of central importance for the practical applicability of SemanticWeb technologies. So far, it is only partly known whether we have already passed this cap. With this challenge, we aim to

  1.  provide objective measures for how well current systems (including 3 commercial systems, which have already expressed their desire to participate) perform on real tasks of industrial relevance and
  2. detect bottlenecks of existing systems to further their development towards practical usage.

The aim of the Mighty Storage Challenge is to test the performance of solutions for SPARQL processing in aspects that are relevant for modern applications. These include ingesting data, answering queries on large datasets and serving as backend for applications driven by Linked Data. The proposed challenge will test the systems against data derived from real applications and with realistic loads. An emphasis will be put on dealing with changing data in form of streams or updates. We aim to run periodically (for at least 3 years) this version of the challenge.

Prizes

The winner of the challenge will get a prize of 250€

Q&A

Contact Email: mightystoragechallenge-contact@googlegroups.com OR georgala@informatik.uni-leipzig.de

 Important Dates

Dry-Run: September 18th, 2017 –  October 29th, 2017
Training Phase: October 30th, 2017* – January 31st, 2018
Test Phase: February 1st, 2018 – February 28th, 2018
Proclamation of results: February 28th, 2018

*Weekly release of evaluation results from the end of October onwards

Tasks and Training Data

The aim of this challenge is to test the performance of solutions for SPARQL processing in aspects that are relevant for modern applications. These include ingesting data, answering queries on large datasets, versioning data, browsing and serving as backend for applications using Linked Data. The MOCHA Open Challenge will test the systems against data derived from real applications and with realistic loads. The open challenge comprises the following tasks:

  • Task 1 (Data Ingestion) will measure how well systems can ingest streams of RDF data.
  • Task 2 (Data Storage) will measure how well data stores perform for different types of queries.
  • Task 3 (Versioning) will measure how well versioning and archiving systems for Linked Data manage evolving datasets and queries evaluated across the multiple versions of said datasets.
  • Task 4 (Browsing) will check if existing solutions perform adequately for applications that address  browsing through large data sets.

Read also the description the Common API and the Implementation Guidelines.

Small test data and documentation are available now for all the MOCHA tasks:
Task 1: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_OC/Task1/
Task 2: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_OC/Task2/
Task 3: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_OC/Task3/
Task 4: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_OC/Task4/

Registration and Submission

TBA

Organization

  • Axel-Cyrille Ngonga Ngomo, Institute for Applied Informatics, Germany
  • Irini Fundulaki, Foundation for Research and Technology – Hellas (FORTH), Greece
  • Mirko Spasic, OpenLink, UK
  • Henning Petzka, Fraunhofer IAIS, Germany
  • Vassiliki Rentoumi, NCSR “Demokritos”, Greece

—————————————————————
Open Mighty Storage Challenge 2017-2018
—————————————————————
After the successful organization of the MOCHA challenge at the ESWC 2017, the HOBBIT project (https://project-hobbit.eu/) is proud to announce that MOCHA will be launched in September as an open challenge. The MOCHA Open Challenge will run periodically with cutoff dates in February and September. Therewith, the challenge will ensure continuous participation and system evaluation. Stay tuned and get ready to participate!

  1. Monetary prizes will be provided to the best winning systems at each cutoff date.
  2. Please express your interest in the MOCHA Open Challenge by adding your email in the link: https://project-hobbit.eu/open-challenges/mocha-open-challenge/express-interest. You will receive further information and notifications regarding the progress of the MOCHA Open Challenge.

—————————————————————-
Important Dates
—————————————————————
Dry-Run                                    September 18th, 2017 – October 29th, 2017
Training Phase                        October 30th, 2017* – January 31st, 2018
Test Phase                                February 1st, 2018 – February 28th, 2018
Proclamation of results         February 28th, 2018

*Weekly release of evaluation results from the end of October onwards

—————————————————————
Prizes
—————————————————————
The winner of the challenge will get a prize of 250€

—————————————————————
MOCHA Open Challenge at a glance
—————————————————————
The aim of this challenge is to test the performance of solutions for SPARQL processing in aspects that are relevant for modern applications. These include ingesting data, answering queries on large datasets, versioning data, browsing and serving as backend for applications using Linked Data. The MOCHA Open Challenge will test the systems against data derived from real applications and with realistic loads. The open challenge comprises the following tasks:

  • Task 1 (Data Ingestion) will measure how well systems can ingest streams of RDF data.
  • Task 2 (Data Storage) will measure how well data stores perform for different types of queries.
  • Task 3 (Versioning) will measure how well versioning and archiving systems for Linked Data manage evolving datasets and queries evaluated across the multiple versions of said datasets.
  • Task 4 (Browsing) will check if existing solutions perform adequately for applications that address browsing through large data sets.

Small test data and documentation are available now for all the MOCHA tasks:
Task 1: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_OC/Task1/
Task 2: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_OC/Task2/
Task 3: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_OC/Task3/
Task 4: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_OC/Task4/

—————————————————————
Organization
—————————————————————
* Axel-Cyrille Ngonga Ngomo, Institute for Applied Informatics, Germany
* Irini Fundulaki, Foundation for Research and Technology – Hellas (FORTH), Greece
* Mirko Spasic, OpenLink, UK
* Henning Petzka, Fraunhofer IAIS, Germany
* Vassiliki Rentoumi, NCSR “Demokritos”, Greece

—————————————————————
Further Information and Contact
—————————————————————
For detailed information, including datasets and submission guidelines,
please visit the challenge website: https://project-hobbit.eu/challenges/mocha-open-challenge
Contact Email: mightystoragechallenge-contact@googlegroups.com OR georgala@informatik.uni-leipzig.de