MOCHA2017 – Mighty Storage Challenge – ESWC 2017

Challenge Motivation

Triple stores are the backbone of most applications based on Linked Data. Hence, devising systems that achieved an acceptable performance on real datasets and real loads is of central importance for the practical applicability of SemanticWeb technologies. So far, it is only partly known whether we have already passed this cap. With this challenge, we aim to

  1.  provide objective measures for how well current systems (including 3 commercial systems, which have already expressed their desire to participate) perform on real tasks of industrial relevance and
  2. detect bottlenecks of existing systems to further their development towards practical usage.

The Mighty Storage Challenge is accepted at ESWC 2017.

 Challenge Overview

The aim of the Mighty Storage Challenge is to test the performance of solutions for SPARQL processing in aspects that are relevant for modern applications. These include ingesting data, answering queries on large datasets and serving as backend for applications driven by Linked Data. The proposed challenge will test the systems against data derived from real applications and with realistic loads. An emphasis will be put on dealing with changing data in form of streams or updates. We aim to run periodically (for at least 3 years) this version of the challenge.

Challenge Result

The results of the MOCHA 2017 can be found on the HOBBIT benchmarking platform (You can login as a guest to see the results).

Q&A

For more information send an e-mail to: mightystoragechallenge-contact@googlegroups.com

 Important Dates

Paper submission deadline: Monday March 20th, 2017
Challenge paper reviews: Tuesday April 5th, 2017
Paper Notifications and invitation to task: Friday April 7th, 2017
Camera ready papers (5 pages document): Sunday April 23rd, 2017
Release of training data and instructions: Friday January 13th, 2017
Release of test dataset: Friday April 7th, 2017
Deadline for system submission: Sunday April 30th, 2017
Running of the systems: Monday May 15th, 2017
Presentation of challenge results: Thursday June 1st, 2017
Camera ready papers for the challenge proceedings (up to 15 pages): Friday June 30th, 2017 (tentative deadline)
Proclamation of winners: During ESWC2017 closing ceremony

Tasks and Training Data

This version of the challenge (which we aim to run periodically for at least 3 years) will comprise the following tasks:

  1. RDF data ingestion
  2. Data storage,
  3. Versioning and
  4. Browsing.

In essence,

  • Task 1 will measure how well systems can ingest streams of RDF data.
  • Task 2 will measure how data stores perform with different types of queries.
  • Task 3 will measure how well versioning and archiving systems for Linked Data perform when they store multiple versions of large data sets.
  • Task 4 will check existing solutions for how well they support applications that need browsing through large data sets.

Read the description of the datasets used across the tasks.**

(**Note that none of the data contains any person-related information and that we are in full compliance with the European Laws on personalized data.)

Prerequisites for participation

Each participant must:

  • provide a storage system that processes INSERT SPARQL queries
  • provide a storage solution that can process SPARQL SELECT queries
  • provide his/her solution as a docker image that abides by the technical specifications which will be released with the challenge

Registration and Submission

  • All challenge papers should be exactly five (5) pages in length in PDF file format and written in English.
  • In order to prepare their manuscripts, authors should follow Springer’s Lecture Notes in Computer Science (LNCS) style. For details and templates see Springer’s Author Instructions.
  • Paper submissions will be handled electronically via the EasyChair conference management system, available at the following address: https://easychair.org/conferences/?conf=mocha2017.
  • Papers must be submitted no later than Monday March 20th, 2017, 23:59 Hawaii Time.
    NOTE: Eligible to submit papers are only authors participating in the challenge.
  • Each submission will be peer-reviewed by members of the challenge program committee.  Papers will be evaluated according to their significance, originality, technical content, style, clarity, and relevance to the challenge.
  • Proceedings will be published by Springer in LNCS volume.
  • After the conference, challenge participants will be able to provide a detailed description of their system and evaluation results in a longer version of their paper (up to 15 pages). This longer paper will be included in the challenge proceedings.

Program & Accepted Papers

Challenge Session
Tuesday, May 30th, 2017
14:30 – 15:00
14:30 – 14:40 MOCHA Challenge Overview
Michael Röder
14:40 – 14:50 Alexander Potocki, Daniel Hladky and Martin Voigt, Challenge Accepted: QUAD Meets MOCHA2017
14:50 – 15:00 Mirko Spasić and Milos Jovanovik, MOCHA 2017 as a Challenge for Virtuoso
Posters and Demos Session
June 1st, 2017
9:00 – 11:00
The QUAD system will be presented as poster
Closing Ceremony
Thursday, June 1st, 2017
17:30
 Announcement of challenge winners during the ESWC closing ceremony

For possible, last minute changes to the program please also check the ESWC 2017 program (http://2017.eswc-conferences.org/program)

Organization

The organization responsibility will be shared by the following organizers who will comprise the Organizing Committee.

  • Axel-Cyrille Ngonga Ngomo, Institute for Applied Informatics, Germany
    Expertise: Knowledge extraction, Machine Learning, Question Answering, Information Retrieval
    Email: ngonga@informatik.uni-leipzig.de
  • Irini Fundulaki, Foundation for Research and Technology – Hellas (FORTH), Greece
    Expertise: Linked Data, Knowledge extraction, Machine Learning, Question Answering, Information Retrieval
    Email: fundul@ics.forth.gr
  • Mirko Spasic, OpenLink, UK
    Expertise: Linked Data, Knowledge extraction, Machine Learning, Question Answering, Information Retrieval
    Email: mspasic@openlinksw.com
  • Henning Petzkam, Fraunhofer IAIS, Germany
    Expertise: Linked Data, Knowledge extraction, Machine Learning, Question Answering, Information Retrieval
    Email: Henning.Petzka@iais.fraunhofer.de
  • Vassiliki Rentoumi, National Center for Scientific Research/Demokritos
    Expertise: Information Extraction, Machine Learning
    Email: vrentoumi@iit.demokritos.gr

The following set of people comprises the list of programme committee members. All the members of the list are experts from research and industry, who will review the paper submissions, independently of the organization team.

  • Saleem Muhammad, AKSW, Leipzig, Germany
  • Manolis Terrovitis, Institute for the Management of Information Systems, RC Athena, Greece
  • Manolis Koubarakis, National and Kapodistrian University of Athens, Greece
  • Olaf Hartig, Hasso Plattner Institute, Sweden
  • Kostas Stefanidis,University of Tampere, Finland
  • Milos Jovanovik, OpenLink Software, United Kingdom
  • Christoph Lange, University of Bonn, Germany
  • Ruben Taelman, Ghent University – iMinds, Belgium
  • Jürgen Umbrich,Vienna University of Economics and Business,
  • Debattista Jeremy, Fraunhofer-IAIS, Germany
  • Axel Polleres, Vienna University of Economics and Business, Austria
  • Michael Martin, University of Leipzig, Germany
  • Maria Ester Vidal, Universidad Simón Bolívar, Venezuela & Fraunhofer IAIS, Germany
  • Ruben Verborgh, Ghent University – iMinds, Belgium
  • Erik Mannens, Ghent University – iMinds, Belgium
  • Stasinos Konstantopoulos, NCSR Demokritos, Greece
  • Anastasia Dimou, Ghent University, Belgium
  • Sören Auer, Fraunhofer-IAIS, Germany
  • Kleanthi Georgala, University of Leipzig, Germany