MOCHA2018 – Mighty Storage Challenge – ESWC 2018

Challenge Motivation

Triple stores are the backbone of most applications based on Linked Data. Hence, devising systems that achieved an acceptable performance on real datasets and real loads is of central importance for the practical applicability of SemanticWeb technologies. So far, it is only partly known whether we have already passed this cap. With this challenge, we aim to

  1. provide objective measures for how well current systems (including 3 commercial systems, which have already expressed their desire to participate) perform on real tasks of industrial relevance and
  2. detect bottlenecks of existing systems to further their development towards practical usage.

MOCHA2018 takes place in conjunction with the 15th European Semantic Web Conference (ESWC 2018), 3rd-7th June 2018, Heraklion, Crete, Greece.

Stay tuned to get ready to participate!       Express your interest!

Challenge Overview

The aim of the Mighty Storage Challenge is to test the performance of solutions for SPARQL processing in aspects that are relevant for modern applications. These include ingesting data, answering queries on large datasets and serving as backend for applications driven by Linked Data. The proposed challenge will test the systems against data derived from real applications and with realistic loads. An emphasis will be put on dealing with changing data in form of streams or updates.

Challenge Result

The results of the MOCHA 2018 will be available on the HOBBIT benchmarking platform (You can login as a guest to see the results).

Sponsorship

The winner of the MOCHA challenge will receive a prize of 250 Euros.
Prize money is provided by USU.

Q&A

For more information send an e-mail to: mightystoragechallenge-contact@googlegroups.com,   georgala@informatik.uni-leipzig.de

Important Dates

Paper submission deadline: Wednesday, April 4th, 2018  (extended)
Challenge paper reviews: Thursday, April 26th, 2018  (extended)
Paper Notifications and invitation to task: Monday, April 30th, 2018  (extended)
Camera ready papers (5 pages document): Monday, May 14th, 2018  (extended)
Camera ready papers for the challenge proceedings (up to 15 pages): Friday, July 6th, 2018 (tentative deadline)
Release of training data and instructions: Monday, February 12, 2018
Release of test dataset: Wednesday, April 25th, 2018  
Deadline for system submission: Sunday, May 13th, 2018
Running of the systems: Tuesday, May 15th, 2018
Presentation of challenge results: During ESWC2018
Proclamation of winners: During ESWC2018 closing ceremony

Tasks

This version of the challenge (which we aim to run periodically for at least 3 years) will comprise the following tasks:

  • Task 1 : The RDF data ingestion  task will measure how well systems can ingest streams of RDF data.
  • Task 2:  The Data storage task will measure how data stores perform with different types of queries.
  • Task 3: The Versioning task will measure how well versioning and archiving systems for Linked Data perform when they store multiple versions of large data sets.
  • Task 4: The Browsing task will check existing solutions for how well they support applications that need browsing through large data sets.

Training data

Training data and documentation are available now for all the MOCHA tasks:

Task 1: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task1/
Task 2: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task2/
Task 3: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task3/
Task 4: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task4/

Read the description of the datasets used across the tasks.**

(**Note that none of the data contains any person-related information and that we are in full compliance with the European Laws on personalized data.)

Read also the description the Common API and the Implementation Guidelines.

 

Prerequisites for participation

Each participant must:

  • provide a storage system that processes INSERT SPARQL queries
  • provide a storage solution that can process SPARQL SELECT queries
  • provide his/her solution as a docker image that abides by the technical specifications which will be released with the challenge
  • Must follow the Common API and the Implementation Guidelines

Registration and Submission

  • All challenge papers should be exactly five (5) pages in length in PDF file format and written in English.
  • In order to prepare their manuscripts, authors should follow Springer’s Lecture Notes in Computer Science (LNCS) style. For details and templates see Springer’s Author Instructions.
  • Paper submissions will be handled electronically via the EasyChair conference management system, available at the following address: https://easychair.org/conferences/?conf=mocha2018 .
  • Papers must be submitted no later than Friday, March 30th, 2018  (extended), 23:59 Hawaii Time.
  • Each submission will be peer-reviewed by members of the challenge program committee.  Papers will be evaluated according to their significance, originality, technical content, style, clarity, and relevance to the challenge.
  • Proceedings will be published by Springer in LNCS volume.
  • After the conference, challenge participants will be able to provide a detailed description of their system and evaluation results in a longer version of their paper (up to 15 pages). This longer paper will be included in the challenge proceedings.

Program & Accepted Papers

Challenge Session
Tuesday, June 5th, 2018
11:45 – 12:15
11:45 – 11:55 MOCHA Challenge Overview
Kleanthi Georgala
11:55 – 12:05 Ruben Taelman, Miel Vander Sande and Ruben Verborgh, Versioned Querying with OSTRICH and Comunica in MOCHA 2018
12:05 – 12:15 Milos Jovanovik and Mirko Spasić, Benchmarking Virtuoso 8 at the Mighty Storage Challenge 2018: Training Results
Closing Ceremony
Thursday, June 7th, 2019
17:30
 Announcement of challenge winners during the ESWC closing ceremony

For possible, last minute changes to the program please also check the ESWC 2018 program (https://2018.eswc-conferences.org/program/)

Organization

The organization responsibility will be shared by the following organizers who will comprise the Organizing Committee.

  • Axel-Cyrille Ngonga Ngomo, Department of  Computer Science,  Paderborn, Germany
  • Irini Fundulaki, Foundation for Research and Technology – Hellas (FORTH), Greece
  • Mirko Spasic, OpenLink, UK
  • Vassiliki Rentoumi, National Center for Scientific Research, “Demokritos”, Greece
  • Kleanthi Georgala, Universität Leipzig, AKSW, Institut für Angewandte Informatik (InfAI)

The following set of people comprises the list of programme committee members. All the members of the list are experts from research and industry, who will review the paper submissions, independently of the organization team.

  • Saleem Muhammad, AKSW, Leipzig
  • Manolis Terrovitis, Institute for the Management of Information Systems, RC Athena
  • Manolis Koubarakis, National and Kapodistrian University of Athens, Greece
  • Kostas Stefanidis, University of Tampere, Finland
  • Milos Jovanovik, OpenLink Software, United Kingdom
  • Christoph Lange, University of Bonn, Germany
  • Ruben Taelman, Ghent University – iMinds, Belgium
  • Jürgen Umbrich, Vienna University of Economics and Business, Vienna
  • Axel Polleres, Vienna University of Economics and Business, Austria
  • Maria-Esther Vidal, Universidad Simón Bolívar, Venezuela & Fraunhofer IAIS, Germany
  • Erik Mannens, Ghent University – iMinds, Belgium
  • Stasinos Konstantopoulos, NCSR Demokritos, Greece
  • Anastasia Dimou, Ghent University, Belgium
  • Sören Auer, Fraunhofer-IAIS, Germany
  • Michael Galkin, University of Bonn, Germany

 

———————————————————————————————
CALL FOR PAPERS
———————————————————————————————

MOCHA 2018 – Mighty Storage Challenge

in conjunction with the 15th European Semantic Web Conference (ESWC 2018)
3rd-7th June 2018, Heraklion, Crete, Greece

———————————————————————————————
URL: https://2018.eswc-conferences.org/,
URL: https://project-hobbit.eu/challenges/mighty-storage-challenge2018/

email: mightystoragechallenge-contact@googlegroups.com
email: georgala@informatik.uni-leipzig.de
———————————————————————————————

The aim of this challenge is to test the performance of solutions for SPARQL processing
in aspects that are relevant for modern applications. These include ingesting data,
answering queries on large datasets, versioning data, browsing and serving as backend for applications using Linked Data. The MOCHA challenge will test the systems against data derived from real
applications and with realistic loads.

This year, the challenge comprises the following tasks:
-Task 1 (Data Ingestion) will measure how well systems can ingest streams of RDF data.
-Task 2 (Data Storage) will measure how well data stores perform for different types of queries.
-Task 3 (Versioning) will measure how well versioning and archiving systems for Linked Data perform when they manage multiple versions of large data sets.
-Task 4 (Browsing) will check if existing solutions perform adequately for applications that address browsing through large data sets.

Training data and documentation are available now for all the MOCHA tasks:
Task 1: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task1/
Task 2: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task2/
Task 3: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task3/
Task 4: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task4/

Participants will be expected to describe their solution and results on the training datasets over a 5 page paper. In particular, a short summary of the approach chosen, a link to the experimental results and an analysis of the strengths and weaknesses of the approach are expected.

———————————————————————————————
Important Dates
———————————————————————————————
– Paper submission deadline (5 pages): Wednesday, April 4th, 2018 (extended)
– Challenge paper reviews: Thursday, April 26th, 2018  (extended)
– Notification of authors and invitation to challenge: Monday, April 30th, 2018  (extended)
– Camera ready papers (5 pages): Monday, May 14th, 2018  (extended)
– Camera ready papers for the challenge post-proceedings (up to 15 pages): Friday, July 6th, 2018 (tentative deadline)

– Release of training data and detailed instructions: Monday, February 12, 2018
– Release of test dataset: Wednesday, April 25th, 2018
– Deadline for system submission: Sunday, May 13th, 2018
– Running of the systems: Tuesday, May 15th, 2018

– Presentation of challenge results: During ESWC 2018
– Proclamation of winners: During ESWC 2018 closing ceremony

———————————————————————————————
Organization
———————————————————————————————
* Axel-Cyrille Ngonga Ngomo, Department of Computer Science, Paderborn, Germany
* Irini Fundulaki, Foundation for Research and Technology – Hellas (FORTH), Greece
* Mirko Spasic, OpenLink, UK
* Vassiliki Rentoumi, National Center for Scientific Research, “Demokritos”, Greece
* Kleanthi Georgala, Universität Leipzig, AKSW, Institut für Angewandte Informatik (InfAI)

For the complete list of organizers and program committee members,
visit the challenge website.

———————————————————————————————
Further Information and Contact
———————————————————————————————
For detailed information, including datasets and submission guidelines,
please visit the challenge website: https://project-hobbit.eu/challenges/mighty-storage-challenge/

Contact Email: mightystoragechallenge-contact@googlegroups.com
or email: georgala@informatik.uni-leipzig.de

TBA