MOCHA2018 – Mighty Storage Challenge – ESWC 2018

Challenge Motivation

Triple stores are the backbone of most applications based on Linked Data. Hence, devising systems that achieved an acceptable performance on real datasets and real loads is of central importance for the practical applicability of SemanticWeb technologies. So far, it is only partly known whether we have already passed this cap. With this challenge, we aim to

  1. provide objective measures for how well current systems (including 3 commercial systems, which have already expressed their desire to participate) perform on real tasks of industrial relevance and
  2. detect bottlenecks of existing systems to further their development towards practical usage.

MOCHA2018 takes place in conjunction with the 15th European Semantic Web Conference (ESWC 2018), 3rd-7th June 2018, Heraklion, Crete, Greece.

Stay tuned to get ready to participate!       Express your interest!

Challenge Overview

The aim of the Mighty Storage Challenge is to test the performance of solutions for SPARQL processing in aspects that are relevant for modern applications. These include ingesting data, answering queries on large datasets and serving as backend for applications driven by Linked Data. The proposed challenge will test the systems against data derived from real applications and with realistic loads. An emphasis will be put on dealing with changing data in form of streams or updates.

Challenge Result

The results of the MOCHA 2018 will be available on the HOBBIT benchmarking platform (You can login as a guest to see the results).

Q&A

For more information send an e-mail to: mightystoragechallenge-contact@googlegroups.com,   georgala@informatik.uni-leipzig.de

Important Dates

Paper submission deadline: Friday, March 30th, 2018  (extended) (*)
Challenge paper reviews: Thursday, April 26th, 2018  (extended)
Paper Notifications and invitation to task: Monday, April 30th, 2018  (extended)
Camera ready papers (5 pages document): Monday, May 14th, 2018  (extended)
Camera ready papers for the challenge proceedings (up to 15 pages): Friday, July 6th, 2018 (tentative deadline)
Release of training data and instructions: Monday, February 12, 2018
Release of test dataset: Monday, May 7th, 2018  (extended)
Deadline for system submission: TBA
Running of the systems: TBA
Presentation of challenge results: During ESWC2018
Proclamation of winners: During ESWC2018 closing ceremony

*Eligible to submit papers are only authors participating in the challenge with a system

Tasks

This version of the challenge (which we aim to run periodically for at least 3 years) will comprise the following tasks:

  • Task 1 : The RDF data ingestion  task will measure how well systems can ingest streams of RDF data.
  • Task 2:  The Data storage task will measure how data stores perform with different types of queries.
  • Task 3: The Versioning task will measure how well versioning and archiving systems for Linked Data perform when they store multiple versions of large data sets.
  • Task 4: The Browsing task will check existing solutions for how well they support applications that need browsing through large data sets.

Training data

Training data and documentation are available now for all the MOCHA tasks:

Task 1: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task1/
Task 2: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task2/
Task 3: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task3/
Task 4: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task4/

Read the description of the datasets used across the tasks.**

(**Note that none of the data contains any person-related information and that we are in full compliance with the European Laws on personalized data.)

Read also the description the Common API and the Implementation Guidelines.

 

Prerequisites for participation

Each participant must:

  • provide a storage system that processes INSERT SPARQL queries
  • provide a storage solution that can process SPARQL SELECT queries
  • provide his/her solution as a docker image that abides by the technical specifications which will be released with the challenge
  • Must follow the Common API and the Implementation Guidelines

Registration and Submission

  • All challenge papers should be exactly five (5) pages in length in PDF file format and written in English.
  • In order to prepare their manuscripts, authors should follow Springer’s Lecture Notes in Computer Science (LNCS) style. For details and templates see Springer’s Author Instructions.
  • Paper submissions will be handled electronically via the EasyChair conference management system, available at the following address: https://easychair.org/conferences/?conf=mocha2018 .
  • Papers must be submitted no later than Friday, March 9th, 2018 , 23:59 Hawaii Time.
  • NOTE: Eligible to submit papers are only authors participating in the challenge.
  • Each submission will be peer-reviewed by members of the challenge program committee.  Papers will be evaluated according to their significance, originality, technical content, style, clarity, and relevance to the challenge.
  • Proceedings will be published by Springer in LNCS volume.
  • After the conference, challenge participants will be able to provide a detailed description of their system and evaluation results in a longer version of their paper (up to 15 pages). This longer paper will be included in the challenge proceedings.

Program & Accepted Papers

TBA

Organization

The organization responsibility will be shared by the following organizers who will comprise the Organizing Committee.

  • Axel-Cyrille Ngonga Ngomo, Department of  Computer Science,  Paderborn, Germany
  • Irini Fundulaki, Foundation for Research and Technology – Hellas (FORTH), Greece
  • Mirko Spasic, OpenLink, UK
  • Vassiliki Rentoumi, National Center for Scientific Research, “Demokritos”, Greece

The following set of people comprises the list of programme committee members. All the members of the list are experts from research and industry, who will review the paper submissions, independently of the organization team.

  • Saleem Muhammad, AKSW, Leipzig
  • Manolis Terrovitis, Institute for the Management of Information Systems, RC Athena
  • Manolis Koubarakis, National and Kapodistrian University of Athens, Greece
  • Kostas Stefanidis, University of Tampere, Finland
  • Milos Jovanovik, OpenLink Software, United Kingdom
  • Christoph Lange, University of Bonn, Germany
  • Ruben Taelman, Ghent University – iMinds, Belgium
  • Jürgen Umbrich, Vienna University of Economics and Business, Vienna
  • Axel Polleres, Vienna University of Economics and Business, Austria
  • Maria-Esther Vidal, Universidad Simón Bolívar, Venezuela & Fraunhofer IAIS, Germany
  • Erik Mannens, Ghent University – iMinds, Belgium
  • Stasinos Konstantopoulos, NCSR Demokritos, Greece
  • Anastasia Dimou, Ghent University, Belgium
  • Sören Auer, Fraunhofer-IAIS, Germany
  • Michael Galkin, University of Bonn, Germany

 

———————————————————————————————
CALL FOR PAPERS
———————————————————————————————

MOCHA 2018 – Mighty Storage Challenge

in conjunction with the 15th European Semantic Web Conference (ESWC 2018)
3rd-7th June 2018, Heraklion, Crete, Greece

———————————————————————————————
URL: https://2018.eswc-conferences.org/,
URL: https://project-hobbit.eu/challenges/mighty-storage-challenge2018/

email: mightystoragechallenge-contact@googlegroups.com
email: georgala@informatik.uni-leipzig.de
———————————————————————————————

The aim of this challenge is to test the performance of solutions for SPARQL processing
in aspects that are relevant for modern applications. These include ingesting data,
answering queries on large datasets, versioning data, browsing and serving as backend for applications using Linked Data. The MOCHA challenge will test the systems against data derived from real
applications and with realistic loads.

This year, the challenge comprises the following tasks:
-Task 1 (Data Ingestion) will measure how well systems can ingest streams of RDF data.
-Task 2 (Data Storage) will measure how well data stores perform for different types of queries.
-Task 3 (Versioning) will measure how well versioning and archiving systems for Linked Data perform when they manage multiple versions of large data sets.
-Task 4 (Browsing) will check if existing solutions perform adequately for applications that address browsing through large data sets.

Training data and documentation are available now for all the MOCHA tasks:
Task 1: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task1/
Task 2: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task2/
Task 3: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task3/
Task 4: http://hobbitdata.informatik.uni-leipzig.de/MOCHA_ESWC2018/Task4/

Participants will be expected to describe their solution and results on the training datasets over a 5 page paper. In particular, a short summary of the approach chosen, a link to the experimental results and an analysis of the strengths and weaknesses of the approach are expected.

———————————————————————————————
Important Dates
———————————————————————————————
– Paper submission deadline (5 pages): Friday, March 30th, 2018(*)
– Challenge paper reviews: Thursday, April 26th, 2018
– Notification of authors and invitation to challenge: Monday, April 30th, 2018
– Camera ready papers (5 pages): Monday, May 14th, 2018
– Camera ready papers for the challenge post-proceedings (up to 15 pages): Friday, July 6th, 2018 (tentative deadline)

– Release of training data and detailed instructions: Monday, February 12, 2018
– Release of test dataset: Monday, May 7th, 2018
– Deadline for system submission: TBA
– Running of the systems: TBA

– Presentation of challenge results: During ESWC 2018
– Proclamation of winners: During ESWC 2018 closing ceremony

[ * Eligible to submit papers are only authors participating in the challenge.]

———————————————————————————————
Organization
———————————————————————————————
* Axel-Cyrille Ngonga Ngomo, Department of Computer Science, Paderborn, Germany
* Irini Fundulaki, Foundation for Research and Technology – Hellas (FORTH), Greece
* Mirko Spasic, OpenLink, UK
* Vassiliki Rentoumi, National Center for Scientific Research, “Demokritos”, Greece

For the complete list of organizers and program committee members,
visit the challenge website.

———————————————————————————————
Further Information and Contact
———————————————————————————————
For detailed information, including datasets and submission guidelines,
please visit the challenge website: https://project-hobbit.eu/challenges/mighty-storage-challenge/

Contact Email: mightystoragechallenge-contact@googlegroups.com
or email: georgala@informatik.uni-leipzig.de

TBA