Grand Challenge – DEBS 2017

Call for Grand Challenge Solutions

The 2017 ACM DEBS Grand Challenge is the seventh in a series of challenges which seek to provide a common ground and uniform evaluation criteria for a competition aimed at both research and industrial event-based systems. The goal of the 2017 DEBS Grand Challenge competition is to evaluate event-based systems for real-time analytics over high velocity and high volume data streams.

The focus of the 2017 Grand Challenge is on the analysis of the RDF streaming data generated by digital and analogue sensors embedded within manufacturing equipment. The goal of the 2017 Grand Challenge is to implement detection of anomalies in the behaviour of such manufacturing equipment.

This year’s Grand Challenge is co-organized by the HOBBIT project represented by AGT International. Both the data set and the automated evaluation platform are provided by the HOBBIT project. This will allow us to offer the possibility of running a distributed solution on multiple VMs.

Details about the data, queries for the Grand Challenge and the evaluation process are provide here.

Prize

Participants in the 2017 DEBS Grand Challenge will have the chance to win two prizes. The first prize is the “Grand Challenge Award” for the best performing, correct submission. The winner (or winning team) of the “Grand Challenge Award” will be additionally awarded with a monetary reward of $1000.

The second prize is the “Grand Challenge Audience Award” – it is determined based on the audience feedback provided after the presentation of the Grand Challenge solutions during the DEBS 2017 conference. The “Grand Challenge Audience Award”, as opposed to the overall “Grand Challenge Award”, does not entail any additional monetary reward as it is based purely on the audience perception of the presentation of a given solution.

Q&A

For additional questions please refer to the Grand Challenge mailing list: http://debs.org/listinfo/gc or send e-mail to: gc@debs.org.

Important Dates

April 14th, 2017 GC solutions due (submission system closes)
March 17th, 2017 Evaluation platform supports single node and distributed performance tests
March 14th, 2017 Hello World example for HOBBIT evaluation platform
February 17th, 2017 Evaluation platform supports correctness tests
December 17th, 2016 Evaluation platform online (team registration open)
December 1st, 2016 Problem description (incl. sample data) online

Challenge Description (Tasks and Training Data)

The 2017 DEBS Grand Challenge focuses on two scenarios that relate to the problem of automatic detection of anomalies for manufacturing equipment. The overall goal of both scenarios is to detect abnormal behaviour of a manufacturing machine based on the observation of the stream of measurements provided by such a machine. The data produced by each sensor is clustered and the state transitions between the observed clusters are modelled as a Markov chain. Based on this classification, anomalies are detected as sequences of transitions that happen with a probability lower than a given threshold.

The difference between the first and the second scenario is that in the first scenario the number of machines to observe is fixed, while in the second scenarios new machines dynamically join and leave the set of observable machines. We provide two sample input data sets under the following FTP address: ftp://hobbitdata.informatik.uni-leipzig.de/DEBS_GC/. A description of the input and output data (together with their format) and the query that will evaluate the behavior of the machines under observation as well as the description of the parameters and the expected output format, can be found here.

DEBS parrot (Hello World) benchmark is ready. It sends certain amount of text messages and expects benchmarked system to send them back in the same order. We also provide an implementation of a system that can pass the benchmark. Source code, metadata, docker file can be found on GitHub: https://github.com/romromov/debs-parrotbenchmark-system.

Support

The organisers of the challenge explicitly thank Weidmüller (http://www.weidmueller.de/) for the provisioning of the original data set that AGT International used to generate a realistic data set for the purpose of this challenge.

Registration and Submission

  1. Submission and registration procedure is documented here: https://github.com/hobbit-project/platform/wiki
  2. The evaluation platform can be reached under following address: http://master.project-hobbit.eu

Organizers

  • Vincenzo Gulisano – Chalmers University of Technology
  • Roman Katerinenko – AGT International
  • Zbigniew Jerzak – SAP SE
  • Martin Strohbach – AGT International
  • Holger Ziekow – Hochschule Furtwangen