Grand Challenge – DEBS 2017

Call for Grand Challenge Solutions

The 2017 ACM DEBS Grand Challenge is the seventh in a series of challenges which seek to provide a common ground and uniform evaluation criteria for a competition aimed at both research and industrial event-based systems. The goal of the 2017 DEBS Grand Challenge competition is to evaluate event-based systems for real-time analytics over high velocity and high volume data streams.

The focus of the 2017 Grand Challenge is on the analysis of the RDF streaming data generated by digital and analogue sensors embedded within manufacturing equipment. The goal of the 2017 Grand Challenge is to implement detection of anomalies in the behaviour of such manufacturing equipment.

This year’s Grand Challenge is co-organized by the HOBBIT project represented by AGT International. Both the data set and the automated evaluation platform are provided by the HOBBIT project. This will allow us to offer the possibility of running a distributed solution on multiple VMs.

Details about the data, queries for the Grand Challenge and the evaluation process are provide here.

Prize

Participants in the 2017 DEBS Grand Challenge will have the chance to win two prizes. The first prize is the “Grand Challenge Award” for the best performing, correct submission. The winner (or winning team) of the “Grand Challenge Award” will be additionally awarded with a monetary reward of $1000.

The second prize is the “Grand Challenge Audience Award” – it is determined based on the audience feedback provided after the presentation of the Grand Challenge solutions during the DEBS 2017 conference. The “Grand Challenge Audience Award”, as opposed to the overall “Grand Challenge Award”, does not entail any additional monetary reward as it is based purely on the audience perception of the presentation of a given solution.

Q&A

For additional questions please refer to the Grand Challenge mailing list: http://debs.org/listinfo/gc or send e-mail to: gc@debs.org.

Important Dates

April 14th, 2017 GC solutions due (submission system closes)
March 17th, 2017 Evaluation platform supports single node and distributed performance tests
March 14th, 2017 Hello World example for HOBBIT evaluation platform
February 17th, 2017 Evaluation platform supports correctness tests
December 17th, 2016 Evaluation platform online (team registration open)
December 1st, 2016 Problem description (incl. sample data) online

Challenge Description (Tasks and Training Data)

The 2017 DEBS Grand Challenge focuses on two scenarios that relate to the problem of automatic detection of anomalies for manufacturing equipment. The overall goal of both scenarios is to detect abnormal behaviour of a manufacturing machine based on the observation of the stream of measurements provided by such a machine. The data produced by each sensor is clustered and the state transitions between the observed clusters are modelled as a Markov chain. Based on this classification, anomalies are detected as sequences of transitions that happen with a probability lower than a given threshold.

The difference between the first and the second scenario is that in the first scenario the number of machines to observe is fixed, while in the second scenarios new machines dynamically join and leave the set of observable machines. We provide two sample input data sets under the following FTP address: ftp://hobbitdata.informatik.uni-leipzig.de/DEBS_GC/. A description of the input and output data (together with their format) and the query that will evaluate the behavior of the machines under observation as well as the description of the parameters and the expected output format, can be found here.

DEBS parrot (Hello World) benchmark is ready. It sends certain amount of text messages and expects benchmarked system to send them back in the same order. We also provide an implementation of a system that can pass the benchmark. Source code, metadata, docker file can be found on GitHub: https://github.com/romromov/debs-parrotbenchmark-system.

Support

The organisers of the challenge explicitly thank Weidmüller (http://www.weidmueller.de/) for the provisioning of the original data set that AGT International used to generate a realistic data set for the purpose of this challenge.

Registration and Submission

  1. Submission and registration procedure is documented here: https://github.com/hobbit-project/platform/wiki
  2. The evaluation platform can be reached under following address: http://master.project-hobbit.eu

Program & Accepted Papers

Grand Challenge Session
Thursday, June 22nd, 2017
15:50—17:30
Grand Challenge Papers:

  1. FlinkMan – Anomaly Detection in Manufacturing Equipment with Apache Flink. Nicolo Rivetti, Yann Busnel and Avigdor Gal.
  2. Anomaly Detection of Manufacturing Equipment via High Performance RDF Data Stream Processing. Nihla Akram, Sachini Siriwardene, Malith Jayasinghe, Miyuru Dayarathna, Isuru Perera, Seshika Fernando, Srinath Perera, Upul Bandara and Sriskandarajah Suhothayan.
  3. Optimized Stage Processing for Anomaly Detection on Numerical Data Streams. Ciprian Amariei, Paul Diac and Emanuel Onica.
  4. Real-time High Performance Anomaly Detection over Data Streams. Dimitrije Jankov, Sourav Sikdar, Rohan Mukherjee, Kia Teymourian and Chris Jermaine.
  5. StreamLearner — Distributed Incremental Machine Learning on Event Streams. Christian Mayer, Ruben Mayer and Majd Abdo.
  6. Runtime Anomaly Detection Method in Smart Factories using Machine Learning on RDF Event Streams. Joong-Hyun Choi, Kang-Woo Lee, Hyungken Jung and Eun-Sun Cho.
  7. Automatic Anomaly Detection over Ordering Sliding Windows. Tarek Zaarour, Niki Pavlopoulou, Soulieman Hasan, Umair Ulhassan and Edward Curry.

A preliminary program of DEBS 2017 in Barcelona (Monday 19th– Friday 23rd of June 2017) can be found here (http://www.debs2017.org/conference-program/). This is still a preliminary schedule, be advised that some slots may change!

Organizers

  • Vincenzo Gulisano – Chalmers University of Technology
  • Roman Katerinenko – AGT International
  • Zbigniew Jerzak – SAP SE
  • Martin Strohbach – AGT International
  • Holger Ziekow – Hochschule Furtwangen