The interest in benchmarking for Linked Data is growing tangibly and HOBBIT is one of the carriers of movement. This is one of the insights we could gather at ISWC 2016, which was held in Kobe, Japan between October 17 to October 21. HOBBIT had a very strong presence at the conference where members of the consortium presented two tutorials (SPARQL Querying Benchmarks and Link Discovery) and organized one workshop (Benchmarking Linked Open Data – BLINK). With the upcoming benchmarking platform, we will release a concrete mean to support reliable, reproducible benchmarking for the Linked Data Lifecycle.
Axel Ngonga along with colleagues Muhammad Saleem, Ricardo Usbeck and Michael Röder from InfAI talked about benchmarks for SPARQL querying engines in their SPARQL Querying Benchmarks tutorial that was presented in the first day of the conference. In this tutorial the authors discussed different benchmarks used for SPARQL query processing over Linked Data and focused on the design, key features, evaluation setup, and pros and cons of the benchmarks used to evaluate the performance of triple stores and federated SPARQL query engines. The authors also discussed state-of-the-art triple stores and federated SPARQL benchmarks that were demonstrated with examples along with hands-on experience and exercises that were carried out by the participants. The slides are accessible here.
The tutorial Link Discovery – Algorithms, Approaches and Benchmarks was prepared by Axel Ngonga and Muhammad Sherif from InfAI as well as Irini Fundulaki from FORTH. The tutorial gave an overview of existing solutions for link discovery. It looked into some of the state-of-art algorithms for the rapid execution of link discovery tasks focusing on algorithms which guarantee result completeness. Algorithms for the detection of complete and correct sets of links with a focus on supervised, active and unsupervised machine learning algorithms were also presented. Last, existing instance matching benchmarks for Linked Data were presented to the audience. The tutorial was concluded with a hands-on experience with one of the most commonly used link discovery frameworks, specifically LIMES. The slides are accessible here: Introduction, Part I, Part II, Part III, Part IV.
Both tutorials were very well attended, with more than 50 participants overall who showed great interest in the algorithms and tools discussed by the presenters of the tutorials.
In addition to the tutorials, Axel Ngonga (InfAI) and Irini Fundulaki (FORTH) were present at the 1st International Workshop on Benchmarking Linked Data (BLINK). Axel opened the workshop by providing a short overview of HOBBIT. Dr. Stasinos Konstantopoulos gave the keynote talk entitled “BigDataEurope: Platform Concept and Architecture and its Application to Data-Intensive Science”. Irini Fundulaki (FORTH) presented two papers on topics addressed at the HOBBIT project: How well does your instance matching system perform? Experimental evaluation with LANCE, where she presented experimental results with the instance matching benchmark generator LANCE that is developed in the context of HOBBIT. She also discussed a short survey of versioning systems and benchmarks for Linked Data in her presentation Versioning for Linked Data: Archiving Systems and Benchmarks. Finally Mirko Spasić from OpenLink presented their work on an RDF Dataset generator that produces Linked Data that bear similar characteristics with real datasets. The very constructive discussion at the end of the tutorial corroborated the results gathered from the HOBBIT survey.
The workshop was very well attended with more than 40 attendees who posed interesting questions to the presenters of the papers.
Join our community to remain informed at http://project-hobbit.eu/get-involved/. If you have any questions, do not hesitate to contact us (see contact details at http://project-hobbit.eu/contacts/). We are looking forward to benchmarking your system soon.