HOBBIT opted for a survey method to analyze benchmarking needs from the Big Linked Data community. During the first year of the project we developed and distributed a survey to gather the requirements for the HOBBIT platform and benchmarks. The aim of the fully anonymized survey was for the HOBBIT consortium to determine how a participant’s’ company or organization would benefit from the HOBBIT platform. The analysis results include an overview of benchmarking interests per different participant profiles such as solutions providers, technology users and scientific community. For each profile the results provide an overview of the:
- Use of benchmarking solutions categorized according to the Linked Data Value Chain, such as Storage/Querying (Storage and Curation), Interlinking (Analytics and Processing), Classification/Enrichment (Analytics and Processing), Discovery (Visualization and Services), Extraction (Generation and Acquisition), among others.
- Interests in Key Performance Indicators such as Accuracy, Correctness, Runtime, etc.
- Nature of the data used in the benchmark, e.g. synthetic, real, mixed
- Data volume
During the second year of the HOBBIT project, we contacted the Big Data Value Association (BDVA) with the aim of elaborating the creation of the HOBBIT association under its umbrella. After several rounds of negotiations, a new subgroup on Big Data Benchmarking was created under the BDVA task force (TF6-Technical). In general, the main objective of the task force is to identify applications in which big data technologies can create the largest impact in Europe. In the context of this task force the newly created subgroup will merge the efforts of several data benchmarking projects. While HOBBIT will take the lead of the activity regarding Linked Data benchmarking, within this subgroup we will aim to expand the scope of HOBBIT benchmarking platform into a more generic context of Big Data Benchmarking. The subgroup description, objectives and activity plan were officially presented during the BDVA AG workshops in Brussels on March 16, 2018. During this meeting we also presented the second version of the HOBBIT survey – a joint survey in collaboration with the DataBench project, which is available online under Big Data Benchmarking Survey. This second survey is an extended version of the original survey targeting a broader context of big data benchmarks.
We invite data scientists, analysts, application developers, system architects, researchers from academia or industry, technology vendors and users to take part in the survey. Your participation will help understanding big data benchmarking needs/interests across diverse areas in Europe. By taking part, your voice is heard in important EU projects as well as in BDVA effort for standardization of the field.