Benchmark I Generation & Acquisition
HOBBIT will provide benchmarks for the generation and acquisition of data to test the performance of the systems that implement approaches for obtaining RDF data from:
- semi-structured data streams such as sensor data (smart metering, geo-spatial information, etc.) and
- unstructured data streams (Twitter, RSS feeds, etc.).
The benchmarks will test for the scalability as well as the accuracy of systems. Currently, tasks including the recognition of entities, the extraction of relations as well as the generation of RDF are foreseen. The key performance indicators for the benchmarks will include the runtime of the approaches, their precision, recall, F-measure (both micro and macro) as well as fine-grained evaluations w.r.t. the types of resources and relations to extract.