Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Data Generator
Conclusions
References
All Topics
Computer Science
Data Science

List.MID: A MIDI-Based Benchmark for Evaluating RDF Lists

Profile image of Enrico DagaEnrico Daga

2019, Lecture Notes in Computer Science

https://doi.org/10.1007/978-3-030-30796-7_16
visibility

…

description

16 pages

link

1 file

Abstract

For guidance on citations see FAQs.

Related papers

The LITMUS Test : Benchmarking RDF and Graph Data Management Systems
Harsh Thakkar

2017

In this paper we demonstrate the working of proposed LITMUS benchmark suite. Benchmarking is an extremely tedious task demanding repetitive manual effort, therefore it is advantageous to automate the whole process. LITMUS, is an automated benchmarking framework which supports benchmarking and comparing diverse DMSs for both RDF and property graph DMS. It also provides custom visualization (i.e. plots) support for the benchmarked DMSs against a wide range of CPU and memoryspecific evaluation parameters.

View PDFchevron_right
Jazzcats: navigating an RDF triplestore of integrated performance metadata
Terhi Nurmikko-Fuller

Proceedings of the 5th International Conference on Digital Libraries for Musicology, 2018

Applying Linked Data techniques to musical metadata can facilitate new paths of musicological inquiry. JazzCats: Jazz Collection of Aggregated Triples is a prototype project interlinking four discrete jazz performance datasets and external sources as references. Tabular, relational, and graph legacy datasets have necessitated different RDF production and ingestion workflows to support scholarly study of performance traditions. This paper highlights critical processes of data curation for digital libraries, including quality assessment of the ingested datasets. In addition, we describe research questions enabled by JazzCats, raise musicological implications, and offer suggestions to overcome current limitations.

View PDFchevron_right
Assessing the Quality of RDF Mappings with EvaMap
Patricia Serrano-alvarado

The Semantic Web: ESWC 2020 Satellite Events, 2020

Linked Data (LD) is a set of best practices to publish reusable data on the web in RDF format. Despite the benefits of LD, many datasets are not published as RDF. Transforming structured datasets into RDF datasets is possible thanks to RDF Mappings. But, for the same dataset, different mappings can be proposed. We believe that a tool capable of evaluating the quality of an RDF mapping would make the creation of mappings easier. In this paper, we present EvaMap, a framework to assess the quality of RDF mappings. The demonstration shows how EvaMap can be used to evaluate and improve RDF mappings.

View PDFchevron_right
Modelling and Querying Lists in RDF. A Pragmatic Study
Enrico Daga

2019

Many Linked Data datasets model elements in their domains in the form of lists: a countable number of ordered resources. When pub- lishing these lists in RDF, an important concern is making them easy to consume. Therefore, a well-known recommendation is to find an existing list modelling solution, and reuse it. However, a specific domain model can be implemented in different ways and vocabularies may provide al- ternative solutions. In this paper, we argue that a wrong decision could have a significant impact in terms of performance and, ultimately, the availability of the data. We take the case of RDF Lists and make the hy- pothesis that the efficiency of retrieving sequential linked data depends primarily on how they are modelled (triple-store invariance hypothe- sis). To demonstrate this, we survey different solutions for modelling sequences in RDF, and propose a pragmatic approach for assessing their impact on data availability. Finally, we derive good (and bad) practices on how...

View PDFchevron_right
Automated approach for quality assessment of RDF resources
Shuxin Zhang

BMC Medical Informatics and Decision Making

Introduction The Semantic Web community provides a common Resource Description Framework (RDF) that allows representation of resources such that they can be linked. To maximize the potential of linked data - machine-actionable interlinked resources on the Web - a certain level of quality of RDF resources should be established, particularly in the biomedical domain in which concepts are complex and high-quality biomedical ontologies are in high demand. However, it is unclear which quality metrics for RDF resources exist that can be automated, which is required given the multitude of RDF resources. Therefore, we aim to determine these metrics and demonstrate an automated approach to assess such metrics of RDF resources. Methods An initial set of metrics are identified through literature, standards, and existing tooling. Of these, metrics are selected that fulfil these criteria: (1) objective; (2) automatable; and (3) foundational. Selected metrics are represented in RDF and semantical...

View PDFchevron_right
Benchmarking Commercial RDF Stores with Publications Office Dataset
Florence Amardeilh

Lecture Notes in Computer Science

This paper presents a benchmark of RDF stores with realworld datasets and queries from the EU Publications Office (PO). The study compares the performance of four commercial triple stores: Stardog 4.3 EE, GraphDB 8.0.3 EE, Oracle 12.2c and Virtuoso 7.2.4.2 with respect to the following requirements: bulk loading, scalability, stability and query execution. The datasets and the selected queries (44) are used in the Linked Data publication workflow at PO. The first results of this study provides some insights into the quantitative performance assessment of RDF stores used in production environment in general, especially when dealing with large amount of triples. Virtuoso is faster in querying and loading scenarios while GraphDB shows better results regarding stability.

View PDFchevron_right
An RDF Dataset Generator for the Social Network Benchmark with Real-World Coherence
Milos Jovanovik, Arnau Prat-Pérez, Mirko Spasić

Synthetic datasets used in benchmarking need to mimic all characteristics of real-world datasets, in order to provide realistic bench-marking results. Synthetic RDF datasets usually show a significant discrepancy in the level of structuredness compared to real-world RDF datasets. This structural difference is important as it directly affects storage , indexing and querying. In this paper, we show that the synthetic RDF dataset used in the Social Network Benchmark is characterized with high-structuredness and therefore introduce modifications to the data generator so that it produces an RDF dataset with a real-world structuredness.

View PDFchevron_right
Storage, Indexing, Query Processing, and Benchmarking in Centralized and Distributed RDF Engines: A Survey
Ch.waqas ali

2020

The recent advancements of the Semantic Web and Linked Data have changed the working of the traditional web. There is a huge adoption of the Resource Description Framework (RDF) format for saving of web-based data. This massive adoption has paved the way for the development of various centralized and distributed RDF processing engines. These engines employ different mechanisms to implement key components of the query processing engines such as data storage, indexing, language support, and query execution. All these components govern how queries are executed and can have a substantial effect on the query runtime. For example, the storage of RDF data in various ways significantly affects the data storage space required and the query runtime performance. The type of indexing approach used in RDF engines is key for fast data lookup. The type of the underlying querying language (e.g., SPARQL or SQL) used for query execution is a key optimization component of the RDF storage solutions. Fi...

View PDFchevron_right
Holistic and scalable ranking of RDF data
Kunal Jha

2017 IEEE International Conference on Big Data (Big Data), 2017

View PDFchevron_right
Benchmarking rdf schemas for the semantic web
Vassilis Christophides

2002

Describing web resources using formal knowledge (ie, creating metadata according to a formal representation of a domain of discourse) is the essence of the next evolution step of the Web, termed the Semantic Web. The W3C's RDF/S (Resource Description Framework/Schema Language) enables the creation and exchange of resource metadata as normal web data. In this paper, we investigate the use of RDFS schemas as a means of knowledge representation and exchange in diverse application domains.

View PDFchevron_right

References (28)

  1. Aluç, G., Hartig, O., Özsu, M.T., Daudjee, K.: Diversified Stress Testing of RDF Data Management Systems. In: Mika, P., Tudorache, T., Bernstein, A., Welty, C., Knoblock, C., Vrandečić, D., Groth, P., Noy, N., Janowicz, K., Goble, C. (eds.) The Semantic Web -ISWC 2014. pp. 197-212. Springer International Publishing, Cham (2014)
  2. Beckett, D., Berners-Lee, T., Prud'hommeaux, E., Carothers, G.: RDF 1.1 Turtle -Terse RDF Triple Language. Tech. rep., World Wide Web Consrotium (2014), https://www.w3.org/TR/turtle/
  3. Beek, W., Rietveld, L., Bazoobandi, H.R., Wielemaker, J., Schlobach, S.: Lod laundromat: a uniform way of publishing other people's dirty data. In: International Semantic Web Conference. pp. 213-228. Springer (2014)
  4. Berners-Lee, T., Hendler, J., Lassila, O., et al.: The semantic web. Scientific amer- ican 284(5), 28-37 (2001)
  5. Bizer, C., Schultz, A.: The Berlin SPARQL Benchmark. International Journal on Semantic Web & Information Systems 5(2), 1-24 (2009)
  6. Brickley, D., Guha, R.: RDF Schema 1.1. Tech. rep., World Wide Web Consrotium (2014), https://www.w3.org/TR/rdf-schema/
  7. Conrads, F., Lehmann, J., Saleem, M., Morsey, M., Ngomo, A.N.: Iguana: A generic framework for benchmarking the read-write performance of triple stores. In: The Semantic Web -ISWC 2017 -16th International Semantic Web Conference, Vienna, Austria, October 21-25, 2017, Proceedings, Part II. pp. 48-65 (2017)
  8. Daga, E., Meroño-Peñuela, A., Motta, E.: Modelling and Querying Lists in RDF. A Pragmatic Study. In: 3rd Workshop on Querying and Benchmarking the Web of Data (QuWeDa 2019), ISWC 2019 (under review) (2019)
  9. Eilbeck, K., Lewis, S.E., Mungall, C.J., Yandell, M., Stein, L., Durbin, R., Ash- burner, M.: The sequence ontology: a tool for the unification of genome annotations. Genome biology 6(5), R44 (2005)
  10. Gangemi, A.: Ontology Design Patterns for Semantic Web Content. In: The Seman- tic Web -ISWC 2005. 4th International Semantic Web Conference. pp. 262-276. Springer (2005)
  11. Gopan, D., Reps, T., Sagiv, M.: A framework for numeric analysis of array oper- ations. SIGPLAN Not. 40(1), 338-350 (Jan 2005), http://doi.acm.org.vu-nl. idm.oclc.org/10.1145/1047659.1040333
  12. Görlitz, O., Thimm, M., Staab, S.: Splodge: Systematic generation of sparql bench- mark queries for linked open data. In: International Semantic Web Conference. pp. 116-132. Springer (2012)
  13. Guo, Y., Pan, Z., Heflin, J.: LUBM: A Benchmark for OWL Knowledge Base Systems. Journal of Web Semantics -Science, Services and Agents on the World Wide Web 3(2), 158-182 (2005)
  14. Hobbs, J.R., Pan, F.: Time Ontology in OWL. W3C working draft 27, 133 (2006)
  15. Hopcroft, J.E., Ullman, J.D.: Data structures and algorithms (1983)
  16. Ley, M.: The dblp computer science bibliography: Evolution, research issues, per- spectives. In: International symposium on string processing and information re- trieval. pp. 1-10. Springer (2002)
  17. Meroño-Peñuela, A., Hoekstra, R.: The Song Remains The Same: Lossless Con- version and Streaming of MIDI to RDF and Back. In: The Semantic Web: ESWC Satellite Events (ESWC 2016). LNCS, vol. 9989, pp. 194-199. Springer (2016)
  18. Meroño-Peñuela, A., Hoekstra, R., Gangemi, A., Bloem, P., de Valk, R., Stringer, B., Janssen, B., de Boer, V., Allik, A., Schlobach, S., Page, K.: The MIDI Linked Data Cloud. In: The Semantic Web -ISWC 2017, 16th International Semantic Web Conference. vol. 10587, pp. 156-164 (2017)
  19. Morsey, M., Lehmann, J., Auer, S., Ngomo, A.C.N.: Dbpedia sparql benchmark- performance assessment with real queries on real data. In: International Semantic Web Conference. pp. 454-469. Springer (2011)
  20. Reingold, E.M., Nievergelt, J., Deo, N.: Combinatorial algorithms: theory and prac- tice. Prentice Hall College Div (1977)
  21. Saleem, M., Ali, M.I., Mehmood, Q., Hogan, A., Ngomo, A.C.N.: LSQ: Linked SPARQL Queries Dataset. In: The Semantic Web -ISWC 2015. LNCS, vol. 9367, pp. 261-269. Springer (2015)
  22. Schmidt, M., Hornung, T., Lausen, G., Pinkel, C.: SPˆ2Bench: a SPARQL perfor- mance benchmark. In: Data Engineering, 2009. ICDE'09. IEEE 25th International Conference on. pp. 222-233. IEEE (2009)
  23. Schreiber, G., Raimond, Y.: RDF 1.1 Primer. Tech. rep., World Wide Web Con- srotium (2014), https://www.w3.org/TR/rdf11-primer/
  24. Sporny, M., Kellogg, G., Lanthaler, M.: JSON-LD 1.0. Tech. rep., World Wide Web Consrotium (2014), https://www.w3.org/TR/2014/REC-json-ld-20140116/
  25. Thakker, D., Osman, T., Gohil, S., Lakin, P.: A pragmatic approach to semantic repositories benchmarking. In: Extended Semantic Web Conference. pp. 379-393. Springer (2010)
  26. The MIDI Manufacturers Association: MIDI 1.0 Detailed Specification. Tech. rep., Los Angeles, CA (1996-2014), https://www.midi.org/specifications
  27. Van Hage, W.R., Malaisé, V., Segers, R., Hollink, L., Schreiber, G.: Design and use of the simple event model (sem). Web Semantics: Science, Services and Agents on the World Wide Web 9(2), 128-136 (2011)
  28. Vandenbussche, P.Y., Atemezing, G.A., Poveda-Villalón, M., Vatant, B.: Linked Open Vocabularies (LOV): a gateway to reusable semantic vocabularies on the Web. Semantic Web 8(3), 437-452 (2017)

Related papers

’ s repository of research publications and other research outputs Modelling and Querying Lists in RDF . A Pragmatic Study Conference or Workshop Item
Enrico Daga

2019

Many Linked Data datasets model elements in their domains in the form of lists: a countable number of ordered resources. When publishing these lists in RDF, an important concern is making them easy to consume. Therefore, a well-known recommendation is to find an existing list modelling solution, and reuse it. However, a specific domain model can be implemented in different ways and vocabularies may provide alternative solutions. In this paper, we argue that a wrong decision could have a significant impact in terms of performance and, ultimately, the availability of the data. We take the case of RDF Lists and make the hypothesis that the efficiency of retrieving sequential linked data depends primarily on how they are modelled (triple-store invariance hypothesis). To demonstrate this, we survey different solutions for modelling sequences in RDF, and propose a pragmatic approach for assessing their impact on data availability. Finally, we derive good (and bad) practices on how to publ...

View PDFchevron_right
How to Benchmark RDF Engines and Not Die Trying
Maria-Esther Vidal

Existing RDF engines have developed optimization techniques and efficient physical operators that speed up execution time. Additionally, some of these approaches have implemented structures to efficiently store and access RDF data, and have developed execution strategies able to reuse data previously stored in cache. In order to study the benefits of the techniques offered by each approach, particular datasets have to be considered as well as different benchmarks of queries. Queries must be of different complexity, number of patterns, selectivity and shape, while datasets have to be of different sizes and correlations between subjects, predicates and objects. In this paper, we describe the benchmarks that we have developed to analyze the performance of existing RDF engines when creating and indexing RDF structures, as well as the quality of the optimization and execution strategies implemented by these engines.

View PDFchevron_right
DBtrends : Publishing and Benchmarking RDF Ranking Functions
Edgard Marx

2016

Providing accurate approaches for keyword search or question answering to access the data available on the Linked Data Web is of central importance to ensure that it can be used by non-experts. In many cases, these approaches return a large number of results that need to be provided in the right order so as to be of relevance to the user. Achieving the goal of improving the access to the Linked Data Web thus demands the provision of ranking approaches that allow sorting potentially large number of results appropriately. While such functions have been designed in previous works, they have not been evaluated exhaustively. This work addresses this research gap by proposing a formal framework designed towards comparing and evaluating different rank functions for RDF data. The framework allows combining these rankings by means of an extension of the Spearman's footrule estimation of the upper bound of this function. We supply a benchmark with a total of 60 manually annotated entity ranks by users from USA and India recruited over Amazon Mechanical Turk. Moreover, we evaluated nine entity ranking functions over the proposed benchmark.

View PDFchevron_right
Characterising RDF data sets
Pablo Fuente Redondo

Journal of Information Science, 2017

The publication of semantic web data, commonly represented in Resource Description Framework (RDF), has experienced outstanding growth over the last few years. Data from all fields of knowledge are shared publicly and interconnected in active initiatives such as Linked Open Data. However, despite the increasing availability of applications managing large-scale RDF information such as RDF stores and reasoning tools, little attention has been given to the structural features emerging in real-world RDF data. Our work addresses this issue by proposing specific metrics to characterise RDF data. We specifically focus on revealing the redundancy of each data set, as well as common structural patterns. We evaluate the proposed metrics on several data sets, which cover a wide range of designs and models. Our findings provide a basis for more efficient RDF data structures, indexes and compressors.

View PDFchevron_right
LITMUS: An Open Extensible Framework for Benchmarking RDF Data Management Solutions
Harsh Thakkar

ArXiv, 2016

Developments in the context of Open, Big, and Linked Data have led to an enormous growth of structured data on the Web. To keep up with the pace of efficient consumption and management of the data at this rate, many data Management solutions have been developed for specific tasks and applications. We present LITMUS, a framework for benchmarking data management solutions. LITMUS goes beyond classical storage benchmarking frameworks by allowing for analysing the performance of frameworks across query languages. In this position paper we present the conceptual architecture of LITMUS as well as the considerations that led to this architecture.

View PDFchevron_right

Related topics

  • Computer Science
  • Information Retrieval
  • Linked Data

    • Cited by

    Sequential linked data: The state of affairs
    Enrico Daga

    Semantic Web, 2021

    Sequences are among the most important data structures in computer science. In the Semantic Web, however, little attention has been given to Sequential Linked Data. In previous work, we have discussed the data models that Knowledge Graphs commonly use for representing sequences and showed how these models have an impact on query performance and that this impact is invariant to triplestore implementations. However, the specific list operations that the management of Sequential Linked Data requires beyond the simple retrieval of an entire list or a range of its elements – e.g. to add or remove elements from a list –, and their impact in the various list data models, remain unclear. Covering this knowledge gap would be a significant step towards the realization of a Semantic Web list Application Programming Interface (API) that standardizes list manipulation and generalizes beyond specific data models. In order to address these challenges towards the realization of such an API, we buil...

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved