Automated approach for quality assessment of RDF resources

First Monday, 2007

Ontologies are important knowledge representation and sharing tools in the workflow of biology research. The high cost of creating and maintaining ontologies encourages their sharing and reuse, and an increasing number of ontologies have been made available from different sources, with different models of curation. To enable effective selection, reuse, integration, and maintenance of these ontologies, however, one needs to have a systematic method of evaluating and connecting their quality to the context of an intended use. Based on an analysis of the activity system and Web server logs of the Morphbank biodiversity research data repository (http://www.morphbank.net/), a model was developed to evaluate ontology quality. The model connects the types of quality problems with the types of research activities and suggests relevant metrics. The paper also describes the structure of some of the research activities and the types and patterns of end-user searches in Morphbank. Contents Introduction Related research Research setup and method Analysis Constructing a quality evaluation model Conclusion

Journal of Information Science, 2017

The publication of semantic web data, commonly represented in Resource Description Framework (RDF), has experienced outstanding growth over the last few years. Data from all fields of knowledge are shared publicly and interconnected in active initiatives such as Linked Open Data. However, despite the increasing availability of applications managing large-scale RDF information such as RDF stores and reasoning tools, little attention has been given to the structural features emerging in real-world RDF data. Our work addresses this issue by proposing specific metrics to characterise RDF data. We specifically focus on revealing the redundancy of each data set, as well as common structural patterns. We evaluate the proposed metrics on several data sets, which cover a wide range of designs and models. Our findings provide a basis for more efficient RDF data structures, indexes and compressors.

Evaluating the Gap Between an RDF Dataset and Its Schema. ER Workshops 2015., 2015

An increasing number of linked datasets is published on the Web, using RDF(S)/OWL. The availability of the schema describing these datasets is crucial for their meaningful usage. A dataset may contain schema-related information, however, languages do not impose any constraint on their structure, and a gap may therefore exist between the schema and the actual instances. In this paper, we tackle the problem of evaluating this gap. We present an approach relying on both type and class profiles, as well as a set of quality metrics. We also present some experimental evaluations to illustrate the use of the proposed metrics.

2016 IEEE Tenth International Conference on Semantic Computing (ICSC), 2016

The Web meanwhile got complemented with a Web of Data. Examples are the Linked Open Data cloud, the RDFa and Microformats data increasingly being embedded in ordinary Web pages, or the schema.org initiative. However, the Web of Data shares many characteristics with the original Web of documents, for example, varying quality. There are a large variety of dimensions and measures of data quality. Hence, the assessment of of quality in terms of fitness for use with respect to a certain use case is challenging. In this article, we present a comprehensive and extensible framework for the automatic assessment of linked data quality. Within this framework we implemented around 30 data quality metrics. A particular focus of our work is on scalability and support for the evolution of data. Regarding scalability, we follow a stream processing approach, which provides an easy interface for the integration of domain specific quality measures. With regard to the evolution of data, we introduce data quality assessment as a stage of a holistic data life cycle.

Semantic Web

RDF Graph Summarization pertains to the process of extracting concise but meaningful summaries from RDF Knowledge Bases (KBs) representing as close as possible the actual contents of the KB. RDF Summarization allows for better exploration and visualization of the underlying RDF graphs, optimizstion of queries or query evaluation in multiple steps, better understanding of connections in Linked Datasets and many other applications. In the literature, there are efforts reported presenting algorithms for extracting summaries from RDF KBs. These efforts though provide different results while applied on the same KB, thus a way to compare the produced summaries and decide on their quality, in the form of a quality framework, is necessary. So in this work, we propose a comprehensive Quality Framework for RDF Graph Summarization that would allow a better, deeper and more complete understanding of the quality of the different summaries and facilitate their comparison. We work at two levels: the level of the ideal summary (or ideal schema) of the KB that could be provided by an expert user and the level of the instances contained by the KB. For the first level, we are computing how close the proposed summary is to the ideal solution (when this is available) by computing its precision and recall against the ideal solution. For the second level, we are computing if the existing instances are covered (i.e. can be retrieved) and in what degree by the proposed summary. We use our quality framework to test the results of three of the best RDF Graph Summarization algorithms, when summarizing different (in terms of content) and diverse (in terms of total size and number of instances, classes and predicates) KBs and we present comparative results for them. We conclude this work by discussing these results and the suitability of the proposed quality framework in order to get useful insights for the quality of the presented results.

arXiv: Digital Libraries, 2015

To ensure high quality of and trust in both metadata and data, their representation in RDF must satisfy certain criteria - specified in terms of RDF constraints. From 2012 to 2015 together with other Linked Data community members and experts from the social, behavioral, and economic sciences (SBE), we developed diverse vocabularies to represent SBE metadata and rectangular data in RDF. The DDI-RDF Discovery Vocabulary (DDI-RDF) is designed to support the dissemination, management, and reuse of unit-record data, i.e., data about individuals, households, and businesses, collected in form of responses to studies and archived for research purposes. The RDF Data Cube Vocabulary (QB) is a W3C recommendation for expressing data cubes, i.e. multi-dimensional aggregate data and its metadata. Physical Data Description (PHDD) is a vocabulary to model data in rectangular format, i.e., tabular data. The data could either be represented in records with character-separated values (CSV) or fixed le...

Proceedings of the 23rd international conference on World wide web - WWW '14, 2014

Linked Open Data (LOD) comprises of an unprecedented volume of structured data on the Web. However, these datasets are of varying quality ranging from extensively curated datasets to crowd-sourced or extracted data of often relatively low quality. We present a methodology for testdriven quality assessment of Linked Data, which is inspired by test-driven software development. We argue, that vocabularies, ontologies and knowledge bases should be accompanied by a number of test-cases, which help to ensure a basic level of quality. We present a methodology for assessing the quality of linked data resources, based on a formalization of bad smells and data quality problems. Our formalization employs SPARQL query templates, which are instantiated into concrete quality test queries. Based on an extensive survey, we compile a comprehensive library of data quality test patterns. We perform automatic test instantiation based on schema constraints or semi-automatically enriched schemata and allow the user to generate specific test instantiations that are applicable to a schema or dataset. We provide an extensive evaluation of five LOD datasets, manual test instantiation for five schemas and automatic test instantiations for all available schemata registered with LOV. One of the main advantages of our approach is that domain specific semantics can be encoded in the data quality test cases, thus being able to discover data quality problems beyond conventional quality heuristics.

2021

Data is of high quality if it is fit for its intended use in operations, decision-making, and planning. There is a colossal amount of linked data available on the web. However, it is difficult to understand how well the linked data fits into the modeling tasks due to the defects present in the data. Faults emerged in the linked data, spreading far and wide, affecting all the services designed for it. Addressing linked data quality deficiencies requires identifying quality problems, quality assessment, and the refinement of data to improve its quality. This study aims to identify existing end-to-end frameworks for quality assessment and improvement of data quality. One important finding is that most of the work deals with only one aspect rather than a combined approach. Another finding is that most of the framework aims at solving problems related to DBpedia. Therefore, a standard scalable system is required that integrates the identification of quality issues, the evaluation, and the improvement of the linked data quality. This survey contributes to understanding the state of the art of data quality evaluation and data quality improvement. A solution based on ontology is also proposed to build an end-to-end system that analyzes quality violations' root causes.

International Journal on Semantic Web and Information Systems, 2016

Ensuring data quality in Linked Open Data is a complex process as it consists of structured information supported by models, ontologies and vocabularies and contains queryable endpoints and links. In this paper, the authors first propose an objective assessment framework for Linked Data quality. The authors build upon previous efforts that have identified potential quality issues but focus only on objective quality indicators that can measured regardless on the underlying use case. Secondly, the authors present an extensible quality measurement tool that helps on one hand data owners to rate the quality of their datasets, and on the other hand data consumers to choose their data sources from a ranked set. The authors evaluate this tool by measuring the quality of the LOD cloud. The results demonstrate that the general state of the datasets needs attention as they mostly have low completeness, provenance, licensing and comprehensibility quality scores.

2017

The success of systems making use of ontology schemas depend mainly on the quality of their underlying ontologies. This has been acknowledged by researchers who responded by suggesting metrics to measure different aspects of quality. Tools have also been designed, but determining the set of quality metrics to use may not be a straightforward task. Research on ontology quality shows that detection of problems at an early stage of the ontology development cycle is necessary to reduce costs and maintenance at later stages, which is more difficult to achieve and requires more effort. Assessment using the right metrics is therefore crucial to identify key quality problems. This ensures that the data and instances of the ontology schema are sound and fit for purpose. Our contribution is a systematic survey on quality metrics applicable to ontologies in the Semantic Web, and preliminary investigation towards methods to visualise quality problems in ontologies.