Evaluating Ontology Modules Using an Entropy Inspired Metric

2014

Ontology evaluation is concerned with ascertain two important aspects of ontologies: quality and correctness. The distinction between the two is attempted in this survey as a way to better approach ontology evaluation. The role that ontologies play on the semantic web at large has been has been argued to have catalyzed the proliferation of ontologies in existence. This has also presented the challenge of deciding one the suitability of an given ontology to one’s purposes as compared with another ontology in a similar domain. This survey intends to analyze the state of the art in ontology evaluate spanning such topics as the approaches to ontology evaluation, the metrics and measures used. Particular interest is given to Data-driven ontology evaluation with special emphasis on the notion of bias and it’s relevance to evaluation results. Chief among the outputs of this survey is the gap analysis on the topic of ontology evaluation.

Description Logics, 2019

Many large ontologies are developed modularly, often using import statements, which are supported by the OWL standard. However, import statements do not provide logical guarantees such as local completeness, which is an established quality criterion for ontology modules: an ontology is locally complete if it uses terms from imported ontologies without changing the knowledge reused from them. To measure the extent to which ontologies separated by import statements are logically modular, we present four new quantitative logic-based metrics: two are strongly related to local completeness and based on module extraction, using some established module notion as a reference; the other two exploit the dependency relation of the atomic decomposition. We formally study the relationship between the measures and evaluate them on a set of ontologies.

2010

Efficiently extracting a module from a given ontology that captures all the ontology's knowledge about a set of specified terms is a well-understood task. This task can be based, for instance, on locality-based modules. In contrast, extracting all modules of an ontology is computationally difficult because there can be exponentially many. However, it is reasonable to assume that, by revealing the modular structure of an ontology, we can obtain information about its topicality, connectedness, structure, superfluous parts, or agreement between actual and intended modeling. Furthermore, incremental reasoning makes use of a number of, although not all possible, modules of an ontology. We report on experiments to estimate the number of modules of real-life ontologies. We also evaluate the modular structure of ontologies that we succeeded to fully modularise. In that evaluation, we look at the number and sizes of the modules, as well as the relation between module size and number and ...

Proceedings of the International Conference on Knowledge Engineering and Ontology Development, 2013

Ontology modularization is crucial to support knowledge reuse on the ever increasing Semantic Web. However, modularization methods that serve the reuse goal are often intended for humans to assist them in building new ontologies, rather than for applications that need only a relevant part of an existing ontology. Moreover, modules obtained are always subject to verification and maintenance by humans to validate the semantic consistency of their contents. In this paper, we investigate how semantic comparisons may provide a module relevant to a set of terms which are not part of the ontology. Our objective is to extract a module which may be usable as a separate ontology. The user does not need to be familiar with the exact terms used inside the ontology beforehand to extract from it a module for a specific application/knowledge sub domain.

An ontology is an explicit formal conceptualization of some domain of interest. Ontologies are increasingly used in various fields such as knowledge management, information extraction, and the semantic web. Ontology evaluation is the problem of assessing a given ontology from the point of view of a particular criterion of application, typically in order to determine which of several ontologies would best suit a particular purpose. This paper presents a survey of the state of the art in ontology evaluation.

2011

In the Information Sciences an ontology specifies the conceptual structure of a knowledge domain through its vocabulary and its meaning, or semantics. Examples of ontologies include database schemas, taxonomies and library classification schemes. Computer-based ontologies specify a common understanding of a domain both for its different human users and for automated information processes such as interpretation, aggregation, manipulation, etc. As an ontology is fundamental to systems in an organisation and to communication between organisations, its quality is critically important for the sound operation of those systems. But there is a problem in that there seems to be no systematic basis or method for conducting an assessment of the quality of an ontology. There is an abundance of literature proposing ideas about what constitutes a 'good' ontology, but it is diverse, incoherent and untested. This paper reports research which reviewed the literature, extracted some 220 quali...

First Monday, 2007

Ontologies are important knowledge representation and sharing tools in the workflow of biology research. The high cost of creating and maintaining ontologies encourages their sharing and reuse, and an increasing number of ontologies have been made available from different sources, with different models of curation. To enable effective selection, reuse, integration, and maintenance of these ontologies, however, one needs to have a systematic method of evaluating and connecting their quality to the context of an intended use. Based on an analysis of the activity system and Web server logs of the Morphbank biodiversity research data repository (http://www.morphbank.net/), a model was developed to evaluate ontology quality. The model connects the types of quality problems with the types of research activities and suggests relevant metrics. The paper also describes the structure of some of the research activities and the types and patterns of end-user searches in Morphbank. Contents Introduction Related research Research setup and method Analysis Constructing a quality evaluation model Conclusion

ECAI Workshop on Ontology Learning and …, 2004

The need for the establishment of evaluation methods that can measure respective improvements or degradations of ontological models -e.g. yielded by a precursory ontology population stage -is undisputed. We propose an evaluation scheme that allows to employ a number of different ontologies and to measure their performance on specific tasks. In this paper we present the resulting task-based approach for quantitative ontology evaluation, which also allows for a bootstrapping approach to ontology population. Benchmark tasks commonly feature a so-called gold-standard defining perfect performance. By selecting ontology-based approaches for the respective tasks, the ontology-dependent part of the performance can be measured. Following this scheme, we present the results of an experiment for testing and incrementally augmenting ontologies using a well-defined benchmark problem based on a evaluation goldstandard. ¥ It might even be impossible, as it is widely acknowledged that ontology engineering, employment and evaluation has many task-dependent features and constraints. ¡

Semantic Web, 2007

Recent years have seen rapid progress in the development of ontologies as semantic models intended to capture and represent aspects of the real world. There is, however, great variation in the quality of ontologies. If ontologies are to become progressively better in the future, more rigorously developed, and more appropriately compared, then a systematic discipline of ontology evaluation must be created to ensure quality of content and methodology. Systematic methods for ontology evaluation will take into account representation of individual ontologies, performance (in terms of accuracy, domain coverage and the efficiency and quality of automated reasoning using the ontologies) on tasks for which the ontology is designed and used, degree of alignment with other ontologies and their compatibility with automated reasoning. A sound and systematic approach to ontology evaluation is required to transform ontology engineering into a true scientific and engineering discipline. This chapter discusses issues and problems in ontology evaluation, describes some current strategies, and suggests some approaches that might be useful in the future.

2008

In the last years, several methodologies for ontology engineering have been proposed. Most of these methodologies guide the engineer from a first paper draft to an implemented -mostly description logics-based -ontology. A quality assessment of how accurately the resulting ontology fits the initial conceptualization and intended application has not been proposed so far. In this paper, we investigate the role of semantic similarity as a quality indicator. Based on similarity rankings, our approach allows for a qualitative estimation whether the domain experts' initial conceptualization is reflected by the developed ontology and whether it fits the users' application area. Our approach does not propose yet another ontology engineering methodology but can be integrated into existing ones. A plug-in to the Protégé ontology editor implementing our approach is introduced and applied to a scenario from hydrology. The benefits and restrictions of similarity as a quality indicator are pointed out.