Semantic ontologies for multimedia indexing (SOMI)

Lecture Notes in Computer Science, 2002

In this paper, an integrated information system is presented that offers enhanced search and retrieval capabilities to users of heterogeneous digital audiovisual (a/v) archives. This novel system exploits the advances in handling a/v content and related metadata, as introduced by MPEG-4 and worked out by MPEG-7, to offer advanced access services characterized by the tri-fold "semantic phrasing of the request (query)", "unified handling" and "personalized response". The proposed system is targeting the intelligent extraction of semantic information from a/v and text related data taking into account the nature of useful queries that users may issue, and the context determined by user profiles. From a technical point of view, it will play the role of an intermediate access server residing between the end users and multiple heterogeneous audiovisual archives organized according to new MPEG standards.

2011

Although oral culture has been part of our history for thousands of years, we have only fairly recently been enabled to record and preserve that part of our heritage. Over the past century millions of hours of audiovisual data have been collected. 2 Typically, audiovisual (A/V) archival institutes are the keepers of these collections, a significant part of which contains spoken word materials, such as interviews, speeches and radio broadcasts.

Despite its scientific, political, and practical value, comprehensive information about human languages, in all their variety and complexity, is not readily obtainable and searchable. One reason is that many language data are collected as audio and video recordings which imposes a challenge to document indexing and retrieval. Annotation of multimedia data provides an opportunity for making the semantics explicit and facilitates the searching of multimedia documents. We have developed OntoELAN, an ontology-based linguistic multimedia annotator that features: (1) support for loading and displaying ontologies specified in OWL; (2) creation of a language profile, which allows a user to choose a subset of terms from an ontology and conveniently rename them if needed; (3) creation of ontological tiers, which can be annotated with profile terms and, therefore, corresponding ontological terms; and (4) saving annotations in the XML format as Multimedia Ontology class instances and, linked to them, class instances of other ontologies used in ontological tiers. To our best knowledge, OntoELAN is the first audio/video annotation tool in linguistic domain that provides support for ontology-based annotation.

Methods, Standards and Tools, 2005

By the end of the last century the question was not whether digital archives are technically and economically viable, but rather how digital archives would be efficient and informative. In this framework, different scientific fields such as, on the one hand, development of database management systems, and, on the other hand, processing and analysis of multimedia data, as well as artificial and computational intelligence methods, have observed a close cooperation with each other during the past few years. The attempt has been to develop intelligent and efficient human-computer interaction systems, enabling the user to access vast amounts of heterogeneous information, stored in different sites and archives.

Journal of Intelligent Information Systems, 2005

To improve the accuracy in terms of precision and recall of an audio information retrieval system we have created a domain-specific ontology (a collection of key concepts and their interrelationships), as well as a novel, pruning algorithm. Given the shortcomings of keyword-based techniques, we have opted to employ a concept-based technique utilizing this ontology. Achieving high precision and high recall is the key problem in the retrieval of audio information. In traditional approaches, high recall is typically achieved at the expense of low precision, and vice versa. Through the use of a domain-specific ontology appropriate concepts can be identified during metadata generation (description of audio) or query generation, thus improving precision.

Multimedia Tools and Applications, 2010

The objective of this special issue is to collect and report on recent work that aims at narrowing the large disparity between the low-level descriptors that can be computed automatically from multimedia content and the richness of subjectivity of semantics in user queries and human interpretations of audiovisual data. It represents a step towards bridging the gap between research cultures at extreme ends of the semantic multimedia spectrum. At one end, the semantic web and its supporting technologies are becoming established in both the open data environment and within specialist domains, such as corporate intranet search, e-Science (particularly life sciences), and cultural heritage. To facilitate the world-wide sharing of media, W3C is developing standard ways of denoting fragments of audiovisual content and of specifying and associating semantics with these. At the other end of the spectrum, media analysis tools continue to grow in sophistication, identifying features that can then be associated with explicit Multimed Tools Appl

Knowledge acquisition from multimedia content workshop, Genova, Italy, 2007

Abstract. The task of multimedia document categorization forms a well-known problem in information retrieval. The task is to assign a multimedia document to one or more categories, based on its contents. In this case, effective management and thematic categorization requires usually the extraction of the underlying semantics. The proposed approach utilizes as input, analyzes and exploits the textual annotation that accompanies a multimedia document, in order to extract its underlying semantics, construct a semantic index and ...

Semantic Web aims to lift current Web into semantic repositories where heterogeneous data can be queried and different services can be mashed up. Here we report some of on-going work with the EASAIER project to enable enhanced access to sound archives by integrating archives based on Music Onotlogy and provide different search results from different mashups.

Lecture Notes in Computer Science, 2003

Describing audio-visual documents amounts to consider documentary aspects (the structure) as well as conceptual aspects (the content). In this paper, we propose an architecture which describes formally the content of the videos and which constrains the structure of their descriptions. This work is based on languages and technologies underlying the Semantic Web and in particular ontologies. Therefore, we propose to combine emerging Web standards, namely MPEG-7/XML Schema for the structural part and OWL/RDF for the knowledge part of the description. Finally, our work offers reasoning support on both aspects when querying a database of videos.