Retrieving Consistent Multimedia Presentation Fragments

1991

This paper proposes a multimedia presentation system Harmony which can deal with temporal and active media such as motion video, computer animation, and music sounds. Harmony is based on the notion of a hyperobject which integrates a hypertext system with an object-oriented framework. Each object is considered a node as is usual in hypertext systems, and the relations between objects are represented as links between nodes. Harmony extends the ordinary notion of a link. A link in Harmony consists of three components: objects connected by the link, conditions specifying when the link is navigated, and messages sent to the target object. In addition to such an extension, each object can include internal objects called subobjects which specify parts of an object and can become a source/destination of a link. Furthermore, the notion of a group object is introduced to represent a synchronization of parallel displayed media information. Through these extended notions of a hypertext model, temporal and active media can be easily handled through the description of temporal relations from the media in hypermedia documents. Based on the design of Harmony, we have implemented a prototype multimedia presentation system which deals with text, music, graphics, motion video, and computer animation as objects. Harmony consists of three subsystems: link manager subsystem (Harmony/LM), user interface subsystem (Harmony/UI), and object-oriented database (Harmony/DB), where a commercial objectoriented database was employed as the Harmony/DB. Scenario viewer was developed for displaying the structure of scenario in a tree graph which can reduce a cognitive overhead in a hypermedia document. The whole system was constructed based on C++ language.

Information and Software Technology, 1993

We propose a simple and efficient way to synchronize multimedia presentation. First, we introduce a conceptual multimedia system architecture in which synchronization actions are performed globally by a Global Synchronization Manager (GSM) which is part of the Multimedia Database Management System, and locally by Media Specific Synchronizers. Next, we propose a presentation scenario convention that is easy to create and manipulate. We assume that synchronization is driven by real time and not by synchronization events, and that coordination of scenario entries with Global Time (GT) is managed by the GSM. User interaction with the presentation creates entries in the GSM log of special events which in coordination with the relationship matrix will adjust the actual time included in the scenario entries. Entries in the presentation scenario contain presentation actions that are fired (activated) when the time field of that entry rendezvous with the GT. multimedia data, system architecture, synchronization, time factor, interaction Vol

Semantic annotation of multimedia data for improving search engine performance is an important issue. We have focused on the automated annotation of video recordings of university lectures. Most times these lectures are supported by a desktop presentation provided by the lecturer. This presentation in general contains an outline of the given lecture and thus, enables automated annotation of the corresponding video recording. We have implemented a tool for automated MPEG-7 annotation of video recorded lectures, which enables efficient content based search within a video database. Given a particular search term our search engine prototype is able to deliver only the relevant sequences of the matching video recordings.

PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON VERY LARGE DATA BASES, 1995

An important requirement for multimedia presentations is the ability to compose new multimedia objects from the existing ones using temporal relationships. When compositions of continuous media objects are speci ed dynamically, the task of displaying these objects poses new challenges. These challenges are addressed in this paper. We show that in the case of a single composite object retrieval, a prefetching technique, simple sliding, provides an approach to reduce latency and bu ering requirements. We extend ...

1992

Schemes for specifying synchronization relations between multimedia data have important rami cations from user level speci cation to system level schedule generation. This paper examines the problems of synchronizing multimedia objects in a multimedia presentation. We present a scheme to handle the general problem of temporal relationship speci cation and scheduling. The scheme proposes a two phased approach with a speci cation phase and a scheduling phase. The rst phase focuses on assisting users to construct a consistent temporal relationship speci cation. The second phase ensures that the actual presentation proceeds according to the consistent temporal relationship speci cation despite dynamic factors such as resource contention. The use of Synchronization Relation Tree (SRT) as a formal temporal relationship speci cation to capture the temporal information is proposed. Consistency checks can be performed on the SRT during the speci cation phase. In order to guarantee the temporal relationships among various media objects as speci ed in a SRT, a pair of message passing protocols is proposed to generate the accurate timing values required for actual presentation of the multimedia objects.

Multimedia Information Systems, 1998

This paper presents a data model tailored for multimedia data representation, along with the main characteristics of a Multimedia Query Language that exploits the features of the proposed model. The model addresses data presentation, manipulation and content-based retrieval. It consists of three parts: a Multimedia Description Model, which provides a structural view of raw multimedia data, a Multimedia Presentation Model, and a Multimedia Interpretation Model which allows semantic information to be associated with multimedia data. The paper focuses on the structuring of a multimedia data model which provides support for content-based retrieval of multimedia data. The Query Language is an extension of a traditional query language which allows restrictions to be expressed on features, concepts, and the structural aspects of the objects of multimedia data and the formulation of queries with imprecise conditions. The result of a query is an approximate set of database objects which partially match such a query.

… the Gap: From Information Extraction to Semantic …, 2006

Semantic annotation of multimedia data for improving search engine performance is an important issue. We have focused on the automated annotation of video recordings of university lectures. Most times these lectures are supported by a desktop presentation provided by the lecturer. This presentation in general contains an outline of the given lecture and thus, enables automated annotation of the corresponding video recording. We have implemented a tool for automated MPEG-7 annotation of video recorded lectures, which enables efficient content based search within a video database. Given a particular search term our search engine prototype is able to deliver only the relevant sequences of the matching video recordings.

EURASIP Journal on Advances in Signal Processing, 2003

The amount of digital video being shot, captured, and stored is growing at a rate faster than ever before. The large amount of stored video is not penetrable without efficient video indexing, retrieval, and browsing technology. Most prior work in the field can be roughly categorized into two classes. One class is based on image processing techniques, often called content-based image and video retrieval, in which video frames are indexed and searched for visual content. The other class is based on spoken document retrieval, which relies on automatic speech recognition and text queries. Both approaches have major limitations. In the first approach, semantic queries pose a great challenge, while the second, speech-based approach, does not support efficient video browsing. This paper describes a system where speech is used for efficient searching and visual data for efficient browsing, a combination that takes advantage of both approaches. A fully automatic indexing and retrieval system has been developed and tested. Automated speech recognition and phonetic speech indexing support text-to-speech queries. New browsable views are generated from the original video. A special synchronized browser allows instantaneous, context-preserving switching from one view to another. The system was successfully used to produce searchable-browsable video proceedings for three local conferences.

1997

We introduce a methodology to dynamically adjust the multimedia presentations as a result of user interactions during the playout. A constraint-driven approach is described for the automated assembly, organization and playout of presentations from multimedia databases. To allow interactive user control over the playout of an automatically generated presentation, an arbitrary number of control buttons are designated in the multimedia presentation system. Through these buttons, the user is able to change/a ect the ow of the presentation in certain ways. User interactions, provided during the presentation playout, can cause delays in the presentation schedule of multimedia segments or they may change playout constraints. To account for these delays and changes, we propose to (a) dynamically reorganize the subpresentations into a presentation, (b) replace segments with their compacted clips, and (c) delete segments on the presentation graph.

IEEE 6th Workshop on Multimedia Signal Processing, 2004.

Multimedia databases are extending the scope of traditional databases to handle the complex structure of multi• media objects. Models for multimedia information must include representations for the structure and content of severa] media in a form that allows flexibility in retrie,'al. Content•based retrienl is the main motivation behind recent research in multimedia databases. The task of searching in video and audio content is made hard by the nature of audiovisual data where, unlike text, there is no direct syntactic channel between the object and its meaning [1]-[3]. We propose a model for multimedia content storage and retrie,'al accounting for both context and content information and taking adl'antage of their dependencies for effective retrieval. We then describe a prototype multimedia database with a retrieval interface. It has been used as a workbench for testing the representation model and integrating tools for feature extraction, information interchange and retriel'al. The workbench allows an easy inclusion of new tools for content analysis and new methods for context• and content-based retrieval while offering storage and access for both the actual digital content and its metadata.