Components and systems for interactive video indexing

2002

The indexing and retrieval of multimedia items is difficult due to the semantic gap between the user’s perception of the data and the descriptions we can derive automatically from the data using computer vision, speech recognition, and natural language processing. In this contribution we consider the nature of the semantic gap in more detail and show examples of methods that help in limiting the gap. These methods can be automatic, but in general the indexing and retrieval of multimedia items should be a collaborative process between the system and the user. We show how to employ the user’s interaction for limiting the semantic gap.

Multimedia Retrieval, 2007

The current level of request for applications involving content-based video coding and video information retrieval is increasing the relevance of systems able to somehow 'understand' the content of video sequences. This type of systems will enable the provision of services where enhanced user interaction with the visual content is supported. Such an object-based video coder is being specified by ISO under the MPEG-4 project [1], while the standardization of video description capabilities to support content-based video indexing and retrieval is being considered by MPEG-7 [2]. In this paper, we discuss the issue of video content identification and characterization, and the importance of providing the means for the user to interact with the analysis process so that the achieved results can have a meaningful and powerful semantic value.

Lecture Notes in Computer Science, 2005

A video database can contain a large number of videos ranging from several minutes to several hours in length. Typically, it is not sufficient to search just for relevant videos, because the task still remains to find the relevant clip, typically less than one minute of length, within the video. This makes it important to direct the users attention to the most promising material and to indicate what material they already investigated. Based on this premise, we created a video search system with a powerful and flexible user interface that incorporates dynamic visualizations of the underlying multimedia objects. The system employes an automatic story segmentation, combines text and visual search, and displays search results in ranked sets of story keyframe collages. By adapting the keyframe collages based on query relevance and indicating which portions of the video have already been explored, we enable users to quickly find relevant sections. We tested our system as part of the NIST TRECVID interactive search evaluation, and found that our user interface enabled users to find more relevant results within the allotted time than other systems employing more sophisticated analysis techniques but less helpful user interfaces.

… Indexing (CBMI), 2011 …, 2011

In this paper we present two interactive systems for video search and browsing; one is a web application based on the Rich Internet Application paradigm, designed to obtain the levels of responsiveness and interactivity typical of a desktop application, while the other exploits multi-touch devices to implement a multi-user collaborative application. Both systems use the same ontology-based video search engine, that is capable of expanding user queries through ontology reasoning, and let users to search for specific video segments that contain a semantic concept or to browse the content of video collections, when it's too difficult to express a specific query.