Efficient Visual Content Retrieval and Mining in Videos

International Journal of Imaging Systems and Technology, 2008

The growth in available online video material over the Internet is generally combined with user-assigned tags or content description, which is the mechanism by which we then access such video. However, user-assigned tags have limitations for retrieval and often we want access where the content of the video itself is directly matched against a user's query rather than against some manually assigned surrogate tag. Content-based video retrieval techniques are not yet scalable enough to allow interactive searching on Internet-scale, but the techniques are proving robust and effective for smaller collections. In this article, we show three exemplar systems which demonstrate the state of the art in interactive, content-based retrieval of video shots, and these three are just three of the more than 20 systems developed for the 2007 iteration of the annual TRECVid benchmarking activity. The contribution of our article is to show that retrieving from video using content-based methods is now viable, that it works, and that there are many systems which now do this, such as the three outlined herein. These systems, and others can provide effective search on hundreds of hours of video content and are samples of the kind of contentbased search functionality we can expect to see on larger video archives when issues of scale are addressed.

2011

IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2000

Video indexing and retrieval have a wide spectrum of promising applications, motivating the interest of researchers worldwide. This paper offers a tutorial and an overview of the landscape of general strategies in visual content-based video indexing and retrieval, focusing on methods for video structure analysis, including shot boundary detection, key frame extraction and scene segmentation, extraction of features including static key frame features, object features and motion features, video data mining, video annotation, video retrieval including query interfaces, similarity measure and relevance feedback, and video browsing. Finally, we analyze future research directions.

1998

In this paper videos are analyzed to get a content-based decription of the video. The structure of a given video is useful to index long videos e ciently and automatically. A comparison between shots gives an overview about cut frequency, cut pattern, and scene bounds. After a shot detection the shots are grouped into clusters based on their visual similarity. A time-constraint clustering procedure is used to compare only those shots that are positioned inside a time range. Shots from di erent areas of the video (e.g., begin/end) are not compared. With this cluster information that contains a list about shots and their clusters it is possible to calculate scene bounds. A labeling of all clusters gives a declaration about the cut pattern. It is easy now to distinguish a dialogue from an action scene. The nal content analysis is done by the ImageMiner system. The ImageMiner system developed at the University of Bremen of the Image Processing Department of the Center for Computing Technology realizes content-based image retrieval for still images through a novel combination of methods and techniques of computer vision and arti cal intelligence. The ImageMiner system consists of three analysis modules for computer vision, namely for color, texture, and contour analysis. Additionally exists a module for object recognition. The output of the object recognition module can be indexed by a text retrieval system. Thus, concepts like forestscene may be searched for. We combine the still image analysis with the results of the video analysis in order to retrieve shots or scenes.

2013

In this paper, we present a system, named ASVT-DECTOR, to retrieve the near duplicate videos with large variations based on an 3D structure tensor model, named ASVT series, over the local descriptors of video segments. Different from the traditional global feature-based video detection systems that incur severe information loss, ASVT model is built over the local descriptor set of each video segment, keeping the robustness of local descriptors. Meanwhile, unlike the traditional local feature-based methods that suffer from the high cost of pair-wise descriptor comparison, ASVT model describes a video segment as an 3D structure tensor that is actually a 3 × 3 matrix, obtaining high retrieval efficiency. In this demonstration, we show that, given a clip, our ASVTDETECTOR system can effectively find the near-duplicates with large variations from a large collection in real time.

Multimedia Tools and Applications, 1997

The increasing development of advanced multimedia applications requires new technologies for organizing and retrieving by content databases of still digital images or digital video sequences. To this aim image and image sequence contents must be described and adequately coded. In this paper we describe a system allowing content-based annotation and querying in video databases. No user action is required during the database population step. The system automatically splits a video into a sequence of shots, extracts a few representative frames (said r-frames) from each shot and computes r-frame descriptors based on color, texture and motion. Queries based on one or more features are possible. Very interesting results obtained during the severe tests the system was subjected to are reported and discussed.

Proceedings of SPIE, 1997

In this paper videos are analyzed to get a content-based decription of the video. The structure of a given video is useful to index long videos efficiently and automatically. A comparison between shots gives an overview about cut frequency, cut pattern, and scene bounds. After a shot detection the shots are grouped into clusters based on their visual similarity. A time-constraint clustering procedure is used to compare only those shots that are positioned inside a time range. Shots from different areas of the video (e.g., begin/end) are not compared. With this cluster information that contains a list about shots and their clusters it is possible to calculate scene bounds. A labeling of all clusters gives a declaration about the cut pattern. It is easy now to distinguish a dialogue from an action scene. The final content analysis is done by the ImageMiner * system. The ImageMiner system developed at the University of Bremen of the Image Processing Department of the Center for Computing Technology realizes content-based image retrieval for still images through a novel combination of methods and techniques of computer vision and artifical intelligence. The ImageMiner system consists of three analysis modules for computer vision, namely for color, texture, and contour analysis. Additionally exists a module for object recognition. The output of the object recognition module can be indexed by a text retrieval system. Thus, concepts like forestscene may be searched for. We combine the still image analysis with the results of the video analysis in order to retrieve shots or scenes.

Nowadays visual information is gaining much more importance than textual information as pictures convey more than words. As image has richer content than text, Video database is increasing exponentially over the Internet. Content-based Video retrieval (CBVR) from large video storage has become a challenging and important issue for researchers over worldwide. There is lack of tools so it is difficult to distinguish between low and high level feature. There has been lot of CBVR systems developed but still there is a lack of efficient retrieval system, as many retrieval techniques are not suitable for extensive search and so it deteriorates the overall performance of system. In this paper we have given an outline of the sequential steps of existing CBVR systems.

J. Inf. Data Manag., 2010

This work is concerned with video indexing and retrieval based on visual features. It puts forth an approach for the automatic summary and indexing of digital videos in order to support queries based on visual content within the indexed video's repository. The proposed approach was applied to a database containing more than 34 hours of broadcast news videos. Visual features extracted from the summarized version of the videos were then used for video content indexing. That provided us with the basis for various experiments and analysis on the retrieval of visual content with the application of various techniques implemented in this work. The approach proposes a method for key frame extraction that summarizes video content in a static storyboard, specifically projectec, for key frame retrieval and video access. Thus, the selected key frames are processed in order to extract statistical features as well as wavelet coefficients to represent the video's essence in a very short am...

Lecture Notes in Computer Science, 2005

Object movie refers to a set of images captured from different perspectives around a 3D object. Object movie is a good representation of a physical object because it can provide 3D interactive viewing effect, but does not require 3D reconstruction. In order to retrieve the desired object movie from the database, we first map an object movie into a manifold in the feature space. Two different sets of feature descriptors, one dense and one condensed, are designed to sample the manifold. Based on these descriptors, we define the dissimilarity measure between the query and the target in the object movie database. The query we considered can be either a complete object movie or simply a subset of views. In this paper, we further propose a relevance feedback approach to improving retrieved results. Some experimental results are shown to show the potential of our approach.