Temporal Video Segmentation

SPIE Proceedings, 2003

Video segmentation is fundamental to a number of applications related to video retrieval and analysis. Shot change detection is the initial step of video segmentation and indexing. There are two basic types of shot changes. One is the abrupt change or cut, and the other is the gradual shot transition. The smooth variations of the video feature values in a gradual transition produced by the editing effects are often confused with those caused by camera or object motions. To overcome this difficulty, it is reasonable to estimate the motions and suppress the disturbance caused by them. In this thesis, we explore the possibility to exploit motion and illumination estimation in a video sequence to detect both abrupt and gradual shot changes. A generalized optical flow constraint that includes an illumination parameter to model local illumination changes is employed in the motion and illumination estimation. An iterative process is used to refine the generalized optical flow constraints step by step. A robust measure that is the likelihood ratio of the corresponding motion-compensated blocks in the consecutive frames is used for detecting abrupt changes. For the detection of gradual shot transitions, we compute the average monotony of intensity variations on the stable pixels in the images in a twin-comparison framework. We test the proposed algorithm on a number of video sequences in TREC 2001 and compare the detection results with the best results reported in the TREC 2001 benchmark. The comparisons indicate that the proposed shot change detection algorithm is competitive against the best existing algorithms.

2014

Temporal segmentation of videos into meaningful image sequences containing some particular activities is an interesting problem in computer vision. We present a novel algorithm to achieve this semantic video segmentation. The segmentation task is accomplished through event detection in a frame-by-frame processing setup. We propose using one-class classification (OCC) techniques to detect events that indicate a new segment, since they have been proved to be successful in object classification and they allow for unsupervised event detection in a natural way. Various OCC schemes have been tested and compared, and additionally, an approach based on the temporal self-similarity maps (TSSMs) is also presented. The testing was done on a challenging publicly available thermal video data-set. The results are promising and show the suitability of our approaches for the task of temporal video segmentation.

Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, 2000

Automatically building high-level segments to structure information in video documents is a challenging task. This paper presents a method based on the cophenetic criterion, a distance between clustered shots which detects breaks between sequences. It describes and compares various implemented options. Experiments have proved that the proposed criterion can be used for achieving segmentation.

IEEE Transactions on Multimedia, 2000

We present a general approach to temporal media segmentation using supervised classification. Given standard low-level features representing each time sample, we build intermediate features via pairwise similarity. The intermediate features comprehensively characterize local temporal structure, and are input to an efficient supervised classifier to identify shot boundaries. We integrate discriminative feature selection based on mutual information to enhance performance and reduce processing requirements. Experimental results using large-scale test sets provided by the TRECVID evaluations for abrupt and gradual shot boundary detection are presented, demonstrating excellent performance.

2014

Video segmentation is fundamental to a number of applications related to video retrieval and analysis. To realize the content based video retrieval, the video information should be organized to elaborate the structure of the video. The segmentation video into shot is an important step to make. This paper presents a new method of shot boundaries detection based on motion activities in video sequence. The proposed algorithm is tested on the various video types and the experimental results show that our algorithm is effective and reliably detects shot boundaries

2017

With the recent advancement in multimedia technologies, in conjunction with the rapid increase of the volume of digital video data and the growth of internet ; it has becom mandatory to have the hability browse and search through information stored in large multimedia databases. For this purpose, content based video retrieval (CBVR) has become an active area of research durinf the last decade. The objective of this thesis is to present applications for temporal video segmentation and video retrieval based on different mathematical models. A shot is considered as the elementary unit of a video, and is defined as a continuous sequence of frames taken from a single camera, representing an action during time. The different types of transitions that may occur in a video sequence are categorized into : abrupt and gradual transition. In this work, through statistical analysis, we segment a video into its constituent units. This is achieved by identifying transitions between adjacent shots....

2003

In this paper, we explore supervised classification methods for video shot segmentation. We transform the temporal segmentation problem into a multi-class categorization issue. This approach provides a uniform framework for using different kinds of features extracted from the video and for detecting various types of shot boundaries. The approach utilizes manual labeled training data and a simple classification structure, which eliminates arbitrary thresholds and achieves more reliable estimation than previous threshold-based methods. Contrastive experiments on 13 videos (~4 hours) show excellent performance on the 2001 TREC Video Track Shot Classification Task in terms of precision and recall.

1999

The application of image processing techniques to achieve substantial compression in digital video is one of the reasons why computersupported video processing and digital TV are now becoming commonplace. The encoding formats used for video, such as the MPEG family of standards, have been developed primarily to achieve high compression rates, but now that this has been achieved, effort is being concentrated on other, content-based activities. MPEG-7, for example is a standard intended to support such developments. In the work described here, we are developing and deploying techniques to support content-based navigation and browsing through digital video (broadcast TV) archives. Fundamental to this is being able to automatically structure video into shots and scenes. In this paper we report our progress on developing a variety of approaches to automatic shot boundary detection in MPEG-1 video, and their evaluation on a large test suite of 8 hours of broadcast TV. Our work to date indicates that different techniques work well for different shot transition types and that a combination of techniques may yield the most accurate segmentation.

Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint

In this paper, a system called PixSO (Pixel-level comparison, Segmentation and Object Tracking) is presented for effective shot change detection using an unsupervised object segmentation algorithm and the technique of object tracking based on the segmentation mask maps. The detection method was tested on TV news, commercial, sports and documentary video sequences which contain different types of shots having different object and camera motions. The Our results have shown that the PixSO system can not only produce accurate shot change detection, but also obtain object level information of the video frames, which is very useful for video content indexing and analysis in multimedia databases.

Pattern Recognition, 2001

In this paper, we present an e$cient video parsing method for automating content-based video indexing and retrieval using shot boundary detection and camera operation analysis techniques. In the shot boundary detection, the local color information is used in order to eliminate the false detection caused by an abrupt change of illumination such as camera #ash or thunder. In order to reduce the computation time in the shot boundary detection, an adaptive time window is applied to this procedure. Local spatio-temporal images and multilayer perceptron are used for analyzing camera operations. The proposed method uses a learning algorithm with spatio-temporal information in the frame and does not process the entire video image to reduce the processing time. In order to verify the performance of the proposed automatic video parsing method, experiments have been carried out with a video database that includes news, documentary and movie. Experimental results demonstrate the e$ciency of the proposed video parsing technique.