Video Segmentation via Temporal Pattern Classification

HAL (Le Centre pour la Communication Scientifique Directe), 2003

Lecture Notes in Computer Science, 2003

We present a shot segmentation method based on the representation of visual contents in video using a coupled Markov chains approach. This representation allows us to combine different image features and to keep information about all the images since the beginning of the shot, instead of simply comparing adjacent frames. We also define an adaptative detection threshold that depends on the distance measures that are obtained, instead of trying to find a fixed threshold. Results show that the combination of color and motion image features in the same representation provides a more robust detection of shot boundaries than using each feature separately.

—Video shot boundary detection (SBD), is the process of segmenting a video sequence into smaller temporal units, dubbed shots. SBD is the primary step for any further video analyses. In this article we propose a new and effective SBD method for detecting cut transitions (CTs) and gradual transitions (GTs) of a video sequence, in a unified scheme. As the first step in our scheme, we introduce a significantly effective candidate segment selection method by use of frame histogram, and a locally-defined adaptive threshold. This step avoids the processing of unnecessary video segments in the subsequent stages and accelerate the whole SBD process. Next, we extract CTs from candidate segments by use of a robust multi-stage thresholding. Then, we pass the remained segments through a novel candidate segment adjustment stage, which is realized by use of K-means classifiers, and its purpose is to give a better initial prediction of the location of GTs. Afterwards, we use some measures of difference along with singular value decomposition (SVD) on histograms of frames in each candidate GT segment, to obtain discriminating feature vectors with desired dimensions. Features are then fed into linear support vector machine (SVM) classifiers to separate segments with GTs from the rest. Finally, we determine the boundary of GTs by evaluating the gradient of a metric associated to each GT segment. By use of gradient for determining the range of GTs, we avoid the iterative procedures that are usually used in this purpose. Experimental results on standard videos, show the capability and supremacy of our method in comparison with state-of-the-art SBD procedures.

International Journal of Signal Processing, Image Processing and Pattern Recognition, 2017

Video content retrieval just like information retrieval requires some pre-processing such as indexing, key-frame selection and most importantly accurate video shot boundary detection. Accurate detection of video shots give way for video information to be stored in a manner that will allow easy access. Several algorithms have been developed in this field of study and tested even at the TRECVID 2002, 2005 and 2007 tasks evaluation conferences. Challenges on accurate detection of these different types of video transitions have always been from large object and camera motions as well as fast zooming, flashlights, and change in luminance. These attributes differ from one video sequence to another and features of one video sequence cannot always match with features of another video. Therefore, in our work we use a video specific machine learning approach that leverages information from several shot boundary detection algorithms in order to improve the detection of the shot boundaries on a video sequence. Our results suggest that a classifier built from a combination of block-based motion estimation, RGB histogram based block-based cross-correlation coefficient and RGB histogram based sum of squared difference provided better results with an average F1 score of 0.752 on shot boundary detection for the seventeen videos tested. This proves that a combination of luminance and motion based algorithms improves the detection of video shot boundaries. We also found that the detection of shot cuts and gradual transitions can be improved by using features generated by several shot boundary detection algorithms.

2014

Video segmentation is fundamental to a number of applications related to video retrieval and analysis. To realize the content based video retrieval, the video information should be organized to elaborate the structure of the video. The segmentation video into shot is an important step to make. This paper presents a new method of shot boundaries detection based on motion activities in video sequence. The proposed algorithm is tested on the various video types and the experimental results show that our algorithm is effective and reliably detects shot boundaries

IEEE Access, 2017

One of the essential pre-processing steps of semantic video analysis is the video shot boundary detection (SBD). It is the primary step to segment the sequence of video frames into shots. Many SBD systems using supervised learning have been proposed for years; however, the training process still remains its principal limitation. In this paper, a multi-modal visual features-based SBD framework is employed that aims to analyze the behaviors of visual representation in terms of the discontinuity signal. We adopt a candidate segment selection that performs without the threshold calculation but uses the cumulative moving average of the discontinuity signal to identify the position of shot boundaries and neglect the non-boundary video frames. The transition detection is structurally performed to distinguish candidate segment into a cut transition and a gradual transition, including fade in/out and logo occurrence. Experimental results are evaluated using the golf video clips and the TREC2001 documentary video data set. Results show that the proposed SBD framework can achieve good accuracy in both types of video data set compared with other proposed SBD methods.

Proceedings of the TREC Video Retrieval …, 2006

In this study we described our participation in the NIST TRECVID 2006 evaluation for Shot Boundary Determination task with 10 submissions. We have used a method based on histogram differences for cut and gradual transition detection. Both the details of our approach used in the SBD task submissions and effects of using different threshold & skip frame intervals on evaluation results are presented.

ArXiv, 2014

With ever increasing computing power and data storage capacity, the potential for large digital video libraries is growing rapidly.However, the massive use of video for the moment is limited by its opaque characteristics. Indeed, a user who has to handle and retrieve sequentially needs too much time in order to find out segments of interest within a video. Therefore, providing an environment both convenient and efficient for video storing and retrieval, especially for content-based searching as this exists in traditional textbased database systems, has been the focus of recent and important efforts of a large research community In this paper, we propose a new automatic video scene segmentation method that explores two main video features; these are spatial-temporal relationship and rhythm of shots. The experimental evidence we obtained from a 80 minutevideo showed that our prototype provides very high accuracy for video segmentation.

2006 2nd International Conference on Information & Communication Technologies, 2006

The development in multimedia applications through the Internet has opened a new field of research in storing, handling, and retrieving digital videos. Video classification and segmentation are fundamental steps for efficient accessing, retrieving, browsing and compressing large amount of video data. The basic operation in analysis of heterogeneous video data is to design system that can accurately and automatically segments video material into shots and scenes. This paper presents a new detailed video segmentation technique based on evaluation of a shot cut-type detector based on a color histogram by using a dual precession technique for adjacent frames similarity analysis. This technique produced a high recall and precision results. 0-7803-9521-2

Segmentation of a video into its constituent shots is a fundamental task for indexing and analysis in content based video retrieval systems. In this paper, a novel approach is presented for accurately detecting the shot boundaries in real time video streams, without any a priori knowledge about the content or type of the video. The edges of objects in a video frame are detected using a spatio-temporal fuzzy hostility index. These edges are treated as features of the frame. The correlation between the features is computed for successive incoming frames of the video. The mean and standard deviation of the correlation values obtained are updated as new video frames are streamed in. This is done to dynamically set the threshold value using the three-sigma rule for detecting the shot boundary (abrupt transition). A look back mechanism forms an important part of the proposed algorithm to detect any missed hard cuts, especially during the start of the video. The proposed method is shown to be applicable for online video analysis and summarization systems. In an experimental evaluation on a heterogeneous test set, consisting of videos from sports, movie songs and music albums, the proposed method achieves 99.24% recall and 99.35% precision on the average.