Consensus-based cross-correlation

Vision Interface, 1995

2005

In this work, we present a formalization of the video synchronization problem that exposes new variants of the problem that have been left unexplored to date. We also present a novel method to temporally synchronize multiple stationary video cameras with overlapping views that: 1) does not rely on certain scene properties, 2) suffices for all variants of the synchronization problem exposed by the theoretical disseration, and 3) does not rely on the trajectory correspondence problem to be solved apriori. The method uses a two stage approach that first approximates the synchronization by tracking moving objects and identifying inflection points. The method then proceeds to refine the estimate using a consensus based matching heuristic to find moving features that best agree with the pre-computed camera geometries from stationary image features. By using the fundamental matrix and the trifocal tensor in the second refinement step we are able to improve the estimation of the first step and handle a broader range of input scenarios and camera conditions.

2014

Given the proliferation of consumer media recording devices, events often give rise to a large number of recordings. These recordings are taken from different spatial positions and do not have reliable timestamp information. In this paper, we present two robust graph-based approaches for synchronizing multiple audio signals. The graphs are constructed atop the over-determined system resulting from pairwise signal comparison using cross-correlation of audio features. The first approach uses a Minimum Spanning Tree (MST) technique, while the second uses Belief Propagation (BP) to solve the system. Both approaches can provide excellent solutions and robustness to pairwise outliers, however the MST approach is much less complex than BP. In addition, an experimental comparison of audio features-based synchronization shows that spectral flatness outperforms the zero-crossing rate and signal energy.

2010 IEEE International Conference on Image Processing, 2010

We present a novel approach for the temporal synchronization of multiple videos which is based on cross-correlating bit rate profiles. The proposed scheme determines the temporal offset without major restrictions on viewing angles, camera properties and camera motion. We propose two extensions of the basic algorithm which reduce the influence of camera motion and distracting background objects. Additionally, we describe how to optimally combine different bit rate components in order to further improve the reliability of the synchronization scheme. The proposed approach, when combined with the three extensions, leads to a reliable, robust, and frame accurate temporal alignment of videos at remarkably low complexity.

Image correspondence and registration techniques have gained popularity in recent times due to advancement of utilization in digital media and its storage. The main problem associated with image processing is when it is applied to fields like robotic vision and machine vision. The problem is due to clutter, i.e. the same frame with different objects has to be matched. Hence there has been need for efficient techniques of Image Registration. This led to development of feature extraction techniques and template matching techniques. The normalized cross correlation technique is one of them. A classical solution for matching two image patches is to use the cross-correlation coefficient. This works well if there is a lot of structure within the patches, but not so well if the patches are close to uniform. This means that some patches are matched with more confidence than others. By estimating this uncertainty more weight can be put on the confident matches than those that are more uncertain. All the simulations have been performed using MATLAB tool.

2005

Video sequence synchronization is often necessary for computer vision applications where multiple simultaneously recorded videos are processed. We present a coarse-to-fine approach to synchronizing two video sequences recorded at the same frame rate by stationary cameras with fixed internal parameters. At the coarse level, each sequence is broken into a set of sub-sequences, which are then matched. A voting scheme determines the range in which the sequences’ temporal offset lies. The fine synchronization step searches for the temporal offset by initially examining integer offsets, and then using the golden-section search to locate the offset to sub-frame accuracy. Our algorithm recovers the temporal offset of two sequences using the motion of a single moving object, is computationally efficient, and it does not require any stationary background points as reference points. We present results for synthetic data and real video sequences, with various degrees of temporal overlap.

IEEE Transactions on Communications, 2000

Frame synchronization is traditionally achieved by inserting a given sequence periodically into the data stream. Here, a new method of synchronization is introduced in which the synchronization sequence is not contiguous, but is interleaved with the data symbols. The properties of these distributed sequences are investigated, and requirements are specified for detectibility of these sequences when inserted in random data. Methods for constructing and searching for good sequences are presented. It is demonstrated that this new method reduces the probability of false synchronization over traditional methods in the presence of additive white Gaussian noise.

Lecture Notes in Computer Science

Strong temporal correlation between adjacent frames of a video signal has been successfully exploited in standard video compression algorithms. In this work, we show that the temporal correlation in a video signal can also be used for fast video to reference image alignment. To this end, we first divide the input video sequence into groups of pictures (GOPs). Then for each GOP, only one frame is completely correlated with the reference image, while for the remaining frames, upper and lower bounds on the correlation coefficient (ρ) are calculated. These newly proposed bounds are significantly tighter than the existing Cauchy-Schwartz inequality based bounds on ρ. These bounds are used to eliminate majority of the search locations and thus resulting in significant speedup, without effecting the value or location of the global maxima. In our experiments, up to 80% search locations are found to be eliminated and the speedup is up to five times the FFT based implementation and up to seven times the spatial domain techniques.

Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429), 2003

Recovery of dynamic scene properties from multiple videos usually requires the manipulation of synchronous (simultaneously captured) frames. This paper is concerned with the automated determination of this synchrony when the temporal alignment of sequences is unknown. A cost function characterising departure from synchrony is first evolved for the case in which two videos are generated by cameras that may be moving. A novel voting method is then presented for minimising the cost function in the case where the ratio of the cameras' frame rates is unknown. Experimental results indicate this relatively general approach holds promise.

2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX), 2014

Quality assessment in the streaming media industry has matured to the stage that it encompasses not only traditional notions of pixel and audio sample integrity but also file format consistency and the media consumption experience itself. Audio/Video synchronisation has already been established as an associated measure of media quality and is well known in video conferencing and movie streaming applications. This paper presents a new system for the assessment of audio and video synchronisation in a media file. The system incorporates the idea of learning features which are robust to coding artefacts to establish robust fingerprints for A/V Sync measurement. Results from large scale testing of 30,000 clips from YouTube show why measurement of A/V Sync is important for file based video repositories and highlights issues that can now be addressed quantitatively.