Department of Electronic & Electrical Engineering

A common task in image editing is to change the colours of a picture to match the desired colour grade of another picture. Finding the correct colour mapping is tricky because it involves numerous interrelated operations, like balancing the colours, mixing the colour channels or adjusting the contrast. Recently, a number of automated tools have been proposed to find an adequate one-to-one colour mapping. The focus in this paper is on finding the best linear colour transformation. Linear transformations have been proposed in the literature but independently. The aim of this paper is thus to establish a common mathematical background to all these methods. Also, this paper proposes a novel transformation, which is derived from the Monge-Kantorovicth theory of mass transportation. The proposed solution is optimal in the sense that it minimises the amount of changes in the picture colours. It favourably compares theoretically and experimentally with other techniques for various images and under various colour spaces.

This article proposes an original method to estimate a continuous transformation that maps one N-dimensional distribution to another. The method is iterative, non-linear, and is shown to converge. Only 1D marginal distributions are used in the estimation process, hence involving low computation costs. As an illustration this mapping is applied to colour transfer between two images of different contents. The paper also serves as a central focal point for collecting together the research activity in this area and relating it to the important problem of Automated Colour Grading.

Contour following is a standard activity in rotoscoping in the digital post production domain. An artist might need to cut out or edit an object separately from its background and it is left to the artist to manually create the cut out. Techniques for automatically tracing the edges of the ...

This paper has two main contributions. The first is a Bayesian framework for removing two common types of degradations on video known as blotches and line scratches. Most removal techniques assume complete obliteration of the original data at the corrupted sites. This often leads to the introduction of restoration artifacts during removal. Our framework is based on modeling corruption as a semi-transparent layer. This model was introduced earlier by Ahmed et al. (ICIP 2009) for the problem of blotch removal. We show much more blotch removal results than the previous work, and we extend the semi-transparent corruption model to the problem of line removal. The second contribution of this paper is an automated technique for ground-truth generation from infrared scans of corruptions. Previous ground-truth generation efforts require manually inpainting the corrupted regions. The restoration results are evaluated by comparing the reconstructed data against the ground-truth estimates. Comparisons with current blotch and line removal techniques show that the proposed corruption removal framework produces better removal and generates less restoration artifacts.

This paper presents a probabilistic framework for off-line multiple object tracking. At each timestep, a small set of deterministic candidates is generated which is guaranteed to contain the correct solution. Tracking an object within video then becomes possible using the Viterbi algorithm. In contrast with particle filter methods where candidates are numerous and random, the proposed algorithm involves a few candidates and results in a deterministic solution. Moreover, we consider here off-line applications where past and future information is exploited. This paper shows that, although basic and very simple, this candidate selection allows the solution of many tracking problems in different real-world applications and offers a good alternative to particle filter methods for off-line applications.

Motion-based video segmentation has been studied for many years and remains challenging. Ill-posed problems must be solved when seeking for a fully automated solution, so it is increasingly popular to maintain users in the processing loop by letting them set parameters or draw mattes to guide the segmentation process. When processing multiple-view videos, however, the amount of user interaction should not be proportional to the number of views. In this paper we present a novel sparse segmentation algorithm for twoview stereoscopic videos that maintains temporal coherence and view consistency throughout. We track feature points on both views with a generic tracker and analyse the pairwise affinity of both temporally overlapping and disjoint tracks, whereas existing similar techniques only exploit the information available when tracks overlap. The use of stereo-disparity also allows our technique to process jointly feature tracks on both views, exhibiting a good view consistency in the segmentation output. To make up for the lack of high level understanding inherent to segmentation techniques, we allow the user to refine the output with a split-and-merge approach so as to obtain a desired view-consistent segmentation output over many frames in a few clicks. We present several real video examples to illustrate the versatility of our technique.

Temporal random variation of luminance in images can manifest in film and video due to a wide variety of sources. Typical in archived films, it also affects scenes recorded simultaneously with different cameras (e.g. for film special effect), and scenes affected by illumination problems. Many applications in Computer Vision and Image Processing that try to match images (e.g. for motion estimation, stereo vision, etc.) have to cope with this problem. The success of current techniques for dealing with this is limited by the non-linearity of severe distortion, the presence of motion and missing data (yielding outliers in the estimation process) and the lack of fast implementations in reconfigurable systems. This paper proposes a new process for stabilizing brightness fluctuations that improves the existing models. The article also introduces a new estimation method able to cope with outliers in the joint distribution of pairs images. The system implementation is based on the novel use of general purpose PC graphics hardware. The overall system presented here is able to deal with much more severe distortion than previously was the case, and in addition can operate at 7 fps on a 1.6GHz PC with broadcast standard definition images. 1

High efficiency video coding has made it possible to stream video over bandwidth constrained communication networks. Depending on bit rate requirements, a video encoder sacrifices some image details which can then introduce visual artefacts. Due to aggressive encoding a contouring staircase artefact called banding can be observed in image regions with very low texture. This paper presents a solution for removing banding artefacts using image filtering and dithering techniques. A new banding index (BI) metric is also presented for quantitatively measuring the amount of banding in an image. Using this BI metric, we assess how much banding YouTube video encoding introduces in a video test dataset. There is a debanding filter in ffmpeg called gradfun. We compare the results of our debanding technique with those of gradfun on the YouTube test dataset.

This paper introduces a new database of 25 recto/verso image pairs from documents suffering from bleed-through degradation, together with manually created foreground text masks. The structure and creation of the database is described, and three bleed-through restoration methods are compared in two ways; visually, and quantitatively using the ground truth masks.

This paper presents a new expression of the relationship between Integral Projections and motion in an image pair. The resulting new multiresolution gradient based approach is used to estimate dominant motion in image sequences degraded by random shake. The paper also describes an implementation using the GPU as a coprocessor for the CPU that allows, for the first time, real time video stabilisation in software on broadcast standard definition television images.

This paper examines the accuracy of audio-video based localization using multiple cameras and multi-microphones. Covariance mapping theory is used to determine the accuracy of audio and video based localization. Both modalities are compared in terms of their ability to provide accurate location estimates of a moving audio-visual source. Relatively, video is found to be significantly more accurate than audio. The problem of audio-video fusion is also examined. The fusion of audio and video location estimates is applied in the audio domain, the video domain and the positional domain. The accuracy of these three fusion strategies for 3D localization are examined from a theoretical basis. The best localization performance is found when fusion is applied in the positional domain. Fusing audio and video data in the video domain is found to exhibit the worst localization performance. This analysis is confirmed by measuring the accuracy of each fusion strategy in localizing a moving audio-visual source.

We study the predictive capabilities of magnetic feature properties (MF) generated by Solar Monitor Active Region Tracker (SMART) Higgins et al.(Adv. Space Res. 47, 2105, 2011) for solar flare forecasting from two datasets: the full dataset of SMART detections from 1996 to 2010 that has been previously studied by Ahmed et al. (Solar Phys. 283(1), 179, 2011) and a subset of that dataset which only includes detections that are NOAA active regions (ARs). Main contributions: we use marginal relevance as a filter feature selection method to identify most useful SMART MF properties for separating flaring from nonflaring detections and logistic regression to derive classification rules to predict future observations. For comparison, we employ a Random Forest, Support Vector Machine and a set of Deep Neural Network models, as well as Lasso for feature selection. Using the linear model with three features we obtain significantly better results (TSS=0.84) to those reported by Ahmed et al. (Solar Phys. 283(1), 179, 2011) for the full dataset of SMART detections. The same model produced competitive results (TSS=0.67) for the dataset of SMART detections that are NOAA ARs which can be compared to a broader section of flare forecasting literature. We show that more complex models are not required for this data.

This article proposes an original method for grading the colours between different images or shots. The first stage of the method is to find a one-to-one colour mapping that transfers the palette of an example target picture to the original picture. This is performed using an original and parameter free algorithm that is able to transform any N-dimensional probability density function into another one. The proposed algorithm is iterative, non-linear and has a low computational cost. Applying the colour mapping on the original picture allows reproducing the same 'feel' as the target picture, but can also increase the graininess of the original picture, especially if the colour dynamic of the two pictures is very different. The second stage of the method is to reduce this grain artefact through an efficient post-processing algorithm that intends to preserve the gradient field of the original picture.

Brightness fluctuations in sequences are a typical impairment in archive and modern footage. These fluctuations are mainly temporal but they may also present spatial variations across each individual frame. Current deflicker techniques consider that these variations are smooth but this paper shows that the flicker localisation can be very fine in some cases. This paper presents a deflicker algorithm that is able to handle within a same framework both very smooth spatial variations and very localised variations. The core of the method lies in the parametrisation of the flicker model with only one parameter per pixel. The viability of using such a fine parametrisation is made possible by employing a temporal filtering that effectively separates occlusions from the flicker estimation.

Content based analysis has traditionally been posed in the context of identifying some material in response to a user query. This paper illustrates that given a content based analysis process that can identify semantic events in a sequence, that sequence can then be changed in various ways. A Motion Keyframe is presented to re-express the viewing of a sequence. The notion of content analysis for control of other media processing engines is introduced. Tennis footage is used to illustrate the ideas since sports in general contains strong contextual information.

This article proposes an original method for grading the colours between different images or shots. The first stage of the method is to find a one-to-one colour mapping that transfers the palette of an example target picture to the original picture. This is performed using an original and parameter free algorithm that is able to transform any N-dimensional probability density function into another one. The proposed algorithm is iterative, non-linear and has a low computational cost. Applying the colour mapping on the original picture allows reproducing the same 'feel' as the target picture, but can also increase the graininess of the original picture, especially if the colour dynamic of the two pictures is very different. The second stage of the method is to reduce this grain artefact through an efficient post-processing algorithm that intends to preserve the gradient field of the original picture.

Large motion displacements in image sequences are still a problem for most motion estimation techniques. Progress in feature matching allows to establish robust correspondences between images for a sparse set of points. Recent works have attempted to use this sparse information to guide the dense motion field estimation. We propose to achieve this in an extended motion estimation framework, which integrates information about the geodesic distance to the sparse features. Results show that by considering a handful of these feature matches, the geodesic distance is able to propagate the information efficiently.

Reflections in image sequences consist of several layers superimposed over each other. This phenomenon causes many image processing techniques to fail as they assume the presence of only one layer at each examined site e.g. motion estimation and object recognition. This work presents an automated technique for detecting reflections in image sequences by analyzing motion trajectories of feature points. It models reflection as regions containing two different layers moving over each other. We present a strong detector based on combining a set of weak detectors. We use novel priors, generate sparse and dense detection maps and our results show high detection rate with rejection to pathological motion and occlusion.