I3S Laboratory - UMR UNS CNRS 7271

In this paper, we present first results obtained in the frame of the EROS-3D project, which aims at dealing with a collection of artwork 3D models, i.e. visualize them, classify them and compare them. Some 3D descriptors are used, in association with our active learning search engine RETIN. 3D features are described as well as our new system of classification and retrieval of objects, which we called RETIN-3D.

This paper deals with video segmentation for MPEG-4 and MPEG-7 applications. Region-based active contour is a powerful technique for segmentation. However most of these methods are implemented using level sets. Although level-set methods provide accurate segmentation, they suffer from large computational cost. We propose to use a regular B-spline parametric method to provide a fast and accurate segmentation. Our B-spline interpolation is based on a fixed number of points 2 j depending on the level of the desired details. Through this spatial multiresolution approach, the computational cost of the segmentation is reduced. We introduce a length penalty. This results in improving both smoothness and accuracy. Then we show some experiments on real-video sequences.

In this paper, we present the first participation of a consortium of French laboratories, IRIM, to the TRECVID 2008 BBC Rushes Summarization task. Our approach resorts to video skimming. We propose two methods to reduce redundancy, as rushes include several takes of scenes. We also take into account low and mid-level semantic features in an ad-hoc fusion method in order to retain only significant content

In most of video shot boundary detection algorithms, proposed in the literature, several parameters and thresholds have to be set in order to achieve good results. In this paper, to get rid of parameters and thresholds, we explore a supervised classification method for video shot segmentation. We transform the temporal segmentation into a class categorization issue. Our approach defines a uniform framework for combining different kinds of features extracted from the video. Our method does not require any pre-processing step to compensate motion or post-processing filtering to eliminate false detected transitions. The experiments, following strictly the TRECVID 2002 competition protocol, provide very good results dealing with a large amount of features thanks to our kernel-based SVM classification method

Video shot boundary detection plays an important role in video processing. It is the first step toward video-content analysis and content-based video retrieval. We develop a hierarchical approach for shot boundary detection based on the assumption that hierarchy helps to take decisions by reducing the amount of indeterminate transitions. Our method consists in first detecting abrupt transitions using a learning-based approach, then non-abrupt transitions are split into gradual transitions and normal frames. We describe in this paper, a machine learning system for shot boundary detection. The core of this system is a kernel-based SVM classifier. We present some results obtained for shot extraction TRECVID 2006 Task.

The actual generation of video search engines offers low-level abstractions of the data while users seek for high-level semantics. The main challenge in video retrieval remains bridging the semantic gap. Thus, the effectiveness of video retrieval is based on the result of the interaction between query selection and a goal-oriented human user. The system exploits the human capability for rapidly scanning imagery augmenting it with an active learning loop, which tries to always present the most relevant material based on the current information. We describe in this paper, a machine learning system for interactive video retrieval. The core of this system is a kernel-based SVM classifier. The video retrieval uses the core as an active learning classifier. We perform an experiment against the 2005 NIST TRECVID benchmark in the high-level task.

This paper deals with fast image and video segmentation using active contours. Region-based active contours using level sets are powerful techniques for video segmentation, but they suffer from large computational cost. A parametric active contour method based on B-Spline interpolation has been proposed in [26] to highly reduce the computational cost, but this method is sensitive to noise. Here, we choose to relax the rigid interpolation constraint in order to robustify our method in the presence of noise: by using smoothing splines, we trade a tunable amount of interpolation error for a smoother spline curve. We show by experiments on natural sequences that this new flexibility yields segmentation results of higher quality at no additional computational cost. Hence, real-time processing for moving objects segmentation is preserved.

In the past ten years, new powerful algorithms based on efficient data structures have been proposed to solve the problem of Nearest Neighbors search (or Approximate Nearest Neighbors search). If the Euclidean Locality Sensitive Hashing algorithm which provides approximate nearest neighbors in a Euclidean space with sub-linear complexity is probably the most popular, the Euclidean metric does not always provide as accurate and as relevant results when considering similarity measure as the Earth-Mover Distance and χ 2 -distance. In this paper, we present a new LSH scheme adapted to χ 2 -distance for approximate nearest neighbors search in high-dimensional spaces. We define the specific hashing functions, we prove their local-sensitivity and compare, through experiments, our method with the Euclidean Locality Sensitive Hashing algorithm in the context of image retrieval on real image databases. The results prove the relevance of such a new LSH scheme either providing far better accuracy in the context of image retrieval than Euclidean scheme for an equivalent speed, or providing an equivalent accuracy but with a high gain in terms of processing speed.

This paper presents our approach to select relevant sequences from raw videos in order to generate summaries to Trecvid 2008 BBC Rush Task. Our system is composed of two major steps: First, the system detects "semantic" shot boundaries and keeps only non-redundant shots; then, the system estimates average motion for each shot, as a criterion of amount of information, to better share out the duration of the summary between remaining shots. The first step is based on a fast near-duplicate retrieval using Locality Sensitive Hashing (LSH) which provides results in few seconds (if we do not take into account decoding and encoding processes). The evaluation of Trecvid shows very promising results, since we ranked 17 th over 43 runs, regarding redundancy measure (RE), and 18 th for object and event inclusion (IN). These balanced results (most of best teams for the first criterion are among the latest for the second one) show that our method offers a quite good trade-off between false negatives (IN) and false positives (RE).

In content based image retrieval, the success of any distance-based indexing scheme depends critically on the quality of the chosen distance metric. We propose in this paper a kernel-based similarity approach working on sets of vectors to represent images. We introduce a method for fast approximate similarity search in large image databases with our kernel-based similarity metric. We evaluate our algorithm on image retrieval task and show it to be accurate and faster than linear scanning.

In this article, we propose a new video object retrieval system. Our approach is based on a Spatio-Temporal data representation, a dedicated kernel design and a statistical learning toolbox for video object recognition and retrieval. Using state-of-the-art video object detection algorithms (for faces or cars, for example) we segment video object tracks from real movies video shots. We then extract, from these tracks, sets of spatio-temporally coherent features that we call Spatio-Temporal Tubes. To compare these complex tube objects, we design a Spatio-Temporal Tube Kernel (STTK) function. Based on this kernel similarity we present both supervised and active learning strategies embedded in Support Vector Machine framework. Additionally, we propose a multi-class classification framework dealing with unbalanced data. Our approach is successfully evaluated on two real movies databases, the french movie "L'esquive" and episodes from "Buffy, the Vampire Slayer" TV series. Our method is also tested on a car database (from real movies) and shows promising results for car identification task.

In content-based image retrieval context, a classic strategy consists in computing off-line a dictionary of visual features. This visual dictionary is then used to provide a new representation of the data which should ease any task of classification or retrieval. This strategy, based on past research works in text retrieval, is suitable for the context of batch learning, when a large training set can be built either by using a strong prior knowledge of data semantics (like for textual data) or with an expensive off-line pre-computation. Such an approach has major drawbacks in the context of interactive retrieval, where the user iteratively builds the training data set in a semi-supervised approach by providing positive and negative annotations to the system in the relevance feedback loop. The training set is thus built for each retrieval session without any prior knowledge about the concepts of interest for this session. We propose a completely different approach to build the dictionary on-line from features extracted in relevant images. We design the corresponding kernel function, which is learnt during the retrieval session. For each new label, the kernel function is updated with a complexity linear with respect to the size of the database. We propose an efficient active learning strategy for the weakly supervised retrieval method developed in this paper. Moreover this framework allows the combination of features of different types. Experiments are carried out on standard databases, and show that a small dictionary can be dynamically extracted from the features with better performances than a global one.

In the domain of multimedia, rapid DVD browsing or multimedia oriented web search require an efficient contentbased image and video retrieval system. In this paper, we present our retrieval system of actors in films combining powerful machine learning techniques with "kernels on bags of bags" design. From a film segmented into shots, we extract video-tubes of actor faces and represent these video objects with sets of temporally coherent features. These visual features are then input in the kernel-based SVM retrieval system. Our approach has been tested on retrieving actors in a real film and proved its efficiency.

In this paper, we present the first participation of a consortium of French laboratories, IRIM, to the TRECVID 2008 BBC Rushes Summarization task. Our approach resorts to video skimming. We propose two methods to reduce redundancy, as rushes include several takes of scenes. We also take into account low and mid-level semantic features in an ad-hoc fusion method in order to retain only significant content

The proliferative activity of breast tumors, which is routinely estimated by counting of mitotic figures in hematoxylin and eosin stained histology sections, is considered to be one of the most important prognostic markers. However, mitosis counting is laborious, subjective and may suffer from low inter-observer agreement. With the wider acceptance of whole slide images in pathology labs, automatic image analysis has been proposed as a potential solution for these issues.

Motion is one of the main characteristics that describe the semantic information of videos. In this work, a global video descriptor based on orientation tensors is proposed. This descriptor is obtained by combining polynomial coefficients calculated for each image in a video. The coefficients are found through the projection of the optical flow on Legendre polynomials, reducing the dimension of per frame motion estimations. The sequence of coefficients are then combined using orientation tensors. The global tensor descriptor created is evaluated by a classification of the KTH video database with a SVM classifier.

This demonstration presents a digital media retrieval system for searching large categories in different media databases. The core of our system is an interactive online classification based on user labeling. The classification is obtained with a statistical learning method: kernels for similarity representation and SVM (Support Vector Machine) using binary user annotations. RETIN applies also an active learning strategy for proposing documents to the user for labeling. The system can deal with images, 3D objects and videos and other media can be added to. A graphical user interface allows easy browsing of different media, simple and user-friendly interaction and fast retrieval.

In the past few years, street-level geoviewers has become a very popular web-application. In this paper, we focus on a first urban concept which has been identified as useful for indexing then retrieving a building or a location in a city: the windows. The work can be divided into three successive processes: first, object detection, then object characterization, finally similarity function design (kernel design). Contours seem intuitively relevant to hold architecture information from building facades. We first provide a robust window detector for our unconstrained data, present some results and compare our method with the reference one. Then, we represent objects by fragments of contours and a relational graph on these contour fragments. We design a kernel similarity function for structured sets of contours which will take into account the variations of contour orientation inside the structure set as well as spatial proximity. One difficulty to evaluate the relevance of our approach is that there is no reference database available. We made, thus, our own dataset. The results are quite encouraging regarding what was expected and what provide methods the literature.

Active contour methods are often methods of choice for demanding segmentation problems, yet segmentation of medical images with complex intensity patterns still remains a challenge for these methods. This paper proposes a method to incorporate interactively specified foreground/background regions into the active model framework while keeping the user interaction to the minimum. To achieve that, the proposed functional to be minimized includes a term to encourage active contour to separate the points close to the specified foreground region from the points close to the specified background region in terms of geodesic distance. The experiments on multi-modal prostate images demonstrate that the proposed method not only can achieve robust and accurate results, but also provides an efficient way to interactively improve the results.