Video Information Retrieval

Counting and classifying blood cells is an important diagnostic tool in medicine. Support Vector Machines are increasingly popular and efficient and could replace artificial neural network systems. Here a method to classify blood cells is proposed using SVM. A set of statistics on images are implemented in C++. The MPEG-7 descriptors Scalable Color Descriptor, Color Structure Descriptor, Color Layout Descriptor and Homogeneous Texture Descriptor are extended in size and combined with textural features corresponding to textural properties perceived visually by humans. From a set of images of human blood cells these statistics are collected. A SVM is implemented and trained to classify the cell images. The cell images come from a CellaVision TM DM-96 machine which classify cells from images from microscopy. The output images and classification of the CellaVision TM machine is taken as ground truth, a truth that is 90-95% correct. The problem is divided in two -the primary and the simplified. The primary problem is to classify the same classes as the CellaVision TM machine. The simplified problem is to differ between the five most common types of white blood cells. An encouraging result is achieved in both cases -error rates of 10.8% and 3.1% -considering that the SVM is misled by the errors in ground truth. Conclusion is that further investigation of performance is worthwhile. Klassificering av cellbilder med hjälp av MPEG-7-inspirerade mått och support vector machines i cellmorfologi Sammanfattning-Att räkna och klassificera blodceller är ett viktigt diagnostiskt redskap inom läkarvetenskapen. Support Vector Machines är effektiva, ökar i popularitet och kan ersätta artificiella neurala nätverkssystem. Här föreslås en metod för att klassificera blodceller m.h.a. SVM. En mängd statistika på bilder implementeras i C++. De s.k. MPEG-7 descriptors Scalable Color Descriptor, Color Structure Descriptor, Color Layout Descriptor och Homogeneous Texture Descriptor utvidgas i storlek och kombineras med textur-mått motsvarande textur-egenskaper som uppfattas visuellt av människor. Från en mängd bilder av mänskliga blodceller samlas dessa mått. En SVM implementeras och tränas att klassificera cellbilderna. Cellbilderna kommer från en CellaVision TM DM-96 som klassificerar celler från mikroskoperade bilder. Bilderna och dess klasser från en CellaVision TM DM-96-maskin tas som facit, ett facit som är 90-95% korrekt. Problemet delas i två -det primära och det förenklade. Det primära problemet är att skilja mellan de klasser som CellaVision TM s maskin gör. Det förenklade problemet är att skilja mellan de fem vanligaste typerna av vita blodkroppar. Ett glädjande resultat uppnås i båda fallen -felfrekvenser om 10,8% och 3,1% -med tanke på att SVM missleddes av felen i det tagna facitet. Slutsatsen är att vidare studier angående prestanda är lönsamma.

Semantic video annotation using ontologies has received a large attention from the scientific community in the recent years. Ontologies are being regarded as an appropriate tool to bridge the semantic gap. In this paper we present an overview of the state-of-the-art of approaches and algorithms that exploit ontologies to perform semantic video annotation and present an approach to automatically learn rules describing high-level concepts. This approach exploits the domain knowledge embedded into an ontology to learn a set of rules for semantic video annotation. The proposed technique is an adaptation of the First Order Inductive Learner (FOIL) technique to the Semantic Web Rule Language (SWRL) standard: Experiments have been performed in two different video domains: i) the TRECVID 2005 broadcast news collection, to detect events related to airplanes, such as taxiing, flying, landing and taking off; ii) surveillance videos, to detect if a person enters or exits a specific area. The promising experimental performance demonstrates the effectiveness and flexibility of the proposed framework.

In this paper, we present an overview of a hybrid approach for event detection from video surveillance sequences that has been developed within the REGIMVid project. This system can be used to index and search the video sequence by the visual content. The platform provides moving object segmentation and tracking, High-level feature extraction and video event detection.We describe the architecture of the system as well as providing an overview of the descriptors supported to date. We then demonstrate the usefulness of the toolbox in the context of feature extraction, events learning and detection in large collection of video surveillance dataset.

Providing a semantic access to video data requires the development of concept detectors. However, semantic concepts detection is a hard task due to the large intra-class and the small inter-class variability of content. Moreover, semantic concepts co-occur together in various contexts and their occurrence may vary from one to another. Thus, it is interesting to exploit this knowledge in order to achieve satisfactory performances. In this paper we present a generic semantic video indexing scheme, called SVI_REGIMVid. It is based on three levels of analysis. The first level (level1) focuses on low-level processing such as video shot boundary/ key-frame detection, annotation tools, key-points detection and visual features extraction tools. The second level (level2) aims to build the semantic models for supervised learning of concepts/contexts. The third level (level3) enriches the semantic interpretation of concepts/ contexts by exploiting fuzzy knowledge. The obtained experimental results are promising for a semantic concept/context detection process. Keywords Semantic indexing. Concepts. Contexts. SVI_REGIMVid 1 Introduction Recent years have witnessed a rapid evolution of technology acquisition and diffusion of multimedia data. The web, television channels and video blogs have led to the explosion of audiovisual databases. This vast amount of audiovisual data currently represents the main challenge research studies on the automatic information processing particularly in the representation and indexing of video contents. The archive professionals need to index their digital video bases for an effective access to video contents.

In the literature, several image retrieval approaches that allow mapping between low-level features and high-level semantics have been proposed. Among these one can cite object recognition, ontologies, and relevance feedback. However, their main limitations concern their high dependence on reliable external resources (existing ontologies, learning sets, etc.) and lack of capacity to combine semantic and visual information and provide relevant results. This paper proposes a system aiming to improve image retrieval results. The proposed system is based on a pattern graph combining semantic and visual features. The idea is (1) to automatically build a modular ontology based on a learning step from textual corpus and terminological resource, (2) to organize visual features in a graph-based model where the combined module and graph represent a unique component called "pattern," and (3) to build a pattern graph. To this end our system has been implemented. The obtained experimental results show that the pattern graph that we propose enables an improvement of retrieval task.

In this paper, we present an overview of a hybrid approach for event detection from video surveillance sequences that has been developed within the REGIMVid project. This system can be used to index and search the video sequence by the visual content. The platform provides moving object segmentation and tracking, High-level feature extraction and video event detection.We describe the architecture of the system as well as providing an overview of the descriptors supported to date. We then demonstrate the usefulness of the toolbox in the context of feature extraction, events learning and detection in large collection of video surveillance dataset.

This paper aims to report the system we used in semantic indexing (SIN) at TRECVID 2013. We participated in all three defined tasks this year, including main semantic indexing, localization and paired task. For the main task our approach uses a 4-stages processing pipeline. Feature extraction, fusion, classification and re-ranking are main stages of our system. For localization tasks we employed Deformable Part-based model followed by a saliency detection method to refine the initial detection candidates and finally for the paired task we use a simple fusion technique to select the right score for the concepts among the keyframes of a video-shot.

The context-based concept fusion (CBCF) is increasingly used in video semantic indexing, which uses various relations among different concepts to refine the original detection results. In this paper, we present a CBCF method called Temporal-Spatial Node Balance algorithm (TSNB). This method is based on a physical model, in which the concepts are regard as nodes and the relations are regard as forces. Then all the spatial and temporal relations and the moving cost of the nodes will be balanced. This method is intuitive and observable to explain a concept how to influence others or be influenced by others. And it uses both the spatial and temporal information to describe the semantic structure of the video. We use TSNB algorithm on the datasets of TRECVid 2005-2010. The results show that this method outperforms all the existed works as we know. Besides, it is faster.

Learning concepts from examples presented in user's query and infer the other items that belong to this query is still a significant challenge for images retrieval systems. Existing models from cognitive science namely Bayesian models of generalization mainly focus on this challenge where they remarkably succeed at explaining how to generalize from few examples in a wide range of domains. However their success largely depends on the validity of examples. They require that each example is a good representative, which is not always the case in the context of images retrieval. In this paper, we will extend the Bayesian models of generalization to identify the appropriate level of generalization for a given query in the context of query by semantic example systems. Our model uses an ontology as the basis of its hypothesis space which allows us to take advantages of its semantic richness and inference capacity. Experimental study using the ImageNet benchmark verifies the efficiency of our model in comparison to the state-of-the-art models of generalization.

This paper presents a novel framework for matching video sequences using the spatiotemporal segmentation of videos. Instead of using appearance features for region correspondence across frames, we use interest point trajectories to generate video volumes. Point trajectories, which are generated using the SIFT operator, are clustered to form motion segments by analyzing their motion and spatial properties. The temporal correspondence between the estimated motion segments is then established based on most common SIFT correspondences. A two pass correspondence algorithm is used to handle splitting and merging regions. Spatiotemporal volumes are extracted using the consistently tracked motion segments. Next, a set of features including color, texture, motion, and SIFT descriptors are extracted to represent a volume. We employ an Earth Mover's Distance (EMD) based approach for the comparison of volume features. Given two videos, a bipartite graph is constructed by modeling the volumes as vertices and their similarities as edge weights. Maximum matching of this graph produces volume correspondences between the videos, and these volume matching scores are used to compute the final video matching score. Experiments for video retrieval were performed on a variety of videos obtained from different sources including BBC Motion Gallery and promising results were achieved. We present qualitative and quantitative analysis of retrieval along with a comparison with two baseline methods.

Many multimedia applications can benefit from techniques for adapting existing classifiers to data with different distributions. One example is cross-domain video concept detection which aims to adapt concept classifiers across various video domains. In this paper, we explore two key problems for classifier adaptation: (1) how to transform existing classifier(s) into an effective classifier for a new dataset that only has a limited number of labeled examples, and (2) how to select the best existing classifier(s) for adaptation. For the first problem, we propose Adaptive Support Vector Machines (A-SVMs) as a general method to adapt one or more existing classifiers of any type to the new dataset. It aims to learn the "delta function" between the original and adapted classifier using an objective function similar to SVMs. For the second problem, we estimate the performance of each existing classifier on the sparsely-labeled new dataset by analyzing its score distribution and other meta features, and select the classifiers with the best estimated performance. The proposed method outperforms several baseline and competing methods in terms of classification accuracy and efficiency in cross-domain concept detection in the TRECVID corpus.

Intelligent video analysis is a problem of great importance for applications such as surveillance and automatic annotation. We present, in this paper, a hybrid, knowledge-based approach for object recognition in video sequences. Objects are modelled, in the signal level, through the visual descriptors defined by MPEG-7, the ISO standard for description of audiovisual content and in the semantic level, through the semantic relations defined by MPEG-7. The method of video analysis is synopsised as follows: first, moving regions are extracted using an active contour technique. Second, visual descriptions of the moving regions are extracted and are matched with the ones defined for recognizable objects.

This paper describes the system we used for the main task of Semantic INdexing (SIN) at TRECVID 2015. Our system uses a fivestage processing pipeline including feature extraction, pooling, encoding, classification and reranking. We employed CNN-based representations, as well as Dense and Root SIFTs as features for our system. We also report results of our experiments with SentiBank features and data augmentation techniques that did not contribute to the performance of the final system. Our second run ‘Rostam’ achieved an infAP of 26.67% on the 30 concepts evaluated for SIN 2015.

Automatic semantic annotation of video events has received a large attention from the scientific community in the latest years, since event recognition is an important task in many applications. Events can be defined by spatio-temporal relations and properties of objects and entities, that change over time; some events can be described by a set of patterns. In this paper we present a framework for semantic video event annotation that exploits an ontology model, referred to as Pictorially Enriched Ontology, and ontology reasoning based on rules. The proposed ontology model includes: high-level concepts, concept properties and concept relations, used to define the semantic context of the examined domain; concept instances, with their visual descriptors, enrich the video semantic annotation. The ontology is defined using the Web Ontology Language (OWL) standard. Events are recognized using patterns defined using rules, that take into account high-level concepts and concept instances. In our approach we propose an adaptation of the First Order Inductive Learner (FOIL) technique to the Semantic Web Rule Language (SWRL) standard to learn rules. We validate our approach on the TRECVID 2005 broadcast news collection, to detect events related to airplanes, such as taxiing, flying, landing and taking off. The promising experimental performance demonstrates the effectiveness of the proposed framework.

Semantic video annotation using ontologies has received a large attention from the scientific community in the recent years. Ontologies are being regarded as an appropriate tool to bridge the semantic gap. In this paper we present an overview of the state-of-the-art of approaches and algorithms that exploit ontologies to perform semantic video annotation and present an approach to automatically learn rules describing high-level concepts. This approach exploits the domain knowledge embedded into an ontology to learn a set of rules for semantic video annotation. The proposed technique is an adaptation of the First Order Inductive Learner (FOIL) technique to the Semantic Web Rule Language (SWRL) standard: Experiments have been performed in two different video domains: i) the TRECVID 2005 broadcast news collection, to detect events related to airplanes, such as taxiing, flying, landing and taking off; ii) surveillance videos, to detect if a person enters or exits a specific area. The promising experimental performance demonstrates the effectiveness and flexibility of the proposed framework.

Our main goal in this study was to develop and validate an intelligent system for video event detection based on spatiotemporel features combining an auto-associative neural network models for feature reduction. Proposed system aims at high accuracy of event classification mainly with the use of an M-SVM model. The core of the system is the auto-associative neural network models which can reduce the size of feature vectors. The proposed model performance, evaluated an important data basis including seven events, was compared to other models found in the literature; it outperforms the other methods in terms of precision.

The multimedia storage is increasing day by day. Also the cost to store these multimedia data is very less. Lots of videos available in the video warehouseare in unstructured format. As per user requirement, it is difficult to retrieve the relevant videos from such a huge video storage. Nowadays it is important to make such unstructured multimedia data easily available with flexibility. In recent years, lots of research is going on to extract the semantic concepts from multimedia data.The main purpose of concept detection is to provide semantic concept based retrieval of multimedia content. In previous content-based image and video retrieval systems, the retrieval was based on querying by examples and measuring the similarity of the database objects (images, video shots) with low-level features automatically extracted from the objects. The low-level features are insufficient to explain the content properly at conceptual level.The semantic gap characterizes the difference between two...

Our main goal in this study was to develop and validate an intelligent system for video event detection based on spatiotemporel features combining an auto-associative neural network models for feature reduction. Proposed system aims at high accuracy of event classification mainly with the use of an M-SVM model. The core of the system is the auto-associative neural network models which can reduce the size of feature vectors. The proposed model performance, evaluated an important data basis including seven events, was compared to other models found in the literature; it outperforms the other methods in terms of precision.

In multimedia classification, the background is usually considered an unwanted part of input data and is often modeled only to be removed in later processing. Contrary to that, we believe that a background model (i.e., the scene in which the picture or video shot is taken) should be included as an essential feature for both indexing and followup content processing. Information about image background, however, is not usually the main target in the labeling process and the number of annotated samples is very limited. Therefore, we propose to use a combination of semi-supervised and active learning to improve the performance of our scene classifier, specifically a combination of self-training with uncertainty sampling. As a result, we utilize a combination of statistical features extractor, a feed-forward neural network and support vector machine classifier, which consistently achieves higher accuracy on less diverse data. With the proposed approach, we are currently able to achieve pr...

Content-Based Video Copy Detection (CBVCD) consists of detecting whether or not a video document is a copy of some known original and to retrieve the original video. CBVCD systems rely on two different tasks: Feature Extraction task, that calculates many representative descriptors for a video sequence, and Similarity Search task, that is the algorithm for finding videos in an indexed collection that match a query video. This work details a CBVCD approach based on a combination of global descriptors, an automatic weighting algorithm, a pivot-based index structure, an approximate similarity search, and a voting algorithm for copy localization. This approach is analyzed using MUSCLE-VCD-2007 corpus, and it was tested at the TRECVID 2010 evaluation together with other state-of-the-art CBVCD systems. The results show that this approach enables global descriptors to achieve competitive results and even outperforms systems based on combination of local descriptors and audio information. This approach has a potential of achieving even higher effectiveness due to its seamless ability of combining descriptors from different sources at the similarity search level.

Your article is protected by copyright and all rights are held exclusively by Springer Science +Business Media New York. This e-offprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to self-archive your work, please use the accepted author's version for posting to your own website or your institution's repository. You may further deposit the accepted author's version on a funder's repository at a funder's request, provided it is not made publicly available until 12 months after publication.

This paper deals with research in the area of automatic extraction of textual and non-textual information and their classification. The main idea is to create a robust method for extraction of image and textual segments to obtain short web document. Thus, developed method consist of two data types extractions, where both image and text data extraction are using Document Object Model tree. Extracted objects are saved in separate databases followed the images analysis that define and describe image object from semantic point of view. Moreover, the semantic description of all modal objects are utilized to short web document creation. To accurate object classification, the fast and powerful hybrid segmentation algorithm based on Mean Shift and Believe Propagation principles are mentioned in this paper, too. Likewise, the image segmentation algorithm was integrated with SIFT descriptor. Finally, in order to obtain a semantic description of objects in static image, the SVM classification is used. The developed method was tested on real unsegmented and segmented images, too.

In content based video retrieval videos are often indexed with semantic labels (concepts) using pre-trained classifiers. These pre-trained classifiers (concept detectors), are not perfect, and thus the labels are noisy. Additionally, the amount of pre-trained classifiers is limited. Often automatic methods cannot represent the query adequately in terms of the concepts available. This problem is also apparent in the retrieval of events, such as bike trick or birthday party. Our solution is to obtain user feedback. This user feedback can be provided on two levels: concept level and video level. We introduce the method Adaptive Relevance Feedback (ARF) on video level feedback. ARF is based on the classical Rocchio relevance feedback method from Information Retrieval. Furthermore, we explore Maaike de Boer

The objective of this work is to visually search large-scale video datasets for semantic entities specified by a text query. The paradigm we explore is constructing visual models for such semantic entities on-the-fly, i.e. at run time, by using an image search engine to source visual training data for the text query. The approach combines fast and accurate learning and retrieval, and enables videos to be returned within seconds of specifying a query. We describe three classes of queries, each with its associated visual search method: object instances (using a bag of visual words approach for matching); object categories (using a discriminative classifier for ranking key frames); and faces (using a discriminative classifier for ranking face tracks). We discuss the features suitable for each class of query, for example Fisher vectors or features derived from convolutional neural networks (CNNs), and how these choices impact on the trade-off between three important performance measures for a real-time system of this kind, namely: (1) accuracy, (2) memory footprint, and (3) speed. We also discuss and compare a number of important implementation issues, such as how to remove 'outliers' in the downloaded images efficiently, and how to best obtain a single descriptor for a face track. We also sketch the architecture of the real-time onthe-fly system. Quantitative results are given on a number of large-scale image and video benchmarks (e.g. TRECVID INS, MIRFLICKR-1M), and we further demonstrate the performance and real-world applicability of our methods over a dataset sourced from 10,000 h of unedited footage from BBC News, comprising 5M+ key frames.

Huge amount of different music material in digital form, that can be found on the Internet, represents a big problem for a user who wants to find some particular music piece. Indexing, retrieving and classification are some of the techniques that can be used to provide faster search. In this paper, one of the methods for classification of the performers is described. We created audio databases, which consist of short audio sequences from several (nine) songs, sang by three distinct performers. Wavelet coefficients were used as feature descriptors for these music sequences. Classification was performed based on linear support vector machines. Our system exhibits very good results in the experiment with two performers (two classes): partial accuracies for classes were 100% and 98%, respectively. In the experiment with three classes (three performers), we used one-versus-one approach, and obtained partial accuracies of 50%, 82% and 35%, respectively.

This paper presents an improved frame-to-frame (F-2-F) compressed video matching technique based on local features extracted from reduced size images, in contrast with previous F-2-F techniques that utilized global features extracted from full size frames. The revised technique addresses both accuracy and computational cost issues of the traditional F-2-F approach. Accuracy is improved through using local features, while computational cost issue is addressed through extracting those local features from reduced size images. For compressed videos, the DC-image sequence, without full decompression, is used. Utilizing such small size images (DC-images) as a base for the proposed work is important, as it pushes the traditional F-2-F from off-line to real-time operational mode. The proposed technique involves addressing an important problem: namely the extraction of enough local features from such a small size images to achieve robust matching. The relevant arguments and supporting evidences for the proposed technique are presented. Experimental results and evaluation, on multiple challenging datasets, show considerable computational time improvements for the proposed technique accompanied by a comparable or higher accuracy than state-of-the-art related techniques.

The rapidly increasing quantity of publicly available videos has driven research into developing automatic tools for indexing, rating, searching and retrieval. Textual semantic representations, such as tagging, labelling and annotation, are often important factors in the process of indexing any video, because of their user-friendly way of representing the semantics appropriate for search and retrieval. Ideally, this annotation should be inspired by the human cognitive way of perceiving and of describing videos. The difference between the low-level visual contents and the corresponding human perception is referred to as the 'semantic gap'. Tackling this gap is even harder in the case of unconstrained videos, mainly due to the lack of any previous information about the analyzed video on the one hand, and the huge amount of generic knowledge required on the other. This paper introduces a framework for the Automatic Semantic Annotation of unconstrained videos. The proposed framework utilizes two nondomain-specific layers: low-level visual similarity matching, and an annotation analysis that employs commonsense knowledgebases. Commonsense ontology is created by incorporating multiple-structured semantic relationships. Experiments and black-box tests are carried out on standard video databases for action recognition and video information retrieval. White-box tests examine the performance of the individual intermediate layers of the framework, and the evaluation of the results and the statistical analysis show that integrating visual similarity matching with commonsense semantic relationships provides an effective approach to automated video annotation.

This paper presents a fast and effective technique for videos' visual similarity detection and measurement using compact fixed-length signatures. The proposed technique (dominant colour graph profile DCGP) extracts and encodes the spatio-temporal information of a given video shot into a graph-based structure (tree) that fully captures this vital information. The graph structured properties are utilized to construct a fixed-length video signature of 112 decimal values per video shot. The encoded spatio-temporal information is extracted following channelling each video frame into a block-based structure, where the positions of respective blocks are tracked across video frames and encoded into multiple DCGP trees. The proposed technique provides a high matching speed (>2000 fps) and robust retrieval performance. The experiments on various standard and challenging datasets shows the framework's robust performance, in terms of both, retrieval and computational performances.

We propose a unified method for recognizing human action and human related
events in a realistic video. We use an efficient pipeline of (a) a 3D representation of the
Improved Dense Trajectory Feature (DTF) and (b) Fisher Vector (FV). Further, a novel
descriptor is proposed, capable of representing human actions and human related events
based on the FV representation of the input video. The proposed unified descriptor is a 168-
dimensional vector obtained from each video sequence by statistically analyzing the motion
patterns of the 3D joint locations of the human body. The proposed descriptor is trained
using binary Support Vector Machine (SVM) for recognizing human actions or human
related events. We evaluate the proposed approach on two challenging action recognition
datasets: UCF sports and CMU Mocap datasets. In addition to the two action recognition
dataset, the proposed approach is tested on the Hollywood2 event recognition dataset. On
all the benchmark datasets for both action and event recognition, the proposed approach has
shown its efficacy compared to the state-of-the-art techniques.

The presentation focuses on finding of similar videos. The notion of «near-duplicates» is introduced. A near-duplicate is an object that approximates another object of the same class. The presentation contains near duplicate problem statement and a generalized model of the video search. Also a method of searching and comparing near duplicate videos is proposed. The method reduces near duplicate videos retrieval to time series indexation. This is achieved through a temporal video segmentation. Also a streaming video filtering declarative language is presented. It helps to implement a video segmentation. A final algorithm for near duplicate videos indexation and retrieval is formulated and compared with analogues.

Доклад посвящён поиску похожих видео. Введено понятие «нечетких дубликатов» видео. Нечетким дубликатом называют объект, частично совпадающий с~другим объектом подобного класса. Рассмотрена актуальность проблемы поиска нечетких дубликатов видео. Построена обобщённая модель поиска по видео. Предложен метод поиска и сравнения нечетких дубликатов, который с помощью сегментации видео по временной оси, сводит задачу к поиску и сравнению временных рядов. Представлен декларативный язык потоковой фильтрации видео, с~помощью которого реализована сегментации. Сформулирован алгоритм индексации и поиска нечётких дубликатов видео. Проведено сравнение с аналогами.

The term “Near-duplicate” is an object that is fully or partly similar to another object. There are natural and artificial near-duplicates. Natural near-duplicates are similar objects within the similar environment, while artificial near-duplicates are objects, which comes from one original object. Now lets, turn to the main aims of the the near duplicate retrieval. Firstly, the near duplicate retrieval may be helpful for the optical navigation of cruise missiles or pilotless planes. Secondly, it is helpful for grouping snippets of the search engines. Moreover, it is necessary for the video classification and copyright management. This is the most important part for the online cinema. Nowadays there are several ways to find near duplicates. First one is based on global video features. Another is based on local and global features of particular frames. Then, youtube.com uses the audio based approach while, Licenzero uses the visual words. Also the combinations of different methods are applied. We propose an algorithm of the near duplicate video retrieval based on the shot detection. A shot is a sequence of the frames, which are considerably different form the frames of another sequence. Imagine our task as the following: There are a lot of the initial videos. Somebody gives us some new video, and we should say if it is a copy of one or more of the initial videos. We compute scene descriptors for every shot of every initial video and for every shot of the given video. Then we compare these descriptors to each other.If several descriptors of the given video are coincide with descriptors of the initial videos, it means these parts of videos are duplicates. Else, the given video is unique. As a result of this work we have the approach for the near-duplicate video retrieval. Firstly, It includes length ratios and relative lengths. Secondly, we use the scene aliment. The same method is used in the computer linguistics for the sentences aliment in the bilingual corpora. It is called Gale-Church algorithm. Thirdly, we propose scene descriptors. Several experiments have been conducted.

Существует широкий круг задач, где требуется анализ, аудио-визуальных моделей реальности. В частности, для многих военных и гражданских приложений, необходимо наличие поиска нечетких дубликатов видео. Для мирного применения, — это группировка подсказок в выдаче поисковиков, поиск неправомерных публикаций видеофайлов, сокращение объема хранимой и передаваемой информации. Для военного применения можно тоже перечислить несколько вариантов.

Презентация доклада «Элементы поиска нечетких дубликатов видео» на XIII Всероссийской научной конференции «Нейрокомпьютеры и их применение» (17 марта 2015 года).

Я хочу рассказать о поиске нечетки дубликатов видео. Понятие «нечеткий дубликат» означает неполное или частичное совпадение текущего документа (изображения) с другим документом подобного класса. Почему интересно заниматься поиском дубликатов именно видео?

Видео

На слайде приведены ретроспектива и прогноз развития Российского интернет видео.

Данные взяты из http://www.ewdn.com/online-video-russia.html

Online Video in Russia / Adrien Henni, Vladimir Baydin, Alexandra Chekareva et al.;
Ed. by Adrien Henni;
East-West Digital News. –– First edition. –– East-West Digital News, 2013. –– October. ––
Main partners: comScore, Ernst & Young, the Moscow Higher School of Economics, The Next Web.
URL: http://www.ewdn.com/online-video-russia.html
Видно, что ожидается рост как и зрителей онлайн‑видео, так и количество просмотров от одного зрителя. Издатели, пользователи и сторонние организации часто редактируют и публикуют копии оригинального видео. Часто одинаковые видео снабжаются совершенно разными, причем очень часто не соблюдаются в том числе и правовые аспекты. Задача поиска таких копий могут быть решены через поиск нечетких дубликатов видео. Кроме того поиск нечетких дубликатов может быть полезен для организации систем рекомендаций по видео.

Пример ошибочной рекомендации видео

На слайде показана страница из Youtube с роликом, посвященным Челябинскому метиориту. Можно увидеть, что в качестве похожих видео нам предложены очень далекие по содержанию материылы.

Поиск

Поиск нечетких дубликатов полезен и для группировки выдачи поисковых систем. Задача казалось бы простая и должна была бы быть уже решена. Но вот например для фильма «Очаровательный Коралловый Риф» Ни одна из популярных поисковых систем не смогла корректно выделить дубликаты. Пример для Mail.ru Аналогичную картину можно видеть для Яндекса. Такая же проблема наблюдается и для Вконтакте. Ну и для упомянутого ранее тут Youtube.

Группировка новостей

НДВ удобно использовать для просмотра видео‑материалов о событиях, размещенных в разных СМИ. Это особенно полезно для новостей. Видео часто показывают очень похожие визуальные изображения и ролики, но с совершенно различной трактовкой. Авторская интерпретация преподносится зрителям, например, в комментариях диктора или субтитрах. Обычно отличается подача информации в СМИ, которые контролируют разные общественные или политические силы. СМИ, поддерживающие противоположные позиции, могут описать ситуацию поразному, но визуальное содержание, всего скорее, будет близким. Обнаружение таких видео и подача их вместе помогут людям получать более полное представление о происходящем с учетом различных точек зрения.

Подходы к определению понятия НДВ

Тема поиска НДВ относительно нова для научного сообщества.

И потому существует целый ряд различных толкований и определений.

Наиболее употребляемые определения Ву, Шена, Башарата, и Керубини, представленные далее, относятся к одному и тому же.

НДВ по Ву — копии друг друга, отличаются набором модификаций.
НДВ по Шену — общая сцена, разные условия съемки и варианты монтажа.
НДВ по Башарату — общая смысловая концепция, разные варианты представления.
НДВ по Керубини — общая семантика и нет дополнительной информации.
Схема поиска по видео

Поиск видео состоит из следующих шагов:

Анализ временно́й структуры видео — деление видео на фрагменты, которое включает обнаружение границ съёмок.
Определение характеристик фрагментов: параметров ключевых кадров, характеристик движения и объектов.
Извлечение информации из характеристик.
Аннотация видео, построение семантического индекса.
Обработка пользовательского запроса и выдача результата.
Обратная связь и переранжирование результатов для улучшения поиска характеристик.
Сегментация видео

Деление видео включает всебя обнаружение границ съёмок, извлечение ключевых кадров, сегментацию сцен иаудио. Съёмка — набор кадров внутри непрерывной временной области, кадры, которой значительно отличаются от кадров соседних областей. Кадры съёмок связаны между собой по смыслу. Съёмки считаются основной структурной единицей видео. Исторически, съёмка (монтажный план, кинематографический кадр) — отрезок киноплёнки, на котором запечатлено непрерывное действие между пуском и остановкой камеры, или между двумя монтажными склейками. Методы сегментации видео работают с уже созданным видео, и достоверно определить была монтажная склейка или нет, не всегда возможно. Именно по этой причине в литературе используется конструктивное определение съёмки, которое мы привели в начале абзаца. Сцена — группа смежных съёмок, связанных конкретной темой или предметом. С точки зрения семантики, самым мелким элементом видео является кадр (фотографический кадр, кадрик). Съёмка является более крупным делением. Из съёмок складываются сцены, а из сцен видео целиком. Деление видео включает в себя обнаружение границ съёмок, извлечение ключевых кадров, сегментацию сцен и аудио. Это может показаться не естественным, но сегментацию сцен и выделение ключевых кадров проще проводить, если уже известны границы съёмок.

Выделение признаков

Из полученных частей видео выделяют признаки. К признакам относят: характеристики ключевых кадров; объекты; движение в кадре; характеристики аудио и текста.

Типы видео‑сигнатур

Локальные:

выражают локальную информацию кадра,
напрямую работают с дескрипторами особых точек.
Глобальные:

выражают одной сигнатурой целый кадр;
выражают одной сигнатурой все видео целиком;
выражают изменении кадров во времени.
Локальные сигнатуры: оценка

Пусть дано некоторое видео:

разрешение 800 × 600 пикселей;
в кадре может быть найдено около 1000 точек;
Если таких видео много:

100 000 видео‑файлов;
по 5 ключевых кадров в каждом виде;
500 000 000 ключевых точек;
Каждая точка представлена вектором:

SIFT (128 компонент)
или PCA‑SIFT (64 компоненты);
Глобальные сигнатуры

Одна сигнатура — целый кадр:

средние цветовые моменты кадра;
цветовые моменты областей кадра (дескриптор GIST);
мешок визуальных слов;
линеаризация, видео как ДНК;
Одна сигнатура — видео целиком:

средний цветовой момент видео;
ссылочная видео‑гистограмма — набор опорных кадров;
кластеризация кадров.
Пространственно‑временные сигнатуры:

отслеживание траекторий;
линеаризация, порядковая сигнатура;
последовательность монтажных склеек;
дерево сцен.
Что такое видео?

Видео содержит в себе несколько видов данных.

метаданные — заголовок, автор и описание;
звуковая дорожка;
тексты полученные при помощи технологии оптического распознавания символов (OCR) или при помощи технологий распознавания речи;
визуальная информация кадров видео.
Важно, что источники данных разной природы не являются независимыми. Они тесно связаны и дополняют друг друга, позволяя воссоздать полную картину происходящего. Таким образом, видео обладает комплексностью. Комплексность (системность, мультимодальность)  — способность взаимодействовать с пользователем по различным каналам информации и извлекать и передавать смысл автоматически. Комплексность видео состоит в возможности автора выражать мысли, используя по крайней мере два информационных канала. Каналы могут быть визуальными, звуковыми или текстовыми.

Видео — это последовательность

Видео — последовательность фактов, развивающихся во времени.

Свойства фактов — пространственная характеристика видео,
продолжительность и порядок фактов — временная.
Что есть факты?

простейший вариант — сцены‑съёмки;
ищем точки смены съёмок (монтажных склеек).
Пространственная характеристика: начальный и конечный кадр («мешок слов», GIST).

Временная характеристика:

отношения длин съёмок к длинам соседних съёмок
алгоритмы выравнивания последовательностей.
Результаты и перспективы

Предложен подход:

относительные длины;
выравнивания;
дескриптор фактов в видео;
Проведены эксперименты (17 тыс. фильмов).

Приложения

алгоритм поиска нечетких дубликатов;
что такое относительные длины;
временные отметки перемены сцен;
выравнивания последовательностей.

XI All-Russian Conference “Neurocomputers and their application”, Мoscow: MSUPE, 19.03.2013.

Понятие «нечеткий дубликат» означает неполное или частичное совпадение текущего документа (изображения) с другим документом подобного класса.

Дубликаты бывают естественные и искусственные. Естественные дубликаты — схожие объекты, при схожих условиях. Искусственные нечеткие дубликаты — полученные на основе одного и того же оригинала.

Поиск нечетких дубликатов может быть полезен для оптической навигации беспилотных аппаратов, для определения характера ландшафта местности, составления каталогов видео, группировки сниппетов поисковых систем, фильтрация видео рекламы, и поиска пиратского видео.

Нечеткие дубликаты тесно связаны с проблемами классификации видео и поиска по видео. Но эти задачи являются самостоятельными.

На данный момент нечеткие дубликаты пытаются искать, сравнивая глобальные особенности видео, глобальные и локальные особенности кадров и множеств кадров. Сравнивают звуковой ряд (так, например делюат в ютубе). Ищут и сравнивают визуальные слова, характерные последовательности (лицензЕро). Используется комбинация методов.

Мы предлагаем алгоритм поиска нечетких дубликатов на основе сцен. Есть множество исходных файлов, для видео сцен этих файлов вычислены дескрипторы сцен. Получаем новое видео. Требуется установить является ли новое видео дубликатом существующих. Вычисляем дескрипторы сцен этого видео . Дескриптор каждой сцены нового видео сравниваем с каждым дескриптором кадой сцены каждого исходного видео.

Если на нектором временном промежутке дескрипторы совпали, считаем, считаем, что временной промежуток является дубликатом одной из частей исходного видео. Если не несовпали ни на одном — считаем видео уникальным.

Что такое сцена?

Существует три различных понятия. Кадр, Фотографический кадр — статическая картинка. Сцена, монтажный кадр — множество кадров связанных единством места и времени. Съемка, кинематографических кадр, множество кадров связанных единством съемки. 1 сцена может включать несколько съемок. Съемку часто называют <<сценой>>. Далее мы будем рассматривать съемку, но называть ее будем <<сценой>>.

Введем формальное определение. Сцена-съемка, набор кадров определенной временной области кадры которой значительно отличаются от кадров соседних опластей. Аналогичным образом можно ввести понятие звуковой сцены, правда возникает отдельная проблема выделения звуковых-кадров.

Выделение сцен происходит на основе трех базовых подходов. Это сравнение гистограм соседних кадров. Сравнение спектров кадров и сравнение векторов оптического потока. На слайде приведены первые кадры сцен рекламы на МТС, выделенные c помощью ffmpeg. ffmpeg использует сравнение гистограмм кадров.

Если исходную рекламу сжать разными кодеками, мы получим нечеткие дубликаты этого видео. Выделяя сцены в каждом видео, увидим, что точки перемены сцен для этих двух файлов не совпадают.

Если взять более продолжительное видео, и выделить сцены при более высокой чувствительности, то можно заметить, что для разных кодеков сцены определяются совсем не однородно. Более того, некоторые сцены могут не распознаться.

Для решения этой проблемы мы предлагаем использовать относительные длины сцен. Для каждой сцены вычисляем отношение длины сцены к длинам остальных сцен. Для первых четрех сцен рекламы МТС получаем следующую матрицу. В практических задачах удобнее вычислять отношения длин, для трех предыдущих сцен. Это удобно и вслучае, если все видео целиком нам недоступно.

Если относительная длина сцены одного видео отличается от длины соответствующей сцены другого видео не более чем в два раза, то можно предположить, что сцены выражают одно и то же явление. В этом случае то длину меньшей сцены можно сложить с длиной следующей сцены этого же видео, и рассматривать объединенную сцену как одну. Подобные метод называют алгоритмом Гейла-Черча. Используется в математической лингвистике для выравнивания праллельных корпусов тесктов на разных языках.


Если относительные длины двух сцен совпали, не факт что сцены действительно одинаковы. Требуется сравнивать внутренние свойства сцен. Можно взять начальные и конечные кадры.

Сравнивать можно попиксельно, на основе детектора краев. В данном случчае удобно использовать глобальные дескриптор GIST, и мешок слов.

GIST представляем из себя простой но не очень точный способ глобального описания изображения. Применим для широкого круга задач.

Мешок слов, обычно более точен, при достаточном размере словаря. Необходимо, чтобы был набор изображений, на которых можно обучиться. Хорош, если работа ведется только в рамках конкретной предметной области. Но работает медленне и потенциально бесконечен по памяти.

Т.о. получили дескриптор сцены. Он состит из вектора отношений длины сцены к длинам других сцен; Его удобно сразу хранить, с объединениями соседних сцен учитывая гиппотезу Гейла Черча; И характеристики начального и конечного кадров. В зависимоти от задачи мешок слов или GIST.

Вернемся к предложеннуму алгоритмы. Мы сравнивали дескрипторы на прямую в пространстве L2. Но это не эффективно, для больших дескрипторов.

Можно предложить использование цифровых семантических подписей. Введем бинарные подписи. Подписи должны быть близки для близких дескипторах в L2.

Самое простое, что можно предложить: Линейно чувствительные хеши. Пространство дескрипторов, делим гиперплоскостью на 2 подпространства. Назначаем дескрипторам этих подпространств подпись 0 или 1. Получили бит подписи. С увеличением числа бить ассимптотически приближаем метрику L2. Для задач подобного рода лучше себя показали обучаемые хеши. К ним относятся бустинг и ограниченная машина Больцмана. В качесвтве бустинга исользуется модификация AdaBoost. Цель — получить такие подписи, расстояние между дескрипторами вычисляется, будет вычисляться как расстояние Хемминга.

Ограниченная машина Больцмана: Вероятностная рекурентная нейронная сеть. Вероятностная версия сети Хопфилда. Или нейросетевая версия Скрытой Модели Маркова. Применяется модификация, без связей внутри слоев. Мощность словем понижается от размера входного вектора до размера требуемого кода.


В результате проделанной работы: был разработан подход поиска нечетких дубликатов видео на основе сцен. Его основные моменты: относительные длины, выравнивания сцен на основе лингвистичского алгоритма Гейла-Черча, дескрипторы сцен; Проведен ряд экспериментов, которые показали удобства подхода.

Далее нам требуется разработать собственный инструмент для поиск перемены, сейчас используется ffmpeg. лучше проработать схемы с использованием Больцмана Провести эксперименты на реальном множестве исxодных данных.

The paper focuses on the algorithms of the event detection in content-based video retrieval. Video has a complex structure and can express the same idea in different ways. This makes the task of searching for video more complicated. Video titles and text descriptions cannot give the whole information about objects and events in the video. This creates a need for content-based video retrieval. There is a semantic gap between low-level video features, that can be extracted, and the users’ perception. The task of event detection is reduced to the task of video segmentation. Complex content-based video retrieval can be regarded as the bridge between traditional retrieval and semantic-based video retrieval. The properties of video as a time series are described. Introduced the concept of anomalies in the video. A method for event detection based on comparing moving averages with windows of different sizes is proposed. According to the classification given at the beginning of this article, our method refers to statistical methods. It differs from other methods with low computational complexity and simplicity. The video stream processing language is proposed for function-based description of video handling algorithms. So, our method is formulated in the form of a declarative description on an interpreted programming language. Unfortunately, most of the existing video processing methods use exclusively imperative approach, which often makes understanding more difficult. Examples of the use of this language are given. Its grammar is described too. As shown by experiments, the implementation of the proposed video events retrieval method, unlike their counterparts, can work for video streams too with a real-time and potentially infinite frame sequences. Such advantages within low computational requirements make implementation of the method helpful in aviation and space technology. The algorithm has some disadvantages due to necessity parameter selection for particular task classes. The theorem on near-duplicates of video is formulated at the end of the article. It asserts the near-duplicate videos express the same sequence of phenomena.

The objective of this work is to visually search large-scale video datasets for semantic entities specified by a text query. The paradigm we explore is constructing visual models for such semantic entities on-the-fly, i.e. at run time, by using an image search engine to source visual training data for the text query. The approach combines fast and accurate learning and retrieval, and enables videos to be returned within seconds of specifying a query. We describe three classes of queries, each with its associated visual search method: object instances (using a bag of visual words approach for matching); object categories (using a discriminative classifier for ranking key frames); and faces (using a discriminative classifier for ranking face tracks). We discuss the features suitable for each class of query, for example Fisher vectors or features derived from convolutional neural networks (CNNs), and how these choices impact on the trade-off between three important performance measures for a real-time system of this kind, namely: (1) accuracy, (2) memory footprint, and (3) speed. We also discuss and compare a number of important implementation issues, such as how to remove 'outliers' in the downloaded images efficiently, and how to best obtain a single descriptor for a face track. We also sketch the architecture of the real-time onthe-fly system. Quantitative results are given on a number of large-scale image and video benchmarks (e.g. TRECVID INS, MIRFLICKR-1M), and we further demonstrate the performance and real-world applicability of our methods over a dataset sourced from 10,000 h of unedited footage from BBC News, comprising 5M+ key frames.

In this paper, we propose a robust approach for text extraction and recognition from Arabic news video sequence. The text included in video sequences is an important needful for indexing and searching system. However, this text is difficult to detect and recognize because of the variability of its size, their low resolution characters and the complexity of the backgrounds. To solve these problems, we propose a system performing in two main tasks: extraction and recognition of text. Our system is tested on a varied database composed of different Arabic news programs and the obtained results are encouraging and show the merits of our approach.

The IRIM group is a consortium of French teams working on Multimedia Indexing and Retrieval. This paper describes our participation to the TRECVID 2010 semantic indexing and instance search tasks. For the semantic indexing task, we evaluated a number of different descriptors and tried different fusion strategies, in particular hierarchical fusion. The best IRIM run has a Mean Inferred Average Precision of 0.0442, which is above the task median performance. We found that fusion of the classification scores from different classifier types improves the performance and that even with a quite low individual performance, audio descriptors can help. For the instance search task, we used only one of the example images in our queries. The rank is nearly in the middle of the list of participants. The experiment showed that HSV features outperform the concatenation of HSV and Edge histograms or the Wavelet features.

Semantically enriched multimedia information is crucial for equipping the kind of multimedia search potentials that professional searchers need. But the semantic interpretation of multimedia is obsolete without some mechanism for understanding semantic content that is not explicitly available. Manual annotation is the only source to overwhelming this, which is not only time consuming and costly but also lacks semantic enrichment in terms of concept diversity and concept enrichability. In this paper, we present semantically enhanced information extraction model that calculate the semantic intensity (SI) of each object in the image and then enhance the tagged concept with the assistance of lexical and conceptual knowledgebases .i.e. WordNet and ConceptNet. Noises, redundant and unusual words are then filtered out by means of various techniques like semantic similarity, stopwords and words unification. The experiment has been carried out on the LabelMe datasets. Results demonstrate the substantial improvement in terms of concept diversity, concept enrichment and retrieval performance.

Large scale labeled datasets are of key importance for the development of automatic video analysis tools as they, from one hand, allow multi-class classifiers training and, from the other hand, support the algorithms' evaluation phase. This is widely recognized by the multimedia and computer vision communities, as witnessed by the growing number of available datasets; however, the research still lacks in annotation tools able to meet user needs, since a lot of human concentration is necessary to generate high quality ground truth data. Nevertheless, it is not feasible to collect large video ground truths, covering as much scenarios and object categories as possible, by exploiting only the effort of isolated research groups. In this paper we present a collaborative web-based platform for video ground truth annotation. It features an easy and intuitive user interface that allows plain video annotation and instant sharing/integration of the generated ground truths, in order to not only alleviate a large part of the effort and time needed, but also to increase the quality of the generated annotations. The tool has been on-line in the last four months and, at the current date, we have collected about 70,000 annotations. A comparative Multimed Tools Appl performance evaluation has also shown that our system outperforms existing state of the art methods in terms of annotation time, annotation quality and system's usability.

In this paper, an automatic content-based video shot indexing framework is proposed employing five types of MPEG-7 low-level visual features (color, texture, shape, motion and face). Once the set of features representing the video content is determined, the question of how to combine their individual classifier outputs according to each feature to form a final semantic decision of the shot must be addressed, in the goal of bridging the semantic gap between the low level visual feature and the high level semantic concepts. For this aim, a novel approach called "perplexity-based weighted descriptors" is proposed before applying our evidential combiner NNET [3], to obtain an adaptive classifier fusion PENN (Perplexity-based Evidential Neural Network). The experimental results conducted in the framework of the TRECVid'07 high level features extraction task report the efficiency and the improvement provided by the proposed scheme.

Although Support Vector Machines (SVMs) have been successfully applied to solve a large number of classification and regression problems, they suffer from the catastrophic forgetting phenomenon. In our previous work, integrating the SVM classifiers into an ensemble framework using Learn++ (SVMLearn++) [1], we have shown that the SVM classifiers can in fact be equipped with the incremental learning capability. However, Learn++ suffers from an inherent out-voting problem: when asked to learn new classes, an unnecessarily large number of classifiers are generated to learn the new classes. In this paper, we propose a new ensemble based incremental learning approach using SVMs that is based on the incremental Learn++.MT algorithm. Experiments on the real-world and benchmark datasets show that the proposed approach can reduce the number of SVM classifiers generated, thus reduces the effect of outvoting problem. It also provides performance improvements over previous approach.

Support Vector Machines (SVMs) have been successfully applied to solve a large number of classification and regression problems. However, SVMs suffer from the catastrophic forgetting phenomenon, which results in loss of previously learned information. Learn ++ have recently been introduced as an incremental learning algorithm. The strength of Learn ++ lies in its ability to learn new data without forgetting previously acquired knowledge and without requiring access to any of the previously seen data, even when the new data introduce new classes. To address the catastrophic forgetting problem and to add the incremental learning capability to SVMs, we propose using an ensemble of SVMs trained with Learn ++ . Simulation results on real-world and benchmark datasets suggest that the proposed approach is promising.

The paper focuses on an overview of the different existing methods in content-based video retrieval. During the last decade there was a~rapid growth of video posted on the Internet. This imposes urgent demands on video retrieval. Video has a complex structure and can express the same idea in different ways. This makes the task of searching for video more complicated. Video titles and text descriptions cannot give the hole information about objects and events in the video. This creates a need for content-based video retrieval. There is a semantic gap between low-level video features, that can be extracted, and the users' perception. Complex content-based video retrieval can be regarded as the bridge between traditional retrieval and semantic-based video retrieval.

Keywords: frames; near-duplicates video; scenes; shots; video annotation; video classification; video mining; video reranking; video retrieval.

В статье предлагается обзор различных существующих методов ассоциативного поиска по видео. В течение прошлого десятилетия наблюдался стремительный рост количества видео размещаемых в Интернете, что создало острую необходимость в появлении поиска по видео. Видео имеет сложную структуру. Одна и та же информация может быть выражена различными способами. Это серьёзно усложняет задачу видео-поиска. Заголовки и описания видео не могут дать полного представления о самом видео, что влечет за собой необходимость использования ассоциативного поиска по видео. Существует семантический разрыв между низкоуровневыми характеристиками видео и восприятием пользователей. Комплексный ассоциативный видео-поиск может рассматриваться как связующее звено между обычным поиском и смысловым поиском по видео.

Ключевые слова: анализ видео; аннотирование видео; видео-поиск; кадры; классификация видео; нечеткие дубликаты видео; ранжирование видео; сцены; съёмки.

В приведенном обзоре рассмотрены некоторые методы поиска нечетких дубликатов видео, их преимущества и недостатки. На основе структурного представления видео построена комбинация методов и предложен дескриптор съёмки.

Методы поиска нечетких дубликатов реализуют следующие шаги.
1. Видео делят на отрезки.
2. Из каждого отрезка выделяют ключевые кадры.
3. Для представления ключевых кадров извлекают последовательности в которых содержатся характеристики кадров. Характеристики ключевых кадров используются для представления всего видео.
4. Подобие между видео как подобие вычисляют наборов характеристик ключевых кадров.
Различают методы, использующие глобальные характеристики (ГХ) и методы, использующие локальные характеристики (ЛХ). ЛХ‑методы сводят задачу поиска похожих видео к задаче поиска дубликатов изображений.
В работе рассмотрены ЛХ‑методы:
отслеживание траекторий особых точек кадра, сравнение особых точек ключевых кадров, сравнение визуальных «видео‑слов» по частотным словарям ГХ‑методы моделируют пространственную, цветовую и временную информацию кадров и сцен видео. Сходство между видео из запроса и видео из базы определяется как соответствие подписей (сигнатур) видео. Глобальные характеристики полезны, для поиска «почти одинаковых» видео и могут выявить незначительные правки в пространственно‑временной области. В работе рассмотрено несколько способов формирования подписей для ГХ‑методов:
локально‑чувствительное хеширование; использование порядковой меры; представление видео как последовательности геномов (ДНК‑представление);
фиксация точке перемены съёмок (сцен);
построение дерева перемены съёмок.
На данный момент наиболее перспективными кажутся ГХ‑методы. Они позволяют без особых вычислительных затрат приближенно решить задачу. Однако, для уточнения могут потребоваться ЛХ‑методы.
В конце работы предложена комбинация методов на основе структурного представления видео. Введено понятие дескриптора съёмки. Он состоит:
из вектора отношений длин съёмок к длинам других съёмок — глобальные характеристики и описания свойств начального и конечного кадров — локальные характеристики.

The paper focuses on an overview of the different existing methods in near duplicate video retrieval. The advantages and disadvantages of the methods are discussed. At the end we suggest an original combination of methods based on structural video representation and propose a novel shot descriptor.

Near duplicate video retrieval techniques implement the following steps.
1. Video is divided into segments.
2. Keyframes are extracted from each segment.
3. Features are extracted from keyframes. Keyframe features represent the whole video.
4. The similarity between the videos is calculated as a similarity between keyframe features.
There are methods using global features and the local ones. Local methods reduce the problem of finding similar videos to the problem of finding image duplicates. The paper discusses the local methods:
tracing the trajectories of the points of interest in the frame; comparison of the points of interest in keyframes; bag of visual keywords.
Global methods simulate the spatial, temporal and color information of frames and scenes. The similarity between the video query and the videos from database is defined as the similarity of video signatures. Global features are useful for retrieve „almost identical“ videos and can identify minor changes in the space‑time domain. In this paper we consider several ways to built global signatures:
locally sensitive hashing; ordinal measure; video DNA‑representation; shot change positions; the shot change tree.
At the end the combination of methods within a novel shot descriptor is 0 proposed. The descriptor consists of:
a vector of the relations of shot lengths to lengths of other shots (relative shot lengths) as a global feature; features of initial and final frames as local ones.

In this paper, an automatic content-based video shot indexing framework is proposed employing five types of MPEG-7 low-level visual features (color, texture, shape, motion and face). Once the set of features representing the video content is determined, the question of how to combine their individual classifier outputs according to each feature to form a final semantic decision of the shot must be addressed, in the goal of bridging the semantic gap between the low level visual feature and the high level semantic concepts. For this aim, a novel approach called "perplexity-based weighted descriptors" is proposed before applying our evidential combiner NNET [3], to obtain an adaptive classifier fusion PENN (Perplexity-based Evidential Neural Network). The experimental results conducted in the framework of the TRECVid'07 high level features extraction task report the efficiency and the improvement provided by the proposed scheme.

Intelligent video analysis is a problem of great importance for applications such as surveillance and automatic annotation. We present, in this paper, a hybrid, knowledge -based approach for object recognition in video sequences. Objects are modelled, in the signal level, through the visual descriptors defined by MPEG-7, the ISO standard for description of audiovisual content and in the semantic level, through the semantic relations defined by MPEG-7. The method of video analysis is synopsised as follows: first, moving regions are extracted using an active contour technique. Second, visual descriptions of the moving regions are extracted and are matched with the ones defined for recognizable objects.