Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Members Methods
Discussion
Data Sharing
References

IRIM at TRECVID 2012: Semantic Indexing and Instance Search

Profile image of Alexandre BenoitAlexandre BenoitProfile image of Patrick LambertPatrick Lambert

2012

Abstract
sparkles

AI

The IRIM group participated in the TRECVID 2012 semantic indexing and instance search tasks, utilizing a six-stage processing pipeline comprising descriptor extraction, optimization, classification, fusion, higher-level fusion, and re-ranking for semantic indexing, achieving a MAP of 0.2378. For instance search, a two-step fusion of similarity scores was employed, resulting in a MAP of 0.1192. The paper details the methodologies, evaluations, and rankings achieved in these tasks.

Related papers

The Stanford/Technicolor/Fraunhofer HHI Video Semantic Indexing System
JOAQUIN ZEPEDA

Video search has become a very important tool, with the ever-growing size of multimedia collections. This work introduces our Video Semantic Indexing system. Our experiments show that Residual Vectors provide an efficient way of aggregating local descriptors, with complementary gain with respect to BoVW. Also, we show that systems using a limited number of descriptors and machine learning techniques can still be quite effective. Our first participation at the TRECVID evaluation has been very fruitful: our team was ranked 6 th in the light version of the Semantic Indexing task.

View PDFchevron_right
IRIM at TRECVID 2013 : Semantic Indexing and Instance Search
Boris Mansencal

TREC Video Retrieval Evaluation, 2013

The IRIM group is a consortium of French teams working on Multimedia Indexing and Retrieval. This paper describes its participation to the TRECVID 2013 semantic indexing and instance search tasks. For the semantic indexing task, our approach uses a six-stages processing pipelines for computing scores for the likelihood of a video shot to contain a target concept. These scores are then used for producing a ranked list of images or shots that are the most likely to contain the target concept. The pipeline is composed of the following steps: descriptor extraction, descriptor optimization, classification, fusion of descriptor variants, higher-level fusion, and re-ranking. We evaluated a number of different descriptors and tried different fusion strategies. The best IRIM run has a Mean Inferred Average Precision of 0.2796, which ranked us 4th out of 26 participants.

View PDFchevron_right
The MediaMill TRECVID 2009 semantic video search engine
Muhammad Tahir

2010

In this paper we describe our TRECVID 2009 video retrieval experiments. The MediaMill team participated in three tasks: concept detection, automatic search, and interactive search. Starting point for the MediaMill concept detection approach is our top-performing bag-of-words system of last year, which uses multiple color descriptors, codebooks with soft-assignment, and kernel-based supervised learning. We improve upon this baseline system by exploring two novel research directions. Firstly, we study a multi-modal extension by the inclusion of 20 audio concepts and fusing using two novel multi-kernel supervised learning methods. Secondly, with the help of recently proposed algorithmic refinements of bag-of-words, a bag-of-words GPU implementation, and compute clusters, we scale-up the amount of visual information analyzed by an order of magnitude, to a total of 1,000,000 i-frames. Our experiments evaluate the merit of these new components, ultimately leading to 64 robust concept detectors for video retrieval. For retrieval, a robust but limited set of concept detectors necessitates the need to rely on as many auxiliary information channels as possible. For automatic search we therefore explore how we can learn to rank various information channels simultaneously to maximize video search results for a given topic. To improve the video retrieval results further, our interactive search experiments investigate the roles of visualizing preview results for a certain browse-dimension and relevance feedback mechanisms that learn to solve complex search topics by analysis from user browsing behavior. The 2009 edition of the TRECVID benchmark has again been a fruitful participation for the MediaMill team, resulting in the top ranking for both concept detection and interactive search.

View PDFchevron_right
A generic framework for semantic video indexing based on visual concepts/contexts detection
Nizar Elleuch

Multimedia Tools and Applications, 2014

Providing a semantic access to video data requires the development of concept detectors. However, semantic concepts detection is a hard task due to the large intra-class and the small inter-class variability of content. Moreover, semantic concepts co-occur together in various contexts and their occurrence may vary from one to another. Thus, it is interesting to exploit this knowledge in order to achieve satisfactory performances. In this paper we present a generic semantic video indexing scheme, called SVI_REGIMVid. It is based on three levels of analysis. The first level (level1) focuses on low-level processing such as video shot boundary/ key-frame detection, annotation tools, key-points detection and visual features extraction tools. The second level (level2) aims to build the semantic models for supervised learning of concepts/contexts. The third level (level3) enriches the semantic interpretation of concepts/ contexts by exploiting fuzzy knowledge. The obtained experimental results are promising for a semantic concept/context detection process. Keywords Semantic indexing. Concepts. Contexts. SVI_REGIMVid 1 Introduction Recent years have witnessed a rapid evolution of technology acquisition and diffusion of multimedia data. The web, television channels and video blogs have led to the explosion of audiovisual databases. This vast amount of audiovisual data currently represents the main challenge research studies on the automatic information processing particularly in the representation and indexing of video contents. The archive professionals need to index their digital video bases for an effective access to video contents.

View PDFchevron_right
The MediaMill TRECVID 2004 Semantic Video Search Engine
Dennis Koelma

2004

This year the UvA-MediaMill team participated in the Feature Extraction and Search Task. We developed a generic approach for semantic concept classification using the semantic value chain. The semantic value chain extracts concepts from video documents based on three consecutive analysis links, named the content link, the style link, and the context link. Various experiments within the analysis links were performed, showing amongst others the merit of processing beyond key frames, the value of style elements, and the importance of learning semantic context. For all experiments a lexicon of 32 concepts was exploited, 10 of which are part of the Feature Extraction Task. Top three system-based ranking in 8 out of the 10 benchmark concepts indicates that our approach is very promising. Apart from this, the lexicon of 32 concepts proved very useful in an interactive search scenario with our semantic video search engine, where we obtained the highest mean average precision of all participants.

View PDFchevron_right

References (39)

  1. A. Smeaton, P. Over and W. Kraaij, Evaluation cam- paigns and TRECVid, In MIR'06: Proceedings of the 8th ACM International Workshop on Multimedia Infor- mation Retrieval, pp321-330, 2006.
  2. P. Over, G. Awad, J. , B. Antonishek, M.2Michel, A. Smeaton, W. Kraaij, and G. Quénot, TRECVID 2012 -An Overview of the Goals, Tasks, Data, Eval- uation Mechanisms, and Metrics In Proceedings of the TRECVID 2012 workshop, Gaithersburg, USA, 26-28 Nov. 2012.
  3. Y.-C. Cheng and S.-Y. Chen. Image classification using color, texture and regions. In Image and Vision Com- puting, 21:759-776, 2003.
  4. P.H. Gosselin, M. Cord, Sylvie Philipp-Foliguet. Com- bining visual dictionary, kernel-based similarity and learning strategy for image category retrieval. In Com- puter Vision and Image Understanding, Special Issue on Similarity Matching in Computer Vision and Multi- media. Volume 110, Issue 3, Pages 403-41, 2008.
  5. M. Redi and B. Merialdo. Saliency moments for image categorization, In ICMR 2011, 1st ACM International Conference on Multimedia Retrieval, April 17-20, 2011, Trento, Italy.
  6. R. Negrel, D. Picard and P.H. Gosselin. Compact Ten- sor Based Image Representation for Similarity Search. In IEEE International Conference on Image Processing, Orlando, Florida, U.S.A, September 2012.
  7. A. Oliva and A. Torralba, Modeling the shape of the scene: A holistic representation of the spatial envelope, In International Journal of Computer Vision, vol 42, number 3, pages 145-175, 2001.
  8. K. E. A. van de Sande, T. Gevers, and C. G. M. Snoek. A comparison of color features for visual concept clas- sification. In ACM International Conference on Image and Video Retrieval, pages 141-150, 2008.
  9. Ivan Laptev, On space-time interest points, Int. J. Comput. Vision, 64:107-123, September 2005.
  10. A. Benoit, A. Caplier, B. Durette, and J. Herault. Us- ing Human Visual System Modeling for Bio-inspired Low Level Image Processing, In Computer Vision and Image Understanding, vol. 114, no. 7, pp. 758-773, 2010.
  11. S. T. Strat, A. Benoit, P. Lambert and A. Caplier, Retina Enhanced SURF Descriptors for Spatio- Temporal Concept Detection, In Multimedia Tools ans Applications, to appear, 2012.
  12. S. Paris, H .Glotin, Pyramidal Multi-level Features for the Robot Vision@ICPR 2010 Challenge, In 20th Inter- national Conference on Pattern Recognition, pp.2949- 2952, 2010
  13. B. Safadi, G. Quénot. Evaluations of multi-learners approaches for concepts indexing in video documents. In RIAO, Paris, France, April 2010.
  14. Georges Quénot. KNNLSB: K Nearest Neighbors Linear Scan Baseline, 2008. Software available at http://mrim.imag.fr/georges.quenot/freesoft/ knnlsb/index.html.
  15. Delezoide et al. IRIM at TRECVID 2011: Semantic In- dexing and Multimedia Instance Search, In Proceedings of the TRECVID 2011 workshop, Gaithersburg, USA, 5-7 Dec. 2011.
  16. Safadi et al. Quaero at TRECVID 2011: Semantic Indexing and Collaborative Annotation, In Proceedings of the TRECVID 2012 workshop, Gaithersburg, USA, 26-28 Nov. 2011.
  17. Nicolas Ballas, Benjamin Labbé, Hervé Le Borgne, Ay- men Shabou CEA LIST at TRECVID 2012: Semantic Indexing and Instance Search, In Proceedings of the TRECVID 2012 workshop, Gaithersburg, USA, 26-28 Nov. 2012.
  18. Stéphane Ayache and Georges Quénot, Video Corpus Annotation using Active Learning, In 30th European Conference on Information Retrieval (ECIR'08), Glas- gow, Scotland, 30th March -3rd April, 2008.
  19. D. Gorisse et al., IRIM at TRECVID 2010: High Level Feature Extraction and Instance Search. In TREC Video Retrieval Evaluation workshop, Gaithers- burg, MD USA, November 2010.
  20. Alice Porebski, Color texture feature selection for im- age classification. Application to flaw identification on decorated glasses printing by a silk-screen process. Phd thesis, Université Lille 1, Sciences et Technologies, Nov. 2009
  21. V. D. Blondel and J. Guillaume and R. Lambiotte and E. Lefebvre, Fast Unfolding of Community Hierarchies in Large Networks, In Computing Research Repository, abs/0803.0, 2008.
  22. B. Safadi, G. Quénot. Re-ranking by Local Re-scoring for Video Indexing and Retrieval, CIKM 2011: 20th ACM Conference on Information and Knowledge Man- agement, Glasgow, Scotland, oct 2011.
  23. J. Sivic and A. Zisserman. Video google: a text retrieval approach to object matching in videos. In ICCV'03, volume 2, pages 1470-1477, 2003.
  24. H. Bay, Herbert, T.Tuytelaars,and L. Van Gool. SURF: Speeded Up Robust Features, In ECCV 2006, pp 404-417, 2006.
  25. R. Vieux, J. Benois-Pineau, and J.-Ph. Domenger. Content based image retrieval using bag of region. In MMM 2012 -The 18th International Conference on Multimedia Modeling, 2012.
  26. P. F. Felzenszwalb and D. P. Huttenlocher. Efficient graph-based image segmentation. International Journal of Computer Vision, 59:167-181, 2004.
  27. Emilie Dumont and Bernard Merialdo. video summarization and evaluation. Multimedia Tools and Applications, Springer, Vol.48, No1, May 2010, 2010.
  28. Edwin Lughofer. Extensions of vector quantization for incremental clustering. Pattern Recognition, 41:995- 1011, 2008.
  29. S. T. Strat, A. Benoit, H. Bredin, G. Quénot and P. Lambert. Hierarchical Late Fusion for Concept Detec- tion in Videos. In ECCV workshop on Information Fusion in Computer Vision for Concept Recognition, Firenze, Italy, 13 Oct. 2012.
  30. A. Shabou and H. Le Borgne. Locality-constrained and spatially regularized coding for scene categoriza- tion, In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3618-3625, 2012.
  31. A. Levinshtein, A. Stere, K. N. Kutulakos, D. J. Fleet, S. J. Dickinson, and Kaleem. Siddiqi. Turbopixels: Fast superpixels using geometric flows. In IEEE Trans. Pat- tern Anal. Mach. Intell., 31(12):2290-2297, December 2009.
  32. C. Zhu, C.-E. Bichot, L. Chen. Color orthogonal lo- cal binary patterns combination for image region de- scription. In Technical Report, LIRIS UMR5205 CNRS, Ecole Centrale de Lyon.
  33. D.G Lowe. Distinctive image features from scale- invariant keypoints. International Journal of Computer Vision, 60:91-110, 2004
  34. R. Hartley, and A. Zisserman. Multiple view geometry in computer vision. Cambridge Univ Press, 2000
  35. R. 0. Stehling, M. A. Nascimento, and A.X. Falcão. A compact and efficient image retrieval approach based on border/interior pixel classification In 11th Interna- tional Conference on Information and Knowledge Man- agement 2002
  36. S. Poullot, M. Crucianu, and S. Satoh Indexing lo- cal configurations of features for scalable content-based video copy detection In 1st ACM workshop on Large- Scale Multimedia Retrieval and Mining (LS-MMRM'09) New York, NY, USA, 2009
  37. B. Mansencal, J. Benois-Pineau, R. Vieux and J.-Ph. Domenger Search of objects of interest in videos In 10th Workshop on Content-Based Multimedia Indexing Annecy, France, 2012
  38. E. Fox and J. Shaw Combination of Multiple searches In Proceedings of the 2nd Text Retrieval Conference Gaithersburg, USA, 1994
  39. G. Csurka and S. Clinchant An empirical study of fu- sion operators for multimodal image retrieval In 10th Workshop on Content-Based Multimedia Indexing An- necy, France, 2012

Related papers

IRIM at TRECVID 2011: Semantic Indexing and Instance Search
L. Granjon, Alexandre Benoit, Patrick Lambert

2011

View PDFchevron_right
IRIM at TRECVID 2010: Semantic Indexing and Instance Search
Boris Mansencal

2011

The IRIM group is a consortium of French teams working on Multimedia Indexing and Retrieval. This paper describes its participation to the TRECVID 2012 semantic indexing and instance search tasks. For the semantic indexing task, our approach uses a six-stages processing pipelines for computing scores for the likelihood of a video shot to contain a target concept. These scores are then used for producing a ranked list of images or shots that are the most likely to contain the target concept. The pipeline is composed of the following steps: descriptor extraction, descriptor optimization, classification, fusion of descriptor variants, higher-level fusion, and re-ranking. We evaluated a number of different descriptors and tried different fusion strategies. The best IRIM run has a Mean Inferred Average Precision of 0.2378, which ranked us 4th out of 16 participants. For the instance search task, our approach uses two steps. First individual methods of participants are used to compute similrity between an example image of instance and keyframes of a video clip. Then a two-step fusion method is used to combine these individual results and obtain a score for the likelihood of an instance to appear in a video clip. These scores are used to obtain a ranked list of clips the most likely to contain the queried instance. The best IRIM run has a MAP of 0.1192, which ranked us 29th on 79 fully automatic runs.

View PDFchevron_right
The MediaMill TRECVID 2010 semantic video search engine
Efstratios Gavves

J Amer Coll Cardiol, 2011

In this paper we describe our TRECVID 2010 video retrieval experiments. The MediaMill team participated in three tasks: semantic indexing, known-item search, and instance search. The starting point for the MediaMill concept detection approach is our top-performing bag-of-words system of TRECVID 2009, which uses multiple color SIFT descriptors, sparse codebooks with spatial pyramids, kernelbased machine learning, and multi-frame video processing. We improve upon this baseline system by further speeding up its execution times for both training and classification using GPU-optimized algorithms, approximated histogram intersection kernels, and several multi-frame combination methods. Being more efficient allowed us to supplement the Internet video training collection with positively labeled examples from international news broadcasts and Dutch documentary video from the TRECVID 2005-2009 benchmarks. Our experimental setup covered a huge training set of 170 thousand keyframes and a test set of 600 thousand keyframes in total. Ultimately leading to 130 robust concept detectors for video retrieval. For retrieval, a robust but limited set of concept detectors justifies the need to rely on as many auxiliary information channels as possible. For automatic known item search we therefore explore how we can learn to rank various information channels simultaneously to maximize video search results for a given topic. To further improve the video retrieval results, our interactive known item search experiments investigate how to combine metadata search and visualization into a single interface. The 2010 edition of the TRECVID benchmark has again been a fruitful participation for the MediaMill team, resulting in the top ranking for concept detection in the semantic indexing task. Feature Extraction Feature Extraction Word Projection Word Projection Machine Learning Machine Learning

View PDFchevron_right
Semantic video search
Marcel Worring

Proceedings - 14th International Conference on Image Analysis and Processing Workshops, ICIAP 2007, 2007

In this paper we describe the current performance of our MediaMill system as presented in the TRECVID 2006 benchmark for video search engines. The MediaMill team participated in two tasks: concept detection and search. For concept detection we use the MediaMill Challenge as experimental platform. The MediaMill Challenge divides the generic video indexing problem into a visual-only, textualonly, early fusion, late fusion, and combined analysis experiment. We provide a baseline implementation for each experiment together with baseline results. We extract image features, on global, regional, and keypoint level, which we combine with various supervised learners. A late fusion approach of visual-only analysis methods using geometric mean was our most successful run. With this run we conquer the Challenge baseline by more than 50%. Our concept detection experiments have resulted in the best score for three concepts: i.e. desert, flag us, and charts. What is more, using LSCOM annotations, our visual-only approach generalizes well to a set of 491 concept detectors. To handle such a large thesaurus in retrieval, an engine is developed which allows users to select relevant concept detectors based on interactive browsing using advanced visualizations. Similar to previous years our best interactive search runs yield top performance, ranking 2nd and 6th overall.

View PDFchevron_right
Semantic Video Indexing
Marcel Worring

Multimedia Retrieval, 2007

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved