Multimodal Fuzzy Fusion System for Semantic Video Indexing

2005

This paper surveys the approaches to video representation, focusing on semantic analysis for content-based indexing and retrieval. A problem of adaptive representation of digital multimedia is critically assessed and some novel ideas are presented. Furthermore, the concept of video multimodality is reevaluated and redefined in order to introduce modalities such as editing technique or affect to the audience.

Signal Processing, 2000

In this paper, a fuzzy representation of visual content is proposed, which is useful for the new emerging multimedia applications, such as content-based image indexing and retrieval, video browsing and summarization. In particular, a multidimensional fuzzy histogram is constructed for each video frame based on a collection of appropriate features, extracted using video sequence analysis techniques. This approach is then applied both for video summarization, in the context of a content-based sampling algorithm, and for content-based indexing and retrieval. In the "rst case, video summarization is accomplished by discarding shots or frames of similar visual content so that only a small but meaningful amount of information is retained (key-frames). In the second case, a content-based retrieval scheme is investigated, so that the most similar images to a query are extracted. Experimental results and comparison with other known methods are presented to indicate the good performance of the proposed scheme on real-life video recordings. (A.D. Doulamis) 0165-1684/00/$ -see front matter

Multimedia Tools and Applications, 2014

Providing a semantic access to video data requires the development of concept detectors. However, semantic concepts detection is a hard task due to the large intra-class and the small inter-class variability of content. Moreover, semantic concepts co-occur together in various contexts and their occurrence may vary from one to another. Thus, it is interesting to exploit this knowledge in order to achieve satisfactory performances. In this paper we present a generic semantic video indexing scheme, called SVI_REGIMVid. It is based on three levels of analysis. The first level (level1) focuses on low-level processing such as video shot boundary/ key-frame detection, annotation tools, key-points detection and visual features extraction tools. The second level (level2) aims to build the semantic models for supervised learning of concepts/contexts. The third level (level3) enriches the semantic interpretation of concepts/ contexts by exploiting fuzzy knowledge. The obtained experimental results are promising for a semantic concept/context detection process.

2009

In this paper we propose a methodology for semantic indexing of images, based on techniques of image segmentation, classification and fuzzy reasoning. The proposed knowledge-assisted analysis architecture integrates algorithms applied on three overlapping levels of semantic information: i) no semantics, i.e. segmentation based on low-level features such as color and shape, ii) mid-level semantics, such as concurrent image segmentation and object detection, region-based classification and, iii) rich semantics, i.e. fuzzy reasoning for extraction of implicit knowledge. In that way, we extract semantic description of raw multimedia content and use it for indexing and retrieval purposes, backed up by a fuzzy knowledge repository. We conducted several experiments to evaluate each technique, as well as the whole methodology in overall and, results show the potential of our approach.

Proceedings of the 13th annual ACM international conference on Multimedia - MULTIMEDIA '05, 2005

Semantic analysis of multimodal video aims to index segments of interest at a conceptual level. In reaching this goal, it requires an analysis of several information streams. At some point in the analysis these streams need to be fused. In this paper, we consider two classes of fusion schemes, namely early fusion and late fusion. The former fuses modalities in feature space, the latter fuses modalities in semantic space. We show by experiment on 184 hours of broadcast video data and for 20 semantic concepts, that late fusion tends to give slightly better performance for most concepts. However, for those concepts where early fusion performs better the difference is more significant.

1998

An integrated framework for content-based indexing and retrieval in video databases is presented in this paper, which has the capability of adapting its performance according to user requirements. Video sequences are represented by extracting a small number of key frames or scenes and constructing multidimensional feature vectors using fuzzy classification of color, motion or texture segment properties. Queries are then performed by employing a parametric distance between feature vectors, and adaptation is achieved by estimating distance parameters according to user requirements, resulting in a content based retrieval system of increased performance and flexibility.

2004

In this paper we discuss the use of knowledge for the analysis and semantic retrieval of video. We follow a fuzzy relational approach to knowledge representation, based on which we define and extract the context of either a multimedia document or a user query. During indexing, the context of the document is utilized for the detection of objects and for automatic thematic categorization.

Video search has become a very important tool, with the ever-growing size of multimedia collections. This work introduces our Video Semantic Indexing system. Our experiments show that Residual Vectors provide an efficient way of aggregating local descriptors, with complementary gain with respect to BoVW. Also, we show that systems using a limited number of descriptors and machine learning techniques can still be quite effective. Our first participation at the TRECVID evaluation has been very fruitful: our team was ranked 6 th in the light version of the Semantic Indexing task.

This paper gives an overview of approaches to video representation targeting semantic analysis for content-based indexing and retrieval. It highlights the major achieve-

Al-Azhar Bulletin of Science, 2020

Video semantic concepts exploration is a fundamental problem in a video indexing and retrieval. It has a long history of investigation since early days till recent achievements. The challenges lie in bridging the gap between low level features and semantic level ones. To stand on the thoroughly situation, video semantic concepts exploration for indexing and retrieval purposes evolution from conventional methods to the state-ofthe-art ones will be reviewed. The main contribution is to unify concepts involved and evolution in this interesting topic.