Supervised multimedia categorization

IEEE Computer Graphics and Applications, 2000

Many multimedia collections include only metadata such as date created and file size and remain largely unannotated. So, browsing them is cumbersome. Automatic content-analysis techniques yield metadata in the form of high-level content-based descriptors. However, these techniques' accuracy is insufficient to automate collection categorization. A human is essential to validate and organize automated techniques' results. MediaTable helps users efficiently categorize

IEEE Multimedia, 2002

2007

There has been a growing demand for effective access to video information from media archives in recent years. Personalized video retrieval is one of the most challenging issues and has spurred a significant interest in many research communities. In this paper, a novel approach is proposed to achieve personalized retrieval of sports video, which includes two research tasks: semantic annotation of sports video and acquisition of user's preference. For semantic annotation, a multi-modal framework is employed to detect sports event and index the sports video content. Web-casting text, as external information, is utilized to detect semantic events in sport videos. The semantic concepts and keywords included in the webcasting text are extracted to annotate and index the sport event segments automatically. For user's preference acquisition, relevance feedback is applied to model user's preference and non-preference, and re-ranking is used to refine the results. First, the user is asked to label some video segments as desirable and undesirable. Then, we use these labels to infer the user's interesting points (e.g. the player, the event type, the team, etc.) by analysis of text keywords; the lowlevel video features are also adopted as a supplementary to reflect the user's preference. The overall new rank of the results is the combination of the user's high-level and low-level preference. Experiments conducted on real-world soccer game videos show that the proposed method has an encouraging performance.

International Journal of Advanced Media and Communication, 2014

We present a learning-based approach to the semantic indexing of multimedia content using cues derived from audio, visual, and text features. We approach the problem by developing a set of statistical models for a predefined lexicon. Novel concepts are then mapped in terms of the concepts in the lexicon. To achieve robust detection of concepts, we exploit features from multiple modalities, namely, audio, video, and text. Concept representations are modeled using Gaussian mixture models (GMM), hidden Markov models (HMM), and support vector machines (SVM). Models such as Bayesian networks and SVMs are used in a latefusion approach to model concepts that are not explicitly modeled in terms of features. Our experiments indicate promise in the proposed classification and fusion methodologies: our proposed fusion scheme achieves more than 10% relative improvement over the best unimodal concept detector.

Multimedia Tools and Applications, 2008

In this paper we present a framework for unified, personalized access to heterogeneous multimedia content in distributed repositories. Focusing on semantic analysis of multimedia documents, metadata, user queries and user profiles, it contributes to the bridging of the gap between the semantic nature of user queries and raw multimedia documents. The proposed approach utilizes as input visual content analysis results, as well as analyzes and exploits associated textual annotation, in order to extract the underlying semantics, construct a semantic index and classify documents to topics, based on a unified knowledge and semantics representation model. It may then accept user queries, and, carrying out semantic interpretation and expansion, retrieve documents from the index and rank them according to user preferences, similarly to text retrieval. All processes are based on a novel semantic processing methodology, employing fuzzy algebra and principles of taxonomic knowledge representation. The first part of this work presented in this paper deals with data and knowledge models, manipulation of multimedia content annotations and semantic indexing, while the second part will continue on the use of the extracted semantic information for personalized retrieval.

Multimedia Data Mining and Knowledge Discovery, 2007

Menno Israël, Egon L. van den Broek, Peter van der Putten, Marten J. den Uyl Summary. The work presented here introduces a real time automatic scene classifier within content-based video retrieval. In our envisioned approach end users like documentalists, not image processing experts, build classifiers interactively, by simply indicating positive examples of a scene. Classification consists of a two stage procedure. First, small image fragments called patches are classified. Second, frequency vectors of these patch classifications are fed into a second classifier for global scene classification (e.g., city, portraits, or countryside). The first stage classifiers can be seen as a set of highly specialized, learned feature detectors, as an alternative to letting an image processing expert determine features a priori. The end user or domain expert thus builds a visual alphabet that can be used to describe the image in features that are relevant for the task at hand. We present results for experiments on a variety of patch and image classes. The scene classifier approach has been successfully applied to other domains of video content analysis, such as content based video retrieval in television archives, automated sewer inspection, and porn filtering.

Social media prevails in every walk of our life. Multimedia shared through social networks has tremendously increased the need for efficient retrieval methods and expects more accuracy in terms of annotating an existing image or video. Retrieval methods and annotation techniques are two sides in the development of an efficient multimedia retrieval system. Annotating the image and video is a challenging task. Collaborative user annotations can be incorporated into multimedia to increase the efficiency and accuracy in the retrieval methods. Collaborative user annotations are useful for two reasons: (i) Multi-label annotation for a multimedia is possible with less time-consuming even for large-scale image corpus (ii) Correlation between images and videos build a multi-class label propagation without much human effort and in reduced cost. There are many areas of research the collaborative annotations are incorporated with small modification in the existing machine learning algorithms. This survey paper presents the state-of-the-art annotation techniques for multimedia in the new era.

2013 11th International Workshop on Content-Based Multimedia Indexing (CBMI), 2013

In this paper we propose an in-depth evaluation of the performance of video descriptors to multimodal video genre categorization. We discuss the perspective of designing appropriate late fusion techniques that would enable to attain very high categorization accuracy, close to the one achieved with user-based text information. Evaluation is carried out in the context of the 2012 Video Genre Tagging Task of the MediaEval Benchmarking Initiative for Multimedia Evaluation, using a data set of up to 15.000 videos (3,200 hours of footage) and 26 video genre categories specific to web media. Results show that the proposed approach significantly improves genre categorization performance, outperforming other existing approaches. The main contribution of this paper is in the experimental part, several valuable interesting findings are reported that motivate further research on video genre classification.

2007

Abstract. The task of multimedia document categorization forms a well-known problem in information retrieval. The task is to assign a multimedia document to one or more categories, based on its contents. In this case, effective management and thematic categorization requires usually the extraction of the underlying semantics.

2008

Manual annotation is both impractical and very expensive, due to the vast amount of data generated at a rapid rate A novel framework is presented for semantic labeling of by such videos. However, automatic annotation is a very video clips, automatically segmented from broadcast video demanding and an extremely challenging computer vision task of soccer (football) games, as highlights and excitement as it involves high-level scene interpretation. In , authors clips etc. The proposed framework provides a generalizable presented a web-casting text based annotation scheme. In method for linking low-level video features with high- [1], authors proposed Finite State Machine based annotation level semantic concepts defined in a commonly understood of soccer video. Barnard et. al. proposed [3] HMM based sports lexicon. Three important contributions are made to framework to fuse audio and video features to recognize the automatic annotation of sports video, as follows. First, domain play and break scenes in soccer video sequences. Li et. al. knowledge combined with an event-lexicon and a four-level proposed rule based algorithm using low-level audio/video hierarchical classifier based on low-level video features is features for football video summarization. Babaguchi et. used to label video segments. Second, a priori event mining al. proposed event detection by recognizing the textual is used to establish probabilistic event-associations that are overlays from football video. used to assign a concept-lexicon, such as goals and saves, to There have been many successful works in soccer video each highlight video segment. And, finally, the collection of analysis as mentioned above. But most of these works fail to highlight video clips is summarized using concept-and eventrespond to action-based queries, such as "extract the goal clips lexicons to facilitate highlight browsing, video skimming, out of this soccer sequence", or "extract the saves from this indexing and retrieval. soccer video", "extract goals scored by team-A", "extract all the red card events from the collection of FIFA 2006 world 1 Introduction cup matches", etc. At higher level, user may ask the specific queries such as, "extract the replay segment from the goal clip