580 California St., Suite 400
San Francisco, CA, 94104
Key research themes
1. How can content-based visual features be effectively used to overcome the semantic gap in multimedia indexing and retrieval?
This theme addresses the challenge of bridging the semantic gap between low-level visual features extracted from multimedia content and high-level human semantic queries. It investigates methods for combining multiple visual features to improve retrieval accuracy and relevance, emphasizing content-based video and image retrieval techniques.
Key finding: The paper demonstrates that employing multiple low-level visual features including color histograms, texture features, and shape in video indexing and retrieval improves discrimination and search tasks, contributing to... Read more
Key finding: Introduces color correlograms that encode spatial correlation of colors in images, outperforming traditional global color histograms and histogram refinement methods for image indexing and retrieval. This approach is robust... Read more
Key finding: Presents a system (HPQS) integrating natural language querying with semantic analysis and fuzzy data fusion for multimedia content-based retrieval that goes beyond low-level features by interpreting textual and image content... Read more
Key finding: Finds that combining multiple automatically-extracted visual and auditory features supports user queries in video retrieval but user adjustments of feature weights do not significantly improve performance. Keywords and... Read more
Key finding: Beyond employing multi-feature extraction, the study emphasizes including human visual perception aspects in designing video retrieval systems to better interpret semantic content, by modeling intellectual and emotional... Read more
2. What scalable algorithms and data structures enable efficient large-scale multimedia indexing and searching?
Focused on the computational challenges in indexing and retrieving multimedia data at scale, this research theme explores data structures like permutation-based indexing and algorithmic advances aimed at optimizing indexing time and retrieval speeds, including GPU and multi-core processing for large datasets.
Key finding: Proposes the Metric Permutation Table (MPT) data structure for efficient approximate nearest neighbor search in very large multimedia databases, with indexing based on permutations relative to reference objects. By... Read more
Key finding: Introduces a linear-time, scalable algorithm for attribute selection that uses fractal dimension to approximate intrinsic data dimensionality, enabling fast identification and elimination of redundant or irrelevant features... Read more
Key finding: Presents a variant of the SALSA link-analysis ranking algorithm called sNorm(p), which incorporates vector norms to better balance hub and authority scores in web page ranking, improving ranking effectiveness metrics such as... Read more
3. How can multimodal and semantic-rich user interfaces and query systems enhance multimedia search experience?
This theme examines the integration of multimodal inputs—such as natural language queries, images, and low-level visual features—to improve search accuracy and user experience in multimedia retrieval. It covers the fusion of semantic metadata, user interaction components, and adaptive query mechanisms to contextualize multimedia search.
Key finding: Demonstrates a comprehensive natural language interface enabling users to submit semantically rich queries which are transformed into formal retrieval requests. This multimodal approach combines semantic analysis of text,... Read more
Key finding: Evaluates the impact of interactive interfaces that allow users to adjust weights on multiple automatically-extracted features for video search. Findings indicate that some semantic features (e.g., Indoors/Outdoors) are more... Read more
Key finding: Proposes a novel method using music taste and audio feature extraction integrated with machine learning for user similarity detection and social network formation. The system incorporates complex audio features beyond FFT,... Read more