Spatially organized visualization of image query results

Electronic Workshops in Computing, 2017

The volume of images available online has increased significantly but the choices offered by image retrieval systems have not kept pace. We describe the design and evaluation of a 'high density' image search interface focusing on the results pages. We quantitatively and qualitatively compared image presentation on a high density interface with a traditional image search interface. Our results show that users had major problems with high-density interfaces for images due to information overload. We point to further work which could improve the user experience.

Lecture Notes in Computer Science, 2010

This paper presents a novel approach for searching images online using textual queries and presenting the resulting images based on both conceptual and visual similarities. Given a user-specified query, the algorithm first finds the related concepts through conceptual query expansion. Each concept, together with the original query, is then used to search for images using existing image search engines. All the images found under different concepts are presented on a 2D virtual canvas using a self-organizing map. Both conceptual and visual similarities among the images are used to determine the image locations so that images from the same or related concepts are grouped together and visually similar images are placed close to each other. When the user browses the search results, a subset of representative images is selected to compose an image collage. Once having identified images of interest within the collage, the user can find more images that are conceptually or visually similar through pan and zoom operations. Experiments on different image query examples demonstrate the effectiveness of the presented approach.

SPIE International Society for Optical Engineering Internet Imaging II Conference, (eds. Beretta and Schettini), pp. 91-101., 2001

The web is enormous and constantly growing. User-interfaces for web-based applications need to make it easy for people to access relevant information without becoming overwhelmed or disoriented. Today's interfaces employ textual representations almost exclusively, typically organized in lists and hierarchies of web-page titles or URL taxonomies. Given the ability of images to assist memory and our frequent exploitation of space in everyday problem solving to simplify choice, perception, and mental computation, it is surprising that so little use is made of images and spatial organizations in accessing and organizing web information. The work we summarize in this paper suggests that spatial and temporal organization of selectable images may offer multiple advantages over textual lists of titles and URLs. We describe several image-based applications, detail basic image representation techniques, and discuss spatial and temporal strategies for organization.

The proliferation of content-based image retrieval techniques has highlighted the need to understand the relationship between image clustering based on low-Ievel image features and image clustering made by human users. In conventional image retrieval systems, images are typically characterized by a range offeatures such as color, texture, and shape. However, little is known to what extent these low-Ievel features can be effectively combined with information visualization techniques such that users may explore images in a digital library according to visual similarities. In this article, we compared and analyzed a number of Pathfinder networks of images generated based on such features. Salient structures of images are visualized according to features extracted .from color, texture, and shape orientation. Implications for visualizing and constructing hypermedia systems are discussed.

Lecture Notes in Computer Science, 2003

Effective opportunistic search in visual interfaces for large information collections, such as documents, web pages, and annotated images, often requires domain knowledge that the information seeker does not have. Relevant semantic information, such as metadata and especially ontologies, are valuable resources for helping the user learn this domain knowledge. In this paper we describe an interactive "focus + context" visualization technique for integrating this semantic information into the representation of the information collection to support effective opportunistic search. We have implemented a prototype using these techniques that integrates WordNet, a general lexical ontology, and a large collection of professionally annotated images. As the user explores the collection, the prototype dynamically changes the visual emphasis and detail of the images and keywords to reflect the relevant semantic relationships. This interaction allows the user to rapidly learn and use the domain knowledge required for effective opportunistic search.

International Journal of Information Management

Recent years have seen a huge increase in the digital data generated by companies, projects and individuals. This has led to significant challenges in visualizing and using large, diverse collections of digital information. Indeed the speed and accuracy with which these large datasets can be effectively mined for information that is relevant and valuable can have a significant effect on company performance. This research investigates the feasibility of using visual representations for the searching and browsing of large, complex, multimedia data sets. This paper introduces the SIZL (Searching for Information in a Zoom Landscape) system, which was developed to enable the authors to effectively test whether a 2.5D graphical representation of a multimedia data landscape produces quantifiable improvements in a user's ability to assess its contents. The usability of this visualization system was analyzed using experiments and a combination of quantitative and qualitative data collection methods. The paper presents these results and discusses potential industrial applications as well as future work that will improve the SIZL data visualization method.

… , 2000. Proceedings. IEEE …, 2000

The proliferation of content-based image retrieval techniques has highlighted the need to understand the relationship between image clustering based on low-Ievel image features and image clustering made by human users. In conventional image retrieval systems, images are typically characterized by a range offeatures such as color, texture, and shape. However, little is known to what extent these low-Ievel features can be effectively combined with information visualization techniques such that users may explore images in a digital library according to visual similarities. In this article, we compared and analyzed a number of Pathfinder networks of images generated based on such features. Salient structures of images are visualized according to features extracted .from color, texture, and shape orientation. Implications for visualizing and constructing hypermedia systems are discussed.

International Journal of Computer Vision, 2004

We present a user-centric system for visualization and layout for content-based image retrieval. Image features (visual and/or semantic) are used to display retrievals as thumbnails in a 2-D spatial layout or “configuration” which conveys all pair-wise mutual similarities. A graphical optimization technique is used to provide maximally uncluttered and informative layouts. Moreover, a novel subspace feature weighting technique can be used to modify 2-D layouts in a variety of context-dependent ways. An efficient computational technique for subspace weighting and re-estimation leads to a simple user-modeling framework whereby the system can learn to display query results based on layout examples (or relevance feedback) provided by the user. The resulting retrieval, browsing and visualization can adapt to the user's (time-varying) notions of content, context and preferences in style and interactive navigation. Monte Carlo simulations with machine-generated layouts as well as pilot user studies have demonstrated the ability of this framework to model or “mimic” users, by automatically generating layouts according to their preferences.

2009

Intuitive interfaces have become increasingly important multimedia applications, from personal photo collection to professional management systems. This paper presents a novel intuitive interactive interface for browsing of large image collections that visualizes underlying structure of the dataset by its size and spatial relations. In order to achieve this, images are initially clustered using an unsupervised graph-based clustering algorithm. By selecting images in a hierarchical layout of the screen, user can intuitively navigate through the collection. The experimental results demonstrate a significant speed-up in a content search scenario compared to a standard browsing interface, as well as inherent intuitiveness of the system.

IEEE Multimedia Magazine, 1997

New visual information in the form of images, graphics, animations, and videos is published on the Web at an incredible rate. However, cataloging it exceeds the capabilities of current text-based Web search engines. WebSeek provides a complete system that collects visual information from the Web by automated agents, then catalogs and indexes it for fast search and retrieval. M any search engines index the plethora of documents on the World Wide Web. For example, recent systems by Lycos, Alta Vista, Infoseek, and Excite index Web documents by their textual content. These systems periodically scan the Web, analyze the documents, and create compact and searchable indexes. Users enter query terms and/or select subjects to more easily find the desired Web documents. A large component of the content on the Web consists of visual information such as images, graphics, animations, and videos. The visual information on the Web proves highly volatile. Web site developers are constantly adding, removing, and replacing images and videos. Cataloging them requires a highly efficient automated system that regularly traverses the Web, detects visual information, and processes it and indexes it in such a way as to allow efficient and effective search and retrieval. We developed WebSeek, a prototype image and video search engine located at http://www. ctr.columbia.edu/webseek to fulfill this need. 1 The system collects images and videos from the Web, catalogs them, and provides tools for searching and browsing. WebSeek uses text and visual information synergically to catalog and search. The complete system possesses several powerful functionalities. It automates collection of visual information from the Web, automates image and video subject classification, performs searches using innovative visual content-based techniques, conducts image and video subject searches and navigations, and text-based searches, compresses images and videos for displaying query results,