PDH: a human-centric interface for image libraries

2017

Finding images or videos in multimedia collections is a difficult task. Many collections only have metadata such as filenames or timestamps, and no other information is available. To augment this, we can employ content based analysis techniques that provides extra content based metadata. This provides a good starting point, but the accuracy is often insufficient to automate full collection categorization. A human in the loop is essential to aid with search and categorization. In this thesis we evaluate how to retrieve elements from multimedia collections for a variety of retrieval tasks. We investigate different user interfaces that extend content based retrieval methods with novel user interface techniques. In one interface, MediaTable, we focus on categorization tasks by leveraging table-style user interfaces with images so users can investigate both the multimedia content and associated metadata at the same time. Users can categorize elements by placing them in buckets, and we pe...

Journal of Visual Languages and Computing, 2008

Image collections are getting larger and larger. To access those collections, systems for managing, searching, and browsing are necessary. Visualization plays an essential role in such systems. Existing visualization systems do not analyze all the problems occurring when dealing with large visual collections. In this paper, we make these problems explicit. From there, we establish three general requirements: overview, visibility, and structure preservation. Solutions for each requirement are proposed, as well as functions balancing the different requirements. We present an optimal visualization scheme, supporting users in interacting with large image collections. Experimental results with a collection of 10,000 Corel images, using simulated user actions, show that the proposed scheme significantly improves performance for a given task compared to the 2D grid-based visualizations commonly used in content-based image retrieval. r (M. Worring). museum archives, and to scientific pictures in medicine, astronomy, or biology. Hence, large image collections are common everywhere.

2009

Managing photos by using visual features (e.g., color and texture) is known to be a powerful, yet imprecise, retrieval paradigm because of the semantic gap problem. The same is true if search relies only on keywords (or tags), derived from either the image context or user-provided annotations. In this paper we present a new multi-faceted image search and browsing system, named Scenique, that allows the user to manage her photo collections by using both visual features and tags, possibly organized into multiple dimensions (or facets). Each facet can be seen as a coordinate of a multidimensional space describing the image content (for example, the visual appearance, the content type, the geographic location, and so on). We present the basic principles of Scenique and provide evidence of the effectiveness of its visual tools. Feedback supplied by a set of real users indicates that the proposed interface is intuitive, easy to use, and that satisfies users' expectations in managing photo collections and quickly locating images of interest.

Electronic Workshops in Computing, 2017

The volume of images available online has increased significantly but the choices offered by image retrieval systems have not kept pace. We describe the design and evaluation of a 'high density' image search interface focusing on the results pages. We quantitatively and qualitatively compared image presentation on a high density interface with a traditional image search interface. Our results show that users had major problems with high-density interfaces for images due to information overload. We point to further work which could improve the user experience.

2004

This paper proposes a novel touch-based multiuser interaction environment for browsing images from the Internet. A large touchable screen serves as a shared data space where multiple users simultaneously interact with Internet images by using simple hands movements and gestures. Users can select, bookmark, clean up and refine the search of the incoming images. The system provides a natural and easy-to-use interface to browse and select items from large databases, with uses varying from discussion and brain storming support to edutainment and fun. Early preliminary results show the suitability of our image interactive visualization method for browsing large image collections.

2001

Human-computer interaction is a decisive factor in effective content-based access to large image repositories. In current image retrieval systems the user refines his query by selecting example images from a relevance ranking. Since the top ranked images are all similar, user feedback often results in rearrangement of the presented images only. For better incorporation of user interaction in the retrieval process, we have developed the Filter Image Browsing method. It also uses feedback through image selection. However, it is based on differences between images rather than similarities. Filter Image Browsing presents overviews of relevant parts of the database to users. Through interaction users then zoom in on parts of the image collection. By repeatedly limiting the information space, the user quickly ends up with a small amount of relevant images. The method can easily be extended for the retrieval of multimedia objects. For evaluation of the Filter Image Browsing retrieval concept, a user simulation is applied to a pictorial database containing 10,000 images acquired from the World Wide Web by a search robot. The simulation incorporates uncertainty in the definition of the information need by users. Results show Filter Image Browsing outperforms plain interactive similarity ranking in required effort from the user. Also, the method produces predictable results for retrieval sessions, so that the user quickly knows if a successful session is possible at all. Furthermore, the simulations show the overview techniques are suited for applications such as hand-held devices where screen space is limited.

Proceedings of the Working Conference on Advanced Visual Interfaces - AVI '02, 2002

The Personal Digital Historian (PDH) is an ongoing research project aimed at allowing groups of people to casually browse, embellish, and explore large collections of their personal data, such as pictures, video, or more business-related items such as spreadsheets or PowerPoint slides. Our initial prototype system is designed for a tabletop display and to be used while people are talking to each other. In this paper, we focus exclusively on describing those aspects of our project which provide a visual interface to support exploration of a database of personal data. The interface allows people to organize their images along the four questions essential to storytelling: who?, when?, where?, and what? Users are provided with a wide variety of flexible interaction methods, including region of interest query specification with in-place freeform stroke input, image-based book marking, suggestion generation via automatic query relaxation, and output summarization. With this interface, the users can enjoy their conversation while having the photos at their finger tips, rather than being distracted by the effort of formulating queries.

IEEE Transactions on Visualization and Computer Graphics, 2000

Social photos, which are taken during family events or parties, represent individuals or groups of people. We show in this paper how a Hasse diagram is an efficient visualization strategy for eliciting different groups and navigating through them. However, we do not limit this strategy to these traditional uses. Instead we show how it can also be used for assisting in indexing new photos. Indexing consists of identifying the event and people in photos. It is an integral phase that takes place before searching and sharing. In our method we use existing indexed photos to index new photos. This is performed through a manual drag and drop procedure followed by a content fusion process that we call 'propagation'. At the core of this process is the necessity to organize and visualize the photos that will be used for indexing in a manner that is easily recognizable and accessible by the user. In this respect we make use of an Object Galois Sub-Hierarchy and display it using a Hasse diagram. The need for an incremental display that maintains the user's mental map also leads us to propose a novel way of building the Hasse diagram. To validate the approach, we present some tests conducted with a sample of users that confirm the interest of this organization, visualization and indexation approach. Finally, we conclude by considering scalability, the possibility to extract social networks and automatically create personalised albums.

The field of content based image retrieval has experienced a lot of advances. These are evidenced by the number of research publications that present new techniques of image feature representation, image indexing and similarity estimation. While all these are important core techniques, they remain methods for implementing systems. There is a relatively less emphasis on the role of user interfaces through which the users and the image retrieval systems interact. In this work, we present an overview of the work that has been carried out in the area of user interfaces for image retrieval systems. Finally we present some recommendations for the direction of research in the field of designing image retrieval system interfaces.

2003

Content-Based Image Retrieval (CBIR) presents several challenges and has been subject to extensive research from many domains, such as image processing or database systems. Database researchers are concerned with indexing and querying, whereas image processing experts worry about extracting appropriate image descriptors. Comparatively little work has been done on designing user interfaces for CBIR systems. This, in turn, has a profound effect on these systems since the concept of image similarity is strongly influenced by user perception. This paper describes an initial effort to fill this gap, combining recent research in CBIR and Information Visualization, studied from a Human-Computer Interface perspective. It presents two visualization techniques based on Spiral and Concentric Rings implemented in a CBIR system to explore query results. The approach is centered on keeping user focus on both the query image, and the most similar retrieved images. Experiments conducted so far suggest that the proposed visualization strategies improves system usability.