Web Image Search Re-ranking Dependent on Diversity

Proceedings of the 2nd International Workshop on Web Intelligence, 2013

Recently, extensive research efforts have been dedicated to tag-based social image search which enables users to formulate their queries using tags. However, tag queries are often ambiguous and typically short yielding to retrieve irrelevant images in top ranked list. To overcome this problem, an effective strategy is to produce diverse images in top ranking list covering various aspects of the query. In this context, we propose a Multi-view Concept-based Query Expansion (MVCQE) process, using a predefined list of semantic concepts and following three main steps. First, we harvest social knowledge to capture different contexts related to the query. Second, we perform a Multi-view Concepts weighting by applying concept-based query expansion for the initial query and for each of its contexts. Third, we select the most representative concepts using an adaptive threshold with respect to the dispersion of concept weights. Experiments using ambiguous queries over the NUS-WIDE dataset confirm the effectiveness of our process to improve the diversification compared to well known query expansion approaches . . .

2016 IEEE International Symposium on Multimedia (ISM), 2016

Multimedia searches often return items that can be categorized into several "topics", allowing users to disambiguate and explore answers more efficiently. In this paper we investigate methods for clustering tags associated with multimedia search results, where each resulting cluster represents a topic computed online for that particular search. We specifically investigate the applicability of community detection algorithms to the tag graph induced from the search results. This type of approach allows us to exploit tag similarity and create ad-hoc topics for each search, without specify the number and sizes of clusters a priori. In this work we experiment with well-known algorithms in this field and propose two new methods based on adaptive island cuts. Using the Social20 dataset (a collection gathered from Flickr) we evaluate several community detection methods, with quantitative analysis of each algorithm in terms of the relative number of communities (which we interpret as topics) that they produce and their sizes, as well as qualitative analysis of topics per human judgement. Our evaluation shows that it is possible to extract ad-hoc topics for search results using community detection, but that different community detection methods produce very different results. In particular, our proposed methods produce more compact and less noisy clusters as well as less relative recall when compared to methods that produce much larger clusters.

In recent years, there has been a proliferation of collaborative tagging systems in Web 2.0 communities. With the increasingly large amount of social data, how to manage and organize them becomes an important and crucial problem for folksonomy applications. To better understand and meet users’ needs, multimedia resources can be organized or indexed from these user perspectives; it is thus important to find latent user communities for social media applications. In this paper, we propose the mechanism of augmented folksonomy graph (AFG) to incorporate multi-faceted relations in social media, along with a novel density-based clustering method to discover latent user community from AFG by combining contents and tags of multimedia resources. To evaluate the proposed method, we conduct experiments on a public dataset, the empirical results of which show that our approach outperforms baseline ones in terms of tag-based and content-based personalized search.

2011

Tag-based retrieval of multimedia content is a difficult problem, not only because of the shorter length of tags associated with images and videos, but also due to mismatch in the terminologies used by searcher and content creator. To alleviate this problem, we propose a simple concept-driven probabilistic model for improving text-based rich-media search. While our approach is similar to existing topic-based retrieval and cluster-based language modeling work, there are two important differences: (1) our proposed model considers not only the query-generation likelihood from cluster, but explicitly accounts for the overall "popularity" of the cluster or underlying concept, and (2) we explore the possibility of inferring the likely concept relevant to a rich-media content through the user-created communities that the content belongs to.

Multimedia Tools and Applications, 2014

This article addresses the diversification of image retrieval results in the context of image retrieval from social media. It proposes a benchmarking framework together with an annotated dataset and discusses the results achieved during the related task run in the MediaEval 2013 benchmark. 38 multimedia diversification systems, varying from graph-based representations, re-ranking, optimization approaches, data clustering to hybrid approaches that included a human in the loop, and their results are described and analyzed in this text. A comparison of the use of expert vs. crowdsourcing annotations shows that crowdsourcing results have a slightly lower inter-rater agreement but results are comparable at a much lower cost than expert annotators. Multimodal approaches have best results in terms of cluster recall. Manual approaches can lead to high precision but often lower diversity. With this detailed results analysis we give future insights into diversity in image retrieval and also for preparing new evaluation campaigns in related areas.

Proceedings of the 10th annual joint conference on Digital libraries - JCDL '10, 2010

In search engines, ranking algorithms measure the importance and relevance of documents mainly based on the contents and relationships between documents. User attributes are usually not considered in ranking. This user-neutral approach, however, may not meet the diverse interests of users, who may demand different documents even with the same queries. To satisfy this need for more personalized ranking, we propose a ranking framework, Social Network Document Rank (SNDocRank), that considers both document contents and the relationship between a searcher and document owners in a social network. This method combines the traditional tf-idf ranking for document contents with our Multi-level Actor Similarity (MAS) algorithm to measure to what extent document owners and the searcher are structurally similar in a social network. We implemented our ranking method in a simulated video social network based on data extracted from YouTube and tested its effectiveness on video search. The results show that compared with the traditional ranking method like tf-idf, the SNDocRank algorithm returns more relevant documents. More specifically, a searcher can get significantly better results by being in a larger social network, having more friends, and being associated with larger local communities in a social network.

2014 IEEE International Conference on Image Processing (ICIP), 2014

This article addresses the issue of retrieval result diversification in the context of social image retrieval and discusses the results achieved during the MediaEval 2013 benchmarking. 38 runs and their results are described and analyzed in this text. A comparison of the use of expert vs. crowdsourcing annotations shows that crowdsourcing results are slightly different and have higher inter observer differences but results are comparable at lower cost. Multimodal approaches have best results in terms of cluster recall. Manual approaches can lead to high precision but often lower diversity. With this detailed results analysis we give future insights on this matter.

2011

Users can rarely reveal their information need in full detail to a search engine within 1-2 words, so search engines need to "hedge their bets" and present diverse results within the precious 10 response slots. Diversity in ranking is of much recent interest. Most existing solutions estimate the marginal utility of an item given a set of items already in the response, and then use variants of greedy set cover. Others design graphs with the items as nodes and choose diverse items based on visit rates (PageRank). Here we introduce a radically new and natural formulation of diversity as finding centers in resistive graphs. Unlike in PageRank, we do not specify the edge resistances (equivalently, conductances) and ask for node visit rates. Instead, we look for a sparse set of center nodes so that the effective conductance from the center to the rest of the graph has maximum entropy. We give a cogent semantic justification for turning PageRankthus on its head. In marked deviation from prior work, our edge resistances are learnt from training data. Inference and learning are NP-hard, but we give practical solutions. In extensive experiments with subtopic retrieval, social network search, and document summarization, our approach convincingly surpasses recently-published diversity algorithms like subtopic cover, max-marginal relevance (MMR), Grasshopper, DivRank, and SVMdiv.

The explosive growth of digital videos/photos, the prevalence of capture devices, and the phenomenal success in WWW search have helped attract increasing interest in investigating new solutions in image/video search and navigation. This interest is gradually extended to the novel domain of community-contributed multimedia as the growing practice of online public media sharing. Billions of images shared on websites such as Flickr 1 bring profound social impact to the human society, and pose a new challenge for the design of efficient indexing, searching and visualizing methods for manipulating them.

Proceedings of the VLDB Endowment, 2008

The popularity of collaborative tagging sites presents a unique opportunity to explore keyword search in a context where query results are determined by the opinion of a network of taggers related to a seeker. In this paper, we present the first in-depth study of network-aware search. We investigate efficient top-k processing when the score of an answer is computed as its popularity among members of a seeker's network. We argue that obvious adaptations of top-k algorithms are too space-intensive, due to the dependence of scores on the seeker's network. We therefore develop algorithms based on maintaining score upper-bounds. The global upper-bound approach maintains a single score upper-bound for every pair of item and tag, over the entire collection of users. The resulting bounds are very coarse. We thus investigate clustering seekers based on similar behavior of their networks. We show that finding the optimal clustering of seekers is intractable, but we provide heuristic methods that give substantial time improvements. We then give an optimization that can benefit smaller populations of seekers based on clustering of taggers. Our results are supported by extensive experiments on del.icio.us datasets.