Review: Automatic Semantic Image Annotation

As the number of Web images is increasing at a rapid rate, searching them semantically presents a significant challenge. Many raw images are constantly uploaded with few meaningful direct annotations of semantic content, limiting their search and discovery. In this paper, a semantic annotation technique based on the use of image parametric dimensions and metadata is presented. Using SVM classifier and rule induction, a rule-based approach is developed to formulate explicit annotations for images fully automatically, so that by the use of this method, a semantic query can be answered. Experimental results indicate that this approach is able to deliver a highly competent performance, attaining good recall and precision rates. This approach enables a new degree of semantic richness to be automatically associated with images which previously could only be performed manually.

In the recent years, multimedia content has grown increasingly over the Internet, especially in social networks, where users often post images using their mobile devices. In these networks such as Flickr, the content is later used in search operations when some users want to find something using a specific query. Nowadays, searching into these networks is primarily made using the title and the keywords associated to resources added by users that have posted the content. The problem we face comes from the fact that in many cases, the title or the related keywords are not relevant to the resource and only after we analyse the image, can we conclude what it contains in reality. The project that we want to present in this article proposes that each image is connected to relevant keywords according to its content. In order to do this, the first step was to create a collection of images that was annotated by human annotators, while the second step was to expand this collection of images performing search on the Internet using keywords associated to the initial collection of annotated images. Currently, for a new picture, we can identify similar images in our collection of images and based on the keywords associated with them, we can determine what keywords characterize this new image. The evaluation of this system has demonstrated that our approach works efficiently for images for which we can find similar images in our collection.

Multimedia Tools and Applications, 2011

Personal memories composed of digital pictures are very popular at the moment. To retrieve these media items annotation is required. During the last years, several approaches have been proposed in order to overcome the image annotation problem. This paper presents our proposals to address this problem. Automatic and semi-automatic learning methods for semantic concepts are presented. The automatic method is based on semantic concepts estimated using visual content, context metadata and audio information. The semi-automatic method is based on results provided by a computer game. The paper describes our proposals and presents their evaluations.

2013

Automatic image annotation is the process of assigning keywords to digital images depending on the content information. Automatically assigning keywords to images is of great interest as it allows one to index, retrieve, and understand large collections of image data. Selection & implementation of proper feature extraction and weight calculation method plays important role in image annotation. Many techniques have been proposed for image annotation in the last decade that gives reasonable performance on standard datasets. This paper presents detailed literature about the Automatic Image Annotation and explains the feature extraction as well as weight calculation method.

2007

This paper addresses the challenge of automatic annotation of images for semantic image retrieval. In this research, we aim to identify visual features that are suitable for semantic annotation tasks. We propose an image classification system that combines MPEG-7 visual descriptors and support vector machines. The system is applied to annotate cityscape and landscape images. For this task, our analysis shows that the colour structure and edge histogram descriptors perform best, compared to a wide range of MPEG-7 visual descriptors. On a dataset of 7200 landscape and cityscape images representing real-life varied quality and resolution, the MPEG-7 colour structure descriptor and edge histogram descriptor achieve a classification rate of 82.8% and 84.6%, respectively. By combining these two features, we are able to achieve a classification rate of 89.7%. Our results demonstrate that combining salient features can significantly improve classification of images.

2007 IEEE International Conference on Signal Processing and Communications, 2007

The focus of this paper is on automatic annotation for semantic image retrieval. This work is aimed at identifying visual descriptors that are most relevant, effective and suitable for semantic annotation tasks. We propose an image annotation system based on support vector machines and a combination of descriptors that includes a gradient direction histogram and several MPEG-7 visual descriptors. The system is tested on a large database of 7200 cityscape and landscape images. The results indicate that when descriptors are used individually, the proposed gradient direction histogram performs best. However, when descriptors are combined, the accuracy is improved. The presented results confirm that combining the gradient direction histogram and colour structure produces the best results.

International Journal of Multimedia Information Retrieval, 2020

Image annotation is the process of assigning metadata to images, allowing effective retrieval by text-based search techniques. Despite the lots of efforts in automatic multimedia analysis, automatic semantic annotation of multimedia is still inefficient due to the problems in modelling high level semantic terms. In this paper we examine the factors affecting the quality of annotations collected through crowdsourcing platforms. An image dataset was manually annotated utilizing: (i) a vocabulary consists of pre-selected set of keywords, (ii) an hierarchical vocabulary, and (iii) free keywords. The results show that the annotation quality is affected by the image content itself and the used lexicon. As we expected while annotation using the hierarchical vocabulary is more representative, the use of free keywords leads to increased invalid annotation. Finally it is shown that images requiring annotations that are not directly related to their content (i.e. annotation using abstract concepts), lead to accrue annotator inconsistency revealing in that way the difficulty in annotating such kind of images is not limited to automatic annotation, but it is generic problem of annotation.

The Institution of Engineering and Technology - Image Processing, 2019

The massive amount of digital content generated daily in the modern world has created the need for an image retrieval system built on image analysis via image processing and machine learning, therefore this study explains the role of machine learning in bridging the semantic gap in content-based image retrieval, proposes an automatic image annotation framework, in which training images are obtained from social media, and semantic indexing is achieved using a combination of supervised and unsupervised machine learning. Furthermore, the study also highlights the need for continuous vocabulary improvement for optimum system performance and recommends hardware implementation of machine learning algorithms to ensure high overall speed of image retrieval systems.

IET Image Processing

The massive amount of digital content generated daily in the modern world has created the need for an image retrieval system built on image analysis via image processing and machine learning, therefore this study explains the role of machine learning in bridging the semantic gap in content-based image retrieval, proposes an automatic image annotation framework, in which training images are obtained from social media, and semantic indexing is achieved using a combination of supervised and unsupervised machine learning. Furthermore, the study also highlights the need for continuous vocabulary improvement for optimum system performance and recommends hardware implementation of machine learning algorithms to ensure high overall speed of image retrieval systems.