Academia.eduAcademia.edu

Outline

Title
Abstract
Figures
Introduction
Overview
Conclusion
References
All Topics
Computer Science
Computer Vision

Bridging the Semantic Gap in Content Based Image Retrieval

Profile image of IJCSMC JournalIJCSMC Journal

2018, IJCSMC

Abstract

Image content on the Web is increasing exponentially. As a result, there is a need for image retrieval systems. Historically, there have been two methodologies, text-based and content-based. In the text-based approach, query systems retrieve images that have been manually annotated using key words. This approach can be problematic: it is labor-intensive and maybe biased according to the subjectivity of the observer. Content based image retrieval (CBIR) searches and retrieves digital images in large databases by analysis of derived-image features. CBIR systems typically use the characteristics of color, texture, shape and their combination for definition of features. Similarity measures that originated in the preceding text-based era are commonly used. However, CBIR struggles with bridging the semantic gap, defined as the division between high-level complexity of CBIR and human perception and the low-level implementation features and techniques. In this paper, CBIR is reviewed in a broad context. Newer approaches is feature generation and similarity measures are detailed with representative studies addressing their efficacy. Color-texture moments, columns-of-interest, harmonysymmetry-geometry, SIFT (Scale Invariant Feature Transform), and SURF (Speeded Up Robust Features) are presented as alternative feature generation modalities. Graph matching, Earth Mover’s Distance, and relevance feedback are discussed with the realm of similarity. We conclude that while CBIR is evolving and continues to slowly close the semantic gap, addressing the complexity of human perception remains a challenge.

Figures (6)
Thereafter, during the retrieval process, the end user selects the visual features and may also specify weights for the representations. Based on selected features and weights, the retrieval system will find similar images for the user’s query. meaningful collections of features. Features are commonly represented as histograms and signatures. A histogram is derived from a fixed partitioning of the domain of the distribution. A single feature histogram is generated when each entry of the histogram corresponds to the number of its features located in the corresponding global partition and usually represent bins in a fixed partitioning of the region of the underlying space. Local clustering of features generates a signature, also called an adaptive binning histogram. Clusters of the objects’ features are included with the corresponding weights and centroids.° Intuitively, a feature signature is then a set of centroids with a corresponding weight of the clusters. Each cluster is represented by a statistical measure such as mean and/or by the fraction of pixels within a spatial cluster. It is argued that each feature signature reflects the feature distribution more meaningfully than any feature histogram. Another critical pre-processing step is segmentation step that describes image content by focusing on regions of interest in an attempt to identify the core meaningful regions.*"°
Thereafter, during the retrieval process, the end user selects the visual features and may also specify weights for the representations. Based on selected features and weights, the retrieval system will find similar images for the user’s query. meaningful collections of features. Features are commonly represented as histograms and signatures. A histogram is derived from a fixed partitioning of the domain of the distribution. A single feature histogram is generated when each entry of the histogram corresponds to the number of its features located in the corresponding global partition and usually represent bins in a fixed partitioning of the region of the underlying space. Local clustering of features generates a signature, also called an adaptive binning histogram. Clusters of the objects’ features are included with the corresponding weights and centroids.° Intuitively, a feature signature is then a set of centroids with a corresponding weight of the clusters. Each cluster is represented by a statistical measure such as mean and/or by the fraction of pixels within a spatial cluster. It is argued that each feature signature reflects the feature distribution more meaningfully than any feature histogram. Another critical pre-processing step is segmentation step that describes image content by focusing on regions of interest in an attempt to identify the core meaningful regions.*"°
Figure 5. Performance of different distance measures on MPEG7 Set B."® The authors found, that, in general, Euclidean distance, city block distance and Chi square statistics are the most desirable distance measurers in terms of retrieval effectiveness and efficiency. These two papers are a sample of the large number of papers that are representative of the high variability in functional accuracy of applying “standard” distance measures to the field of CBIR. LA ar mffLlraypt tr AA; rnlaAan altanrnatniszn Aiatannan RrARrAMAIIKRA Ee Tran nmatinntarn haven database consists of 7,200 images classified into 100 different classes, the MIR Flickr contains 25,000 images, while the 101 Objects Database contains 9,196 images classified into 101 categories. The authors then use themes, classes, textual annotations and categories to measure precision and recall values. The extracted feature signatures exist in feature spaces containing to seven dimensions. They then evaluate the performance of various similarity measures on color only, color plus texture, color plus position, and color plus position plus texture. Among the various databases queried, the precision varied between 0.315 and 0.790 and computation times range from 10,003 to 1329 msec. They conclude, in general, that the signature quadratic form distance exhibits the highest mean average precision values, while the Hausdorff distance and Perceptually Modified Hausdorff Distance (PMHD) exhibit the lowest computational time values. 4Q
Figure 5. Performance of different distance measures on MPEG7 Set B."® The authors found, that, in general, Euclidean distance, city block distance and Chi square statistics are the most desirable distance measurers in terms of retrieval effectiveness and efficiency. These two papers are a sample of the large number of papers that are representative of the high variability in functional accuracy of applying “standard” distance measures to the field of CBIR. LA ar mffLlraypt tr AA; rnlaAan altanrnatniszn Aiatannan RrARrAMAIIKRA Ee Tran nmatinntarn haven database consists of 7,200 images classified into 100 different classes, the MIR Flickr contains 25,000 images, while the 101 Objects Database contains 9,196 images classified into 101 categories. The authors then use themes, classes, textual annotations and categories to measure precision and recall values. The extracted feature signatures exist in feature spaces containing to seven dimensions. They then evaluate the performance of various similarity measures on color only, color plus texture, color plus position, and color plus position plus texture. Among the various databases queried, the precision varied between 0.315 and 0.790 and computation times range from 10,003 to 1329 msec. They conclude, in general, that the signature quadratic form distance exhibits the highest mean average precision values, while the Hausdorff distance and Perceptually Modified Hausdorff Distance (PMHD) exhibit the lowest computational time values. 4Q

Related papers

Unifying Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web
Marco La Cascia

Computer Vision and Image Understanding, 1999

A system is proposed that combines textual and visual statistics in a single index vector for content-based search of a WWW image database. Textual statistics are captured in vector form using latent semantic indexing based on text in the containing HTML document. Visual statistics are captured in vector form using color and orientation histograms. By using an integrated approach, it becomes possible to take advantage of possible statistical couplings between the content of the document (latent semantic content) and the contents of images (visual statistics). The combined approach allows improved performance in conducting content-based search. Search performance experiments are reported for a database containing 350,000 images collected from the WWW. c 1999 Academic Press Key Words: content-based image database indexing and retrieval; vector space representation; latent semantic indexing; world wide web search engines.

View PDFchevron_right
Survey of the Adequate Descriptor for Content-Based Image Retrieval on the Web: Global versus Local Features
lobna hlaoua

Départements des sciences d'informatiques,

View PDFchevron_right
Comparison Study Between Image Retrieval Methods
Dr. Ayad A. Ani

Iraqi journal of information and communication technology, 2022

Searching for a relevant image in an archive is a problematic research issue for the computer vision research community. The majority of search engines retrieve images using traditional text-based approaches that rely on captions and metadata. Extensive research has been reported in the last two decades for content-based image retrieval (CBIR), analysis, and image classification. Content-Based Image Retrieval is a process that provides a framework for image search, and low-level visual features are commonly used to retrieve the images from the image database. The essential requirement in any image retrieval process is to sort the images with a close similarity in terms of visual appearance. The shape, color, and texture are examples of low-level image features. In image classification-based models and CBIR, high-level image visuals are expressed in the form of feature vectors made up of numerical values. The researcher's findings that there is a significant gap between image feature representation and human visual understanding. Due to this reason, the research presented in this area is focused to reduce the semantic gap between the image feature representation and human visual understanding. In this paper, we plan to present a comparison study and a comprehensive overview of the recent developments in the field of CBIR and image representation. We analyzed the main aspects of various image retrieval and image representation models. The important concepts and major research studies based on CBIR and image representation are discussed in detail, and future research directions are concluded to inspire further research in this area.

View PDFchevron_right
A Study on Content-Based Image Retrieval
Reema Reema

International Journal of Trend in Scientific Research and Development, 2018

The development of Internet causes an eruptive expansion of digital images, and also gives people more ways to get those images. Because the dissemination of video and image data in digital form has grown, Content Based Image Retrieval (CBIR) has become an eminent research topic. The importance of an effective technique in searching and retrieving images from the huge collection cannot be overemphasized. Therefore an important problem that needs to be addressed is fast retrieval of images from large databases. To perceive images that are perceptually similar to a query image, image retrieval systems attempt to search through a database. Content-based image retrieval utilizes representations of features that are automatically extracted from the images themselves.

View PDFchevron_right
A Review of Different Content Based Image Retrieval Techniques
Asmita Moghe

2017

Content-based image retrieval (CBIR) is the application of computer vision techniques to the image retrieval downside, that is, the matter of finding out digital images in massive databases. CBIR applies to techniques for retrieving similar images from image databases based on automated feature extraction methods. In this paper we’ve analyzed different techniques of content-based image retrieval. The development of CBIR systems involves analysis on databases, image processing and handling problems that vary from storage issues to friendly user interfaces. The accuracy of retrieval has subsequently increased with the use of low level features such as color, texture, shape etc. From many research activities performed on CBIR system shows that low-level image features cannot always describe highlevel semantic concepts within the users mind because, using low level features only does not include human perception. If human perception is allowed in the image retrieval system the efficienc...

View PDFchevron_right
Combining visual features with semantics for a more effective image retrieval
Djemel Ziou, Mohammed Lamine Kherfi

Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., 2004

View PDFchevron_right
A Survey on Content Based Image Retrieval Methods and their Combinations of Features
Ijrdt Journal

Abstract - Image retrieval has been one of the most interesting and vivid research areas in the field of computer vision over the last decades. Content-Based Image Retrieval (CBIR) systems are used in order to automatically index, search, retrieve, and browse image databases. Color, texture and shape information have been the primitive image descriptors in content based image retrieval systems. Content Based Image Retrieval (CBIR) is a technique which uses feature extraction and we extract and execute all the low level feature color features, texture features and shape features and their methods. User need to give a query image and similarity measurement engine which is responsible in estimating the similarity between the query image and database images and then ranking. of images according to similarity to the input query image and reduces computational time for image retrieval by using MapReduce for distributed dataset as well as increases precision of image by combining all three feature extraction techniques. We have compared different techniques as well as the combinations of them to improve the performance. We have also compared the effect of different matching techniques on the retrieval process.

View PDFchevron_right
Combining textual and visual cues for content-based image retrieval on the World Wide Web
Marco La Cascia

2002

A system is proposed that combines textual and visual statistics in a single index vector for content-based search of a WWW image database. Textual statistics are captured in vector form using latent semantic indexing (LSI) based on text in the containing HTML document. Visual statistics are captured in vector form using color and orientation histograms. By using an integrated approach, it becomes possible to take advantage of possible statistical couplings between the content of the document (latent semantic content) and the contents of images (visual statistics). The combined approach allows improved performance in conducting content-based search. Search performance experiments are reported for a database containing 100,000 images collected from the WWW.

View PDFchevron_right
An Effective Combined Feature For Web Based Image Retrieval
Rivikantha Herath

Technology advances as well as the emergence of large scale multimedia applications and the revolution of the World Wide Web has changed the world into a digital age. Anybody can use their mobile phone to take a photo at any time anywhere and upload that image to ever growing image databases. Development of effective techniques for visual and multimedia retrieval systems is one of the most challenging and important directions of the future research. This paper proposes an effective combined feature for web based image retrieval. Frequently used colour and texture features are explored in order to develop a combined feature for this purpose. Widely used three colour features: Colour moments, Colour coherence vector and Colour Correlogram, and three texture features: Grey Level Co-occurrence matrix, Tamura features and Gabor filter were analyzed for their performance. Precision and Recall were used to evaluate the performance of each of these techniques. By comparing precision and recall values the methods that performed best were taken and combined to form a hybrid feature. The developed combined feature was evaluated by developing a web based CBIR system. A web crawler was used to first crawl through Web sites and images found in those sites are downloaded and the combined feature representation technique was used to extract image features. The test results indicated that this web system can be used to index web images with the combined feature representation schema and to find similar images. Random image retrievals using the web system shows that the combined feature can be used to retrieve images belonging to the general image domain. Accuracy of the retrieval can be noted high for natural images like outdoor scenes, images of flowers etc. Also, images which have a similar colour and texture distribution were retrieved as similar, even though the images were belonging to deferent semantic categories. This can be ideal for an artist who wants to retrieve images which are aesthetically similar and not interested in semantic similarity.

View PDFchevron_right
Content-Based Image Retrieval: Survey
Mudassar Raza

Journal of Engineering and Sustainable Development

The requirement for development of CBIR is enhanced due to tremendous growth in volume of images as well as the widespread application in multiple fields. Texture, color, shape and spatial layout are the underlying traits to represent and index the images. These peculiar features of images are extracted and implemented for a similarity check among images. The problem of content based image retrieval is based on generation of peculiar query. For relevant images that meet their information need, an automated search is initiated by drawing a sketch or with the submission of image having similar features. Similarity between extracted features can be measured by using different algorithms. The use of relevance feedback as a post retrieval step enhances the optimization of the process. The necessity to explore the ever growing volume of image and video is motivating the development of efficient CBIR algorithms. Different algorithms and models for the retrieval of images have been explored over the last twenty years. In this paper an analysis of visual contents of image is done with respect to features related to low level after extracting from image that are color, texture and shape. Here most popular algorithms of feature extraction and relevance feedback that try to bridge extracted low level features and features with high level semantics gap from image are discussed. A brief overview of an algorithm has been presented which is based on fuzzy logic and is used for the selection of peculiar features.

View PDFchevron_right

References (20)

  1. Murala S, Maheshwari R, Balasubramanian R. Local tetra patterns: a new feature descriptor for content-based image retrieval. Image Processing, IEEE Transactions on. 2012;21(5):2874-2886.
  2. Rafiee G, Dlay SS, Woo WL. A review of content-based image retrieval. Paper presented at: Communication Systems Networks and Digital Signal Processing (CSNDSP), 2010 7th International Symposium on2010.
  3. Müller H, Michoux N, Bandon D, Geissbuhler A. A review of content-based image retrieval systems in medical applications-clinical benefits and future directions. International journal of medical informatics. 2004;73(1):1-23.
  4. Akgül CB, Rubin DL, Napel S, Beaulieu CF, Greenspan H, Acar B. Content-based image retrieval in radiology: current status and future directions. Journal of Digital Imaging. 2011;24(2):208-222.
  5. Rehman M, Iqbal M, Sharif M, Raza M. Content based image retrieval: survey. World Applied Sciences Journal. 2012;19(3):404-412.
  6. Eidenberger H, Breiteneder C. Semantic feature layers in content-based image retrieval: implementation of human world features. Paper presented at: Control, Automation, Robotics and Vision, 2002. ICARCV 2002. 7th International Conference on2002.
  7. Smeulders AW, Worring M, Santini S, Gupta A, Jain R. Content-based image retrieval at the end of the early years. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2000;22(12):1349-1380.
  8. Zhang D, Islam MM, Lu G. A review on automatic image annotation techniques. Pattern Recognition. 2012;45(1):346-362.
  9. Beecks C, Uysal MS, Seidl T. A comparative study of similarity measures for content-based multimedia retrieval. Paper presented at: Multimedia and Expo (ICME), 2010 IEEE International Conference on2010.
  10. Rubner Y, Tomasi C, Guibas LJ. The earth mover's distance as a metric for image retrieval. International journal of computer vision. 2000;40(2):99-121.
  11. Yu H, Li M, Zhang H-J, Feng J. Color texture moments for content-based image retrieval. Paper presented at: Image Processing. 2002. Proceedings. 2002 International Conference on2002.
  12. Tamura H, Mori S, Yamawaki T. Textural features corresponding to visual perception. Systems, Man and Cybernetics, IEEE Transactions on. 1978;8(6):460-473.
  13. Kim J, Cai W, Feng D, Wu H. A new way for multidimensional medical data management: volume of interest (VOI)-based retrieval of medical images with visual and functional features. Information Technology in Biomedicine, IEEE Transactions on. 2006;10(3):598-607.
  14. Grigorova A, De Natale FG, Dagli C, Huang TS. Content-based image retrieval by feature adaptation and relevance feedback. Multimedia, IEEE Transactions on. 2007;9(6):1183-1192.
  15. Collins J, Okada K. A Comparative Study of Similarity Measures for Content-Based Medical Image Retrieval. Paper presented at: CLEF (Online Working Notes/Labs/Workshop)2012.
  16. Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded-up robust features (SURF). Computer vision and image understanding. 2008;110(3):346-359.
  17. Juan L, Gwun O. A comparison of sift, pca-sift and surf. International Journal of Image Processing (IJIP). 2009;3(4):143-152.
  18. Zhang D, Lu G. Evaluation of similarity measurement for image retrieval. Paper presented at: Neural Networks and Signal Processing, 2003. Proceedings of the 2003 International Conference on2003.
  19. Bunke H. Graph matching: Theoretical foundations, algorithms, and applications. Paper presented at: Proc. Vision Interface2000.
  20. Vailaya A, Figueiredo MA, Jain AK, Zhang H-J. Image classification for content-based indexing. Image Processing, IEEE Transactions on. 2001;10(1):117-130.

Related papers

Content-based image retrieval
Margaret Graham

1999

View PDFchevron_right
An analysis of content-based image retrieval
Manish Dixit

International Advanced Researches and Engineering Journal, 2021

View PDFchevron_right
Content-Based Image Retrieval Features: A Survey
Muhammad Alyas Shahid

International Journal of Advanced Networking Applications, 2018

Content-Based Image Retrieval (CBIR) systems have been used for the searching of relevant images in various research areas. In CBIR systems features such as shape, texture and color are used. The extraction of features is the main step on which the retrieval results depend. Color features in CBIR are used as in the color histogram, color moments, conventional color correlogram and color histogram. Color space selection is used to represent the information of color of the pixels of the query image. The shape is the basic characteristic of segmented regions of an image. Different methods are introduced for better retrieval using different shape representation techniques; earlier the global shape representations were used but with time moved towards local shape representations. The local shape is more related to the expressing of result instead of the method. Local shape features may be derived from the texture properties and the color derivatives. Texture features have been used for images of documents, segmentation-based recognition,and satellite images. Texture features are used in different CBIR systems along with color, shape, geometrical structure and sift features.

View PDFchevron_right
Efficiency of Similarity Measures in Content based Retrieval Using Texture, Shape and Color
Poornima Raikar

International Journal of Advanced Trends in Computer Science and Engineering, 2019

Content based image retrieval (CBIR) provides efficient way to retrieve the images from the databases. The feature extraction and similar attribute measures are the key factors for retrieval performance. We need efficient way to access the visual content from large database. Content based image retrieval (CBIR) provides the solution for efficient retrieval of images from large image database. In this work hybrid feature based CBIR system is proposed with comparison of various distance measures. Spatial features like color histogram, color auto-correlogram, color moments, HSV histogram features and Frequency domain features like Semantic image features Gabor wavelet mean entropy, amplitude, energy. Wavelet moments like mean and the standard deviation of the transform coefficients, Shape feature histogram of oriented gradient , Hu moments are used to form the feature vector. The experiments are performed on flower database which consists of 1360 images from 17 different classes. For our experiment we have chosen 3 different classes of flowers of same color consisting of 100 images each. Experimental result shows that the proposed approach performs better in terms of precision, recall, accuracy of classifier and similarity measures .Shape feature play an prominent role for images with same color, texture. We made the comparison of efficiency of different similarity measures like Mahalanobis , Euclidean, Correlation, Spearman, City block(Manhattan) distances approaches on different images based on color, texture, shape features and found which distance measure is best based on performance.

View PDFchevron_right
Text-based, Content-based, and Semantic-based Image Retrievals: A Survey
Mohammed Al-khawlani, Hazem El-bakry, Mohammed M Elmogy

Image retrieval from databases or from the Internet needs an efficient and effective technique due to the explosive growth of digital images. Image retrieval is considered as an area of extensive research, especially in content based image retrieval (CBIR). CBIR retrieves similar images from large image database based on image features, which has been a very active research area recently. The content, that can be derived from image such as color, texture, shape…etc., are called features. This paper will present a survey and discuss the current literature of different types of image retrieval (IR) systems. An overview of the important techniques in image retrieval will be discussed. Finally, some urgent challenges in IR, that have been raised recently, will be presented as well as possible directions for future research.

View PDFchevron_right

Related topics

  • Computer Science
  • Image Processing
  • Content-Based Information Retrie...
  • Image Features Extraction
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved