Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Materials and Methods
Participants
Experimental Setup
Subject Scoring
Objective Measures
Results
Discussions
Conclusions
References

Predicting Complexity Perception of Real World Images

Profile image of Gianluigi CioccaGianluigi Ciocca

2016, PLOS ONE

https://doi.org/10.1371/JOURNAL.PONE.0157986
visibility

…

description

22 pages

link

1 file

Abstract

The aim of this work is to predict the complexity perception of real world images. We propose a new complexity measure where different image features, based on spatial, frequency and color properties are linearly combined. In order to find the optimal set of weighting coefficients we have applied a Particle Swarm Optimization. The optimal linear combination is the one that best fits the subjective data obtained in an experiment where observers evaluate the complexity of real world scenes on a web-based interface. To test the proposed complexity measure we have performed a second experiment on a different database of real world scenes, where the linear combination previously obtained is correlated with the new subjective data. Our complexity measure outperforms not only each single visual feature but also two visual clutter measures frequently used in the literature to predict image complexity. To analyze the usefulness of our proposal, we have also considered two different sets of stimuli composed of real texture images. Tuning the parameters of our measure for this kind of stimuli, we have obtained a linear combination that still outperforms the single measures. In conclusion our measure, properly tuned, can predict complexity perception of different kind of images.

Related papers

Identifying the perceptual dimensions of visual complexity of scenes
Aude Oliva

2004

Abstract Scenes are composed of numerous objects, textures and colors which are arranged in a variety of spatial layouts. This presents the question of how visual complexity is represented by a cognitive system. In this paper, we aim to study the representation of visual complexity for real-world scene images. Is visual complexity a perceptual property simple enough so that it can be compressed along a unique perceptual dimension? Or is visual complexity better represented by a multi-dimensional space?

View PDFchevron_right
Testing a computational model of visual complexity in background scenes
Patrik Holt

This paper reports the results of experiments conducted to determine whether a mathematical/computational model reflects the perception of complexity of background image scenes. Background scenes, such as landscapes, are composed of non-discrete image data. They contain repetitive patterns and colour schemes and therefore the images have to be described using pattern analysis techniques. A brief outline is given regarding the difference in composition of the foreground and background in images. Complexity measures and theories are then explored and results of a pilot test and experiment reported which correlate human perceptual data with a pattern measure which was devised by . This measure had previously only been tested on binary data and not on more realistic image data. The results show a very strong, positive correlation (r=0.899, p<0.01) which seems to suggest that the human perception of complexity of non-determinate image data is modelled well by the pattern measure. The paper concludes by outlining potential implications of the work and uses of the model.

View PDFchevron_right
Linking Computerized and Perceived Attributes of Visual Complexity
Vered Aharonson

2020

Psychological studies explore visual complexity as perceived by humans. Image complexity is studied extensively in the mathematical, computational sciences. The two disciplines often define visual complexity differently and are thus disjointed. This is manifested in differences between subjective human-perceived complexity, and computer vision algorithms’ performance in visual tasks. Our study investigates this discrepancy in the context of cognitive tests that employ visual stimuli to assess a subject’s primal cognitive functions. A database of cognitive tests including visual recognition tasks and the performance of 403 subjects in terms of response times was used. Computer vision and information theory features were extracted from the images in these tasks. Inspired by cognitive psychology studies, the features were categorized into whole-image and object-specific features. A random forest classifier was used to map the computed features into three complexity labels in the tasks,...

View PDFchevron_right
Visual complexity assessment of painting images
Takio Kurita

2013 IEEE International Conference on Image Processing, 2013

In this paper, we propose a framework to assess visual complexity of paintings. This framework provides a machine learning scheme for investigating the relationship between human visual complexity perception and low-level image features. Since the global and local characteristics of paintings affect human's holistic impression and detail perception, we design a set of methods to extract the features that represent the global and local characteristics of paintings. By feature selection, we look into the role that each image feature plays in assessing visual complexity. Then the selected features are combined by a Support Vector Machine for classification. Experimental results indicate that the proposed work can predict the visual complexity perception of paintings with the accuracy of 88.13%, which is highly close to the assessments given by humans. Compared with the conventional measure of complexity, our approach considers human visual perception and performs more efficiently in assessing visual complexity of painting images.

View PDFchevron_right
Image complexity measure based on visual attention
Vincent Courboulay

2011 18th IEEE International Conference on Image Processing, 2011

Digital images can be analyzed at wide range of levels going from pixel arrangement to semantics. As a consequence, finding a visual complexity estimator is a difficult task. In this article we propose a definition of attention based perceptual complexity. We study the performance of human eye movements and of different models of computational attention against a ground-truth of image complexity based on the observation time of an image description task. The results obtained show that besides its lack of semantic processing, attentional behavior is a good estimator of image complexity.

View PDFchevron_right
Does Color Influence Image Complexity Perception?
Gianluigi Ciocca

Lecture Notes in Computer Science, 2015

In this paper we investigate if color influences the perception of image complexity. To this end we perform two different types of psycho-physical experiments on color and grayscale images. In the first experiment, images are ranked based on their complexity (image ranking), while in the second experiment the complexity of each image is assessed on a continuous scale (image scaling). Moreover, we investigate if ten image features, that measure colors as well as other spatial properties of the images, correlate with the collected subjective data. The performance of these correlations are evaluated in terms of Pearson correlation coefficients and Spearman rank-order correlation coefficients. We observe that for each type of experiment, subjective scores for color images are highly correlated with those of the corresponding grayscale versions suggesting that color is not a relevant attribute in evaluating image complexity. Moreover none of the tested simple image features seem to be adapt to predict the image complexity according to the human judgments.

View PDFchevron_right
Analysis of texture characteristics associated with visual complexity perception
Takio Kurita

Optical Review, 2012

In our previous work we determined that five important characteristics affect the perception of visual complexity of a texture: regularity, roughness, directionality, density, and understandability. In this paper, a set of objective methods for measuring these characteristics is proposed: regularity is estimated by an autocorrelation function; roughness is computed based on local changes; directionality is measured by the maximum line-likeness of edges in different directions; and density is calculated from the edge density. Our analysis shows a significant correlation between the objective measures and subjective evaluations. In addition, for the estimation of understandability, a new approach is proposed. We asked the respondents to name each texture, and then we sorted all these names into different types, including names that were similar. We discovered that understandability is affected by two factors of a texture: the maximum number of similar names assigned to a specific type and the total number of types.

View PDFchevron_right
A Review on Various Measures for Finding Image Complexity
DOLLEY SHUKLA

2017

The complexity of an image tells many aspects of the image content and is an important factor in the selection of source material for testing various image processing methods. Databases of pictures or videos explained with subjective evaluations constitute basic ground truth for preparing, testing, what&#39;s more, benchmarking calculations for target quality appraisal. We propose a few criteria for quantitative correlations of source substance, test conditions, and subjective evaluations, which are utilized as the reason for the resulting investigations and dialog. This review paper presents some of the existing image quality metrics through which image quality can be classified or measured.

View PDFchevron_right
Comparison of Outlier-Tolerant Models for Measuring Visual Complexity
Adrian Carballal

Entropy

Providing the visual complexity of an image in terms of impact or aesthetic preference can be of great applicability in areas such as psychology or marketing. To this end, certain areas such as Computer Vision have focused on identifying features and computational models that allow for satisfactory results. This paper studies the application of recent ML models using input images evaluated by humans and characterized by features related to visual complexity. According to the experiments carried out, it was confirmed that one of these methods, Correlation by Genetic Search (CGS), based on the search for minimum sets of features that maximize the correlation of the model with respect to the input data, predicted human ratings of image visual complexity better than any other model referenced to date in terms of correlation, RMSE or minimum number of features required by the model. In addition, the variability of these terms were studied eliminating images considered as outliers in prev...

View PDFchevron_right
Visual complexity modelling based on image features fusion of multiple kernels
Adrian Carballal

PeerJ

Humans’ perception of visual complexity is often regarded as one of the key principles of aesthetic order, and is intimately related to the physiological, neurological and, possibly, psychological characteristics of the human mind. For these reasons, creating accurate computational models of visual complexity is a demanding task. Building upon on previous work in the field (Forsythe et al., 2011; Machado et al., 2015) we explore the use of Machine Learning techniques to create computational models of visual complexity. For that purpose, we use a dataset composed of 800 visual stimuli divided into five categories, describing each stimulus by 329 features based on edge detection, compression error and Zipf’s law. In an initial stage, a comparative analysis of representative state-of-the-art Machine Learning approaches is performed. Subsequently, we conduct an exhaustive outlier analysis. We analyze the impact of removing the extreme outliers, concluding that Feature Selection Multiple...

View PDFchevron_right

References (56)

  1. Forsythe A, Sheehy N, Sawey M. Measuring icon complexity: An automated analysis. Behavior Research Methods, Instruments, & Computers. 2003; 35(2):334-342. doi: 10.3758/BF03202562
  2. Reinecke K, Yeh T, Miratrix L, Mardiko R, Zhao Y, Liu J, et al. Predicting users' first impressions of web- site aesthetics with a quantification of perceived visual complexity and colorfulness. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2013. p. 2049-2058.
  3. Ramanarayanan G, Bala K, Ferwerda JA, Walter B. Dimensionality of visual complexity in computer graphics scenes. In: Electronic Imaging 2008. International Society for Optics and Photonics; 2008. p. 68060E-68060E.
  4. Wei B, Guan T, Duan L, Yu J, Mao T. Wide area localization and tracking on camera phones for mobile augmented reality systems. Multimedia Systems. 2015; 21(4):381-399. doi: 10.1007/s00530-014- 0364-2
  5. Yaghmaee F, Jamzad M. Estimating watermarking capacity in gray scale images based on image com- plexity. EURASIP Journal on Advances in Signal Processing. 2010; 2010:8. doi: 10.1155/2010/851920
  6. Perkiö J, Hyvärinen A. Modelling image complexity by independent component analysis, with applica- tion to content-based image retrieval. In: Artificial Neural Networks-ICANN 2009. Springer; 2009. p. 704-714.
  7. Guan T, Wang Y, Duan L, Ji R. On-Device Mobile Landmark Recognition Using Binarized Descriptor with Multifeature Fusion. ACM Transactions on Intelligent Systems and Technology (TIST). 2015; 7 (1):12.
  8. Zhang Y, Guan T, Duan L, Wei B, Gao J, Mao T. Inertial sensors supported visual descriptors encoding and geometric verification for mobile visual location recognition applications. Signal Processing. 2015; 112:17-26. doi: 10.1016/j.sigpro.2014.08.029
  9. Wei B, Guan T, Yu J. Projected residual vector quantization for ANN search. MultiMedia, IEEE. 2014; 21(3):41-51. doi: 10.1109/MMUL.2013.65
  10. Huahui Wu MC, Kinicki R. A study of video motion and scene complexity. In: Tech. Rep. WPI-CS-TR- 06-19, Worcester Polytechnic Institute; 2006.
  11. Peters RA, Strickland RN. Image complexity metrics for automatic target recognizers. In: Automatic Target Recognizer System and Technology Conference; 1990. p. 1-17.
  12. Xu J, Jiang M, Wang S, Kankanhalli MS, Zhao Q. Predicting human gaze beyond pixels. Journal of vision. 2014; 14(1):28. doi: 10.1167/14.1.28 PMID: 24474825
  13. Zhao Q, Koch C. Advances in Learning Visual Saliency: From Image Primitives to Semantic Contents. In: Neural Computation, Neural Devices, and Neural Prosthesis. Springer; 2014. p. 335-360.
  14. Donderi DC. Visual complexity: a review. Psychological Bulletin. 2006; 132(1):73. doi: 10.1037/0033- 2909.132.1.73 PMID: 16435958
  15. Marin MM, Leder H. Examining Complexity across Domains: Relating Subjective and Objective Mea- sures of Affective Environmental Scenes, Paintings and Music. PLoS ONE. 2013; 8(8):e72412. doi: 10. 1371/journal.pone.0072412 PMID: 23977295
  16. Kolmogorov AN. Three approaches to the quantitative definition ofinformation'. Problems of information transmission. 1965; 1(1):1-7.
  17. Snodgrass JG, Vanderwart M. A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. Journal of experimental psychology: Human learning and memory. 1980; 6(2):174.
  18. Heaps C, Handel S. Similarity and features of natural textures. Journal of Experimental Psychology: Human Perception and Performance. 1999; 25(2):299.
  19. Birkhoff GD. Collected mathematical papers. New York; 1950.
  20. Forsythe A. Visual Complexity: Is That All There Is? In: Engineering Psychology and Cognitive Ergo- nomics. Springer; 2009. p. 158-166.
  21. Cardaci M, Di Gesù V, Petrou M, Tabacchi ME. On the evaluation of images complexity: A fuzzy approach. In: Fuzzy Logic and Applications. Springer; 2006. p. 305-311.
  22. Palumbo L, Ogden R, Makin AD, Bertamini M. Examining visual complexity and its influence on per- ceived duration. Journal of vision. 2014; 14(14):3. doi: 10.1167/14.14.3 PMID: 25487112
  23. Chacon MIM, Aguilar LED, Delgado AS. Fuzzy adaptive edge definition based on the complexity of the image. In: 10 th IEEE International Conference on Fuzzy Systems; 2001. p. 675-678.
  24. Mario I, Chacon M, Alma D, Corral S. Image complexity measure: a human criterion free approach. In: Fuzzy Information Processing Society, 2005. NAFIPS 2005. Annual Meeting of the North American. IEEE; 2005. p. 241-246.
  25. Yin K, Wang L, Guo Y. Fusing Multiple Visual Features for Image Complexity Evaluation. In: Advances in Multimedia Information Processing-PCM 2013. Springer; 2013. p. 308-317.
  26. Rigau J, Feixas M, Sbert M. An information-theoretic framework for image complexity. In: Proceedings of the First Eurographics conference on Computational Aesthetics in Graphics, Visualization and Imag- ing. Eurographics Association; 2005. p. 177-184.
  27. Rosenholtz R, Li Y, Nakano L. Measuring visual clutter. Journal of Vision. 2007; 7(2):17. doi: 10.1167/ 7.2.17 PMID: 18217832
  28. Mack M, Oliva A. Computational estimation of visual complexity. In: the 12th Annual Object, Perception, Attention, and Memory Conference; 2004.
  29. Chikhman V, Bondarko V, Danilova M, Goluzina A, Shelepin Y. Complexity of images: Experimental and computational estimates compared. Perception. 2012; 41:631-647. doi: 10.1068/p6987 PMID: 23094454
  30. Oliva A, Mack ML, Shrestha M. Identifying the Perceptual Dimensions of Visual Complexity of Scenes. In: Proc. 26th Annual Meeting of the Cognitive Science Society; 2004.
  31. Purchase HC, Freeman E, Hamer J. Predicting Visual Complexity. In: Proceedings of the 3rd Interna- tional Conference on Appearance, Edinburgh, UK; 2012. p. 62-65.
  32. Cavalcante A, Mansouri A, Kacha L, Barros AK, Takeuchi Y, Matsumoto N, et al. Measuring street- scape complexity based on the statistics of local contrast and spatial frequency. PLoS ONE. 2014; 9 (2). doi: 10.1371/journal.pone.0087097
  33. Ciocca G, Corchs S, Gasparini F, Bricolo E, Tebano R. Does color infuence image complexity percep- tion? In: Fifth IAPR Computational Color Imaging Workshop (CCIW'15). vol. 9016 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg; 2015. p.
  34. Guo X, Asano CM, Asano A, Kurita T, Li L. Analysis of texture characteristics associated with visual complexity perception. Optical review. 2012; 19(5):306-314. doi: 10.1007/s10043-012-0047-1
  35. Ciocca G, Corchs S, Gasparini F. Complexity Perception of Texture Images. In: Murino V, Puppo E, Sona D, Cristani M, Sansone C, editors. New Trends in Image Analysis and Processing-ICIAP 2015
  36. Workshops. vol. 9281 of Lecture Notes in Computer Science. Springer International Publishing; 2015. p. 119-126.
  37. Heaps C, Handel S. Similarity and features of natural textures. Journal of Experimental Psychology: Human Perception and Performance. 1999; 25(2):299.
  38. Guo X, Kurita T, Asano CM, Asano A. Visual complexity assessment of painting images. In: Image Pro- cessing (ICIP), 2013 20th IEEE International Conference on. IEEE; 2013. p. 388-392.
  39. J K, R E. Particle swarm optimization. In: Proc IEEE Int Conf Neural Networks. vol. 4; 1995. p. 1942- 1948.
  40. Bianco S, Schettini R. Two new von Kries based chromatic adaptation transforms found by numerical optimization. Color Research & Application. 2010; 35(3):184-192. doi: 10.1002/col.20573
  41. Yu CP, Samaras D, Zelinsky GJ. Modeling visual clutter perception using proto-object segmentation. Journal of vision. 2014; 14(7):4. doi: 10.1167/14.7.4 PMID: 24904121
  42. Sheik H, Wang Z, Cormakc L, Bovik A. In: LIVE Image Quality Assessment Database Release 2. http:// live.ece.utexas.edu/research/quality; 2006.
  43. Sheikh HR, Sabir MF, Bovik AC. A statistical evaluation of recent full reference image quality assess- ment algorithms. Image Processing, IEEE Transactions on. 2006; 15(11):3440-3451. doi: 10.1109/ TIP.2006.881959
  44. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to struc- tural similarity. Image Processing, IEEE Transactions on. 2004; 13(4):600-612. doi: 10.1109/TIP.2003. 819861
  45. Corchs S, Gasparini F, Schettini R. No Reference Image Quality classification for JPEG-Distorted Images. Digital Signal Processing. 2014; 30:86-100. doi: 10.1016/j.dsp.2014.04.003
  46. MIT Media Lab, Vision texture homepage, http://vismod.media.mit.edu/vismod/imagery/VisionTexture/.
  47. Cusano C, Napoletano P, Schettini R. Evaluating color texture descriptors under large variations of controlled lighting conditions. Journal of the Optical Society of America A. 2016 1; 33(1):17-30. doi: 10. 1364/JOSAA.33.000017
  48. Freedman DA. Statistical models: theory and practice. Cambridge University Press; 2009.
  49. Haralick RM, Shanmugam K, Dinstein IH. Textural features for image classification. Systems, Man and Cybernetics, IEEE Transactions on. 1973;( 6):610-621. doi: 10.1109/TSMC.1973.4309314
  50. Corchs S, Gasparini F, Schettini R. Grouping strategies to improve the correlation between subjective and objective image quality data. In: IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics; 2013. p. 86530D-86530D.
  51. Corchs S, Gasparini F, Schettini R. No reference image quality classification for JPEG-distorted images. Digital Signal Processing. 2014; 30:86-100. doi: 10.1016/j.dsp.2014.04.003
  52. Comaniciu D, Meer P. Mean shift: A robust approach toward feature space analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2002; 24(5):603-619. doi: 10.1109/34.1000236
  53. Hasler D, Suesstrunk SE. Measuring colorfulness in natural images. In: Electronic Imaging 2003. Inter- national Society for Optics and Photonics; 2003. p. 87-95.
  54. Artese MT, Ciocca G, Gagliardi I. Good 50x70 Project: A portal for Cultural And Social Campaigns. In: Archiving Conference. vol. 2014. Society for Imaging Science and Technology; 2014. p. 213-218.
  55. Solli M, Lenz R. Color harmony for image indexing. In: Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on. IEEE; 2009. p. 1885-1892.
  56. Brodatz P. Textures: a photographic album for artists and designers. Dover Pubns; 1966.

Related papers

Computerized measures of visual complexity
Penousal Machado

Acta Psychologica, 2015

Visual complexity influences people's perception of, preference for, and behaviour toward many classes of objects, from artworks to web pages. The ability to predict people's impression of the complexity of different kinds of visual stimuli holds, therefore, great potential for many domains, basic and applied. Here we use edge detection operations and several image metrics based on image compression error and Zipf's law to estimate the visual complexity of images. The experiments involved 800 images, each previously rated by thirty participants on perceived complexity. In a first set of experiments we analysed the correlation of individual features with the average human response, obtaining correlations up to r s = .771. In a second set of experiments we employed Machine Learning techniques to predict the average visual complexity score attributed by humans to each stimuli. The best configurations obtained a correlation of r s = .832. The average prediction error of the Machine Learning system over the set of all stimuli was .096 in a normalized 0 to 1 interval, showing that it is possible to predict, with high accuracy human responses. Overall, edge density and compression error were the strongest predictors of human complexity ratings.

View PDFchevron_right
Predicting Visual Complexity
John Hamer

Inspired by the contrast between &#39;classical&#39; and &#39;expressive&#39; visual aesthetic design, this paper explores the &#39;visual complexity&#39; of images. We wished to investigate whether the visual complexity of an image could be quantified so that it matched participants&#39; view of complexity. An empirical study was conducted to collect data on the human view of the complexity of a set of images. The results were then related to a set of computational metrics applied to these images, so as to identify which objective metrics best encapsulate the human subjective opinion. We conclude that the subjective notion of &#39;complexity&#39; is consistent both to an individual and to a group, but that it does not easily relate to the most obvious computational metrics.

View PDFchevron_right
Predicting human complexity perception of real-world scenes
Fintan Nagle

Royal Society Open Science

Perceptual load is a well-established determinant of attentional engagement in a task. So far, perceptual load has typically been manipulated by increasing either the number of task-relevant items or the perceptual processing demand (e.g. conjunction versus feature tasks). The tasks used often involved rather simple visual displays (e.g. letters or single objects). How can perceptual load be operationalized for richer, real-world images? A promising proxy is the visual complexity of an image. However, current predictive models for visual complexity have limited applicability to diverse real-world images. Here we modelled visual complexity using a deep convolutional neural network (CNN) trained to learn perceived ratings of visual complexity. We presented 53 observers with 4000 images from the PASCAL VOC dataset, obtaining 75 020 2-alternative forced choice paired comparisons across observers. Image visual complexity scores were obtained using the TrueSkill algorithm. A CNN with weig...

View PDFchevron_right
Modeling the Perception of Visual Complexity in Texture Images
Takio Kurita

International Journal of Affective Engineering, 2013

Perception of visual complexity in textures is very important for visual understanding and visual aesthetic evaluation. In this paper, we propose a new model of estimating subjective visual complexity perception of texture images. Compared with the traditional complexity measures based on information theory and fuzzy theory, the proposed model considers human visual perception, and it predicts the visual complexity of a texture corresponding to the subjective visual impression. Multiple linear regression (MLR) is used as a mapping function to map the relationship between the visual complexity perception and five texture characteristics including regularity, roughness, directionality, density and understandability. F-test and correlation analysis are applied to stimulated data and predicted data. The results of F-test (P < 0.01) prove that the proposed model can significantly predict the visual complexity of a texture, and the correlation coefficient between calculated complexity and subjective complexity (r = 0.951) of the testing textures shows that the results predicted by the proposed model are very close to the visual complexity judged by human subjects.

View PDFchevron_right
Visual Complexity Perception and Texture Image Characteristics
Takio Kurita

2011 International Conference on Biometrics and Kansei Engineering, 2011

Visual complexity perception is an important issue in the fields of psychology and computer vision because it leads to the better understanding of the nature of human perception as well as the properties of the objects being perceived.

View PDFchevron_right

Cited by

Visual complexity modelling based on image features fusion of multiple kernels
Adrian Carballal

PeerJ

Humans’ perception of visual complexity is often regarded as one of the key principles of aesthetic order, and is intimately related to the physiological, neurological and, possibly, psychological characteristics of the human mind. For these reasons, creating accurate computational models of visual complexity is a demanding task. Building upon on previous work in the field (Forsythe et al., 2011; Machado et al., 2015) we explore the use of Machine Learning techniques to create computational models of visual complexity. For that purpose, we use a dataset composed of 800 visual stimuli divided into five categories, describing each stimulus by 329 features based on edge detection, compression error and Zipf’s law. In an initial stage, a comparative analysis of representative state-of-the-art Machine Learning approaches is performed. Subsequently, we conduct an exhaustive outlier analysis. We analyze the impact of removing the extreme outliers, concluding that Feature Selection Multiple...

View PDFchevron_right
CROCUFID: A Cross-Cultural Food Image Database for Research on Food Elicited Affective Responses
Martin van Schaik

Frontiers in Psychology

We present CROCUFID: a CROss-CUltural Food Image Database that currently contains 840 images, including 479 food images with detailed metadata and 165 images of non-food items. The database includes images of sweet, savory, natural, and processed food from Western and Asian cuisines. To create sufficient variability in valence and arousal we included images of food with different degrees of appetitiveness (fresh, unfamiliar, molded or rotten, spoiled, and partly consumed). We used a standardized photographing protocol, resulting in high resolution images depicting all food items on a standard background (a white plate), seen from a fixed viewing (45 •) angle. CROCUFID is freely available under the CC-By Attribution 4.0 International license and hosted on the OSF repository. The advantages of the CROCUFID database over other databases are its (1) free availability, (2) full coverage of the valence-arousal space, (3) use of standardized recording methods, (4) inclusion of multiple cuisines and unfamiliar foods, (5) availability of normative and demographic data, (6) high image quality and (7) capability to support future (e.g., virtual and augmented reality) applications. Individuals from the United Kingdom (N = 266), North-America (N = 275), and Japan (N = 264) provided normative ratings of valence, arousal, perceived healthiness, and desire-to-eat using visual analog scales (VAS). In addition, for each image we computed 17 characteristics that are known to influence affective observer responses (e.g., texture, regularity, complexity, and colorfulness). Significant differences between groups and significant correlations between image characteristics and normative ratings were in accordance with previous research, indicating the validity of CROCUFID. We expect that CROCUFID will facilitate comparability across studies and advance experimental research on the determinants of food-elicited emotions. We plan to extend CROCUFID in the future with images of food from a wide range of different cuisines and with non-food images (for applications in for instance neuro-physiological studies). We invite researchers from all parts of the world to contribute to this effort by creating similar image sets that can be linked to this collection, so that CROCUFID will grow into a truly multicultural food database.

View PDFchevron_right
Predicting human complexity perception of real-world scenes
Fintan Nagle

Royal Society Open Science

Perceptual load is a well-established determinant of attentional engagement in a task. So far, perceptual load has typically been manipulated by increasing either the number of task-relevant items or the perceptual processing demand (e.g. conjunction versus feature tasks). The tasks used often involved rather simple visual displays (e.g. letters or single objects). How can perceptual load be operationalized for richer, real-world images? A promising proxy is the visual complexity of an image. However, current predictive models for visual complexity have limited applicability to diverse real-world images. Here we modelled visual complexity using a deep convolutional neural network (CNN) trained to learn perceived ratings of visual complexity. We presented 53 observers with 4000 images from the PASCAL VOC dataset, obtaining 75 020 2-alternative forced choice paired comparisons across observers. Image visual complexity scores were obtained using the TrueSkill algorithm. A CNN with weig...

View PDFchevron_right
Visual complexity analysis using deep intermediate-layer features
Mona Jalal

Computer Vision and Image Understanding, 2020

In this paper, we focus on visual complexity, an image attribute that humans can subjectively evaluate based on the level of details in the image. We explore unsupervised information extraction from intermediate convolutional layers of deep neural networks to measure visual complexity. We derive an activation energy metric that combines convolutional layer activations to quantify visual complexity. To show the effectiveness of our proposed metric for various applications, we introduce Savoias, a visual complexity dataset that compromises of more than 1,400 images from seven diverse image categories (e.g., advertisement and interior design). We demonstrate high correlations of our deep neural network-based measure of visual complexity with human-curated ground-truth (GT) scores on various widely used network architectures, e.g., VGG16, ResNet-v2-152, and EfficientNet, and in networks trained on two classification tasks (object and scene classification). This result reveals that intermediate convolutional layers of deep neural networks carry information about the complexity of images that is meaningful to people. Furthermore, we show that our method of measuring visual complexity outperforms traditional methods on Savoias and two other state-of-the-art benchmark datasets. Moreover, we perform extensive analysis on the performance difference between our unsupervised method and a supervised method trained on the feature map, and show that by supervision, we can improve the prediction. Finally, we demonstrate that, within the context of a category, visually more complex images are also more memorable to human observers.

View PDFchevron_right
Color Image Complexity versus Over-Segmentation: A Preliminary Study on the Correlation between Complexity Measures and Number of Segments
Emilia Nicolae

Journal of Imaging, 2020

It is said that image segmentation is a very difficult or complex task. First of all, we emphasize the subtle difference between the notions of difficulty and complexity. Then, in this article, we focus on the question of how two widely used color image complexity measures correlate with the number of segments resulting in over-segmentation. We study the evolution of both the image complexity measures and number of segments as the image complexity is gradually decreased by means of low-pass filtering. In this way, we tackle the possibility of predicting the difficulty of color image segmentation based on image complexity measures. We analyze the complexity of images from the point of view of color entropy and color fractal dimension and for color fractal images and the Berkeley data set we correlate these two metrics with the segmentation results, more specifically the number of quasi-flat zones and the number of JSEG regions in the resulting segmentation map. We report on our exper...

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved