Assessing the contribution of color in visual attention

Frontiers in Human Neuroscience, 2023

Many visual attention models have been presented to obtain the saliency of a scene, i.e., the visually significant parts of a scene. However, some mechanisms are still not taken into account in these models, and the models do not fit the human data accurately. These mechanisms include which visual features are informative enough to be incorporated into the model, how the conspicuity of di erent features and scales of an image may integrate to obtain the saliency map of the image, and how the structure of an image a ects the strategy of our attention system. We integrate such mechanisms in the presented model more e ciently compared to previous models. First, besides low-level features commonly employed in state-of-the-art models, we also apply medium-level features as the combination of orientations and colors based on the visual system behavior. Second, we use a variable number of center-surround di erence maps instead of the fixed number used in the other models, suggesting that human visual attention operates di erently for diverse images with di erent structures. Third, we integrate the information of di erent scales and di erent features based on their weighted sum, defining the weights according to each component's contribution, and presenting both the local and global saliency of the image. To test the model's performance in fitting human data, we compared it to other models using the CAT dataset and the Area Under Curve (AUC) metric. Our results show that the model has high performance compared to the other models (AUC =. and sAUC = .) and suggest that the proposed mechanisms can be applied to the existing models to improve them.

2004

The saliency-based or bottom-up model of visual attention presented in this paper deals with still color images. The model we built is based on numerous properties of the human visual system (HVS), thus providing a biologically plausible system. The computation of early visual features such as color and orientation is a key step for any bottom-up model and the way to extract these visual features easily permits to differentiate a model from an other. The novelty of the proposed approach lies on the fact that the computation of early visual features is fully based on a HVS model consisting in projecting the picture into an opponent-colors space, applying a perceptual decomposition, contrast sensitivity and masking functions. Moreover, a strategy essentially based on a center surround mechanism and on the perceptual grouping phenomena underscores conspicuous locations by combining visual feature maps. A saliency map which is defined as a 2D topographic representation of conspicuity is then deduced. The model is applied to a number of natural images. Our results are then compared with the results of a well-know bottom-up model.

Journal of the Optical Society of America A, 2010

In this paper, computational methods are proposed to compute color edge saliency based on the information content of color edges. The computational methods are evaluated on bottom-up saliency in a psychophysical experiment, and on a more complex task of salient object detection in real-world images. The psychophysical experiment demonstrates the relevance of using information theory as a saliency processing model and that the proposed methods are significantly better in predicting color saliency (with a human-method correspondence up to 74.75% and an observer agreement of 86.8%) than state-of-the-art models. Furthermore, results from salient object detection confirm that an early fusion of color and contrast provide accurate performance to compute visual saliency with a hit rate up to 95.2%.

SPIE Proceedings, 2004

The saliency-based or bottom-up model of visual attention presented in this paper deals with still color images. The model we built is based on numerous properties of the human visual system (HVS), thus providing a biologically plausible system. The computation of early visual features such as color and orientation is a key step for any bottom-up model and the way to extract these visual features easily permits to differentiate a model from an other. The novelty of the proposed approach lies on the fact that the computation of early visual features is fully based on a HVS model consisting in projecting the picture into an opponent-colors space, applying a perceptual decomposition, contrast sensitivity and masking functions. Moreover, a strategy essentially based on a center surround mechanism and on the perceptual grouping phenomena underscores conspicuous locations by combining visual feature maps. A saliency map which is defined as a 2D topographic representation of conspicuity is then deduced. The model is applied to a number of natural images. Our results are then compared with the results of a well-know bottom-up model.

2008

Certain locations attract human gaze in natural visual scenes. Are there measurable features, which distinguish these locations from others? While there has been extensive research on luminance-defined features, only few studies have examined the influence of color on overt attention. In this study, we addressed this question by presenting color-calibrated stimuli and analyzing color features that are known to be relevant for the responses of LGN neurons. We recorded eye movements of 15 human subjects freely viewing colored and grayscale images of seven different categories. All images were also analyzed by the saliency map model (L. Itti, C. Koch, & E. Niebur, 1998). We find that human fixation locations differ between colored and grayscale versions of the same image much more than predicted by the saliency map. Examining the influence of various color features on overt attention, we find two extreme categories: while in rainforest images all color features are salient, none is salient in fractals. In all other categories, color features are selectively salient. This shows that the influence of color on overt attention depends on the type of image. Also, it is crucial to analyze neurophysiologically relevant color features for quantifying the influence of color on attention.

To determine Regions of Interest (ROI) in a scene, perceptual saliency of regions has to be measured. When scenes are viewed with the same context and motivation, these ROIs are often highly correlated among different people. As a result, it is possible to develop a computational model of visual attention that can analyze a scene and accurately estimate the location of viewers' ROIs. Color saliency is investigated in this paper. In particular, a subjective experiment has been carried out to estimate which hues attract more human attention. The performance of the visual attention model including color saliency are assessed in the context of a segmentation evaluation application.

Human Vision and Electronic Imaging IX, 2004

The saliency-based or bottom-up model of visual attention presented in this paper deals with still color images. The model we built is based on numerous properties of the human visual system (HVS), thus providing a biologically plausible system. The computation of early visual features such as color and orientation is a key step for any bottom-up model and the way to extract these visual features easily permits to differentiate a model from an other. The novelty of the proposed approach lies on the fact that the computation of early visual features is fully based on a HVS model consisting in projecting the picture into an opponent-colors space, applying a perceptual decomposition, contrast sensitivity and masking functions. Moreover, a strategy essentially based on a center surround mechanism and on the perceptual grouping phenomena underscores conspicuous locations by combining visual feature maps. A saliency map which is defined as a 2D topographic representation of conspicuity is then deduced. The model is applied to a number of natural images. Our results are then compared with the results of a well-know bottom-up model.

Lecture Notes in Computer Science, 2005

Visual attention is the ability of a vision system, be it biological or artificial, to rapidly detect potentially relevant parts of a visual scene. The saliency-based model of visual attention is widely used to simulate this visual mechanism on computers. Though biologically inspired, this model has been only partially assessed in comparison with human behavior. The research described in this paper aims at assessing its performance in the case of natural scenes, i.e. real 3D color scenes. The evaluation is based on the comparison of computer saliency maps with human visual attention derived from fixation patterns while subjects are looking at the scenes. The paper presents a number of experiments involving natural scenes and computer models differing by their capacity to deal with color and depth. The results point on the large range of scene specific performance variations and provide typical quantitative performance values for models of different complexity.

Attention, Perception, & Psychophysics, 2009

While inspecting complex natural scenes, human observers sequentially allocate attention to subsets of the stimulus (James, 1890). Under natural conditions, shifts in attention are typically associated with shifts of gaze (Rizzolatti, Raggio, Dascola, & Umiltà, 1987). Several factors guide this overt attention (Buswell, 1935; Yarbus, 1967), such as the task, the observer's experience, and the features of the stimulus. Models of the latter, bottomup factors are often based on the concept of a so-called saliency map (Koch & Ullman, 1985): Various feature channels (luminance, color, orientation, etc.) are analyzed independently, local center-surround filters yield maps of differences (contrasts) in these features, and these maps are added up. Following the saliency map literature, such maps in a single feature are referred to as conspicuity maps. These conspicuity maps are then added linearly across features to obtain the saliency map, which represents the likelihood that a location will be attended. Various studies have demonstrated that implementations of this model predict human fixations in natural scenes at

Visual Saliency aims to detect the most important regions of an image from a perceptual point of view. More in detail, the goal of Visual Saliency is to build a Saliency Map revealing the salient subset of a given image by analyzing bottom-up and top-down factors of Visual Attention. In this paper we proposed a new method for Saliency detection based on colour and scale analysis, extending our previous work based on SIFT spatial density inspection. We conducted several experiments to study the relationships between saliency methods and the object attention processes and we collected experimental data by tracking the eye movements of thirty viewers in the first three seconds of observation of several images. More precisely, we used a dataset that consists of images with an object in the foreground on an homogeneous background. We are interested in studying the performance of our saliency method with respect to the real fixation maps collected during the experiments. We compared the performances of our method with several state of the art methods with very encouraging results.