Spatio-temporal prediction and inference by V1 neurons

Nature Neuroscience, 1999

We describe a model of visual processing in which feedback connections from a higher-to a lowerorder visual cortical area carry predictions of lower-level neural activities, whereas the feedforward connections carry the residual errors between the predictions and the actual lower-level activities. When exposed to natural images, a hierarchical network of model neurons implementing such a model developed simple-cell-like receptive fields. A subset of neurons responsible for carrying the residual errors showed endstopping and other extra-classical receptive-field effects. These results suggest that rather than being exclusively feedforward phenomena, nonclassical surround effects in the visual cortex may also result from cortico-cortical feedback as a consequence of the visual system using an efficient hierarchical strategy for encoding natural images.

Proceedings of the National Academy of Sciences of the United States of America, 2010

Even within the early sensory areas, the majority of the input to any given cortical neuron comes from other cortical neurons. To extend our knowledge of the contextual information that is transmitted by such lateral and feedback connections, we investigated how visually nonstimulated regions in primary visual cortex (V1) and visual area V2 are influenced by the surrounding context. We used functional magnetic resonance imaging (fMRI) and pattern-classification methods to show that the cortical representation of a nonstimulated quarter-field carries information that can discriminate the surrounding visual context. We show further that the activity patterns in these regions are significantly related to those observed with feed-forward stimulation and that these effects are driven primarily by V1. These results thus demonstrate that visual context strongly influences early visual areas even in the absence of differential feed-forward thalamic stimulation.

Journal of Neuroscience, 2010

In this functional magnetic resonance imaging study we tested whether the predictability of stimuli affects responses in primary visual cortex (V1). The results of this study indicate that visual stimuli evoke smaller responses in V1 when their onset or motion direction can be predicted from the dynamics of surrounding illusory motion. We conclude from this finding that the human brain anticipates forthcoming sensory input that allows predictable visual stimuli to be processed with less neural activation at early stages of cortical processing.

Nature Neuroscience, 2002

A fundamental issue in the function of the visual cortex is whether its properties are adapted to the statistics of natural stimuli 1−4 . Although visual cortical neurons have a rich repertoire of responses to elementary features of visual stimuli, including contextual and learned features 5−8 , exactly how cortical responses relate to environmental statistics is just beginning to be explored quantitatively 4,9−11 . Natural images, despite their complexity, have certain common statistical properties. Simple inspection of a natural scene quickly reveals that neighboring image patches are highly correlated in local features such as orientation, contrast and spatial frequency 9,12−16 , whereas distant image patches are poorly correlated . How does the visual system take advantage of this correlational structure of natural images?

Proceedings of the National Academy of Sciences, 2008

Visual awareness has been proposed to depend on recurrent processing in early visual cortex areas including the primary visual cortex (V1). Here, we address this hypothesis with high spatiotemporal resolution magnetoencephalographic recordings in subjects performing a substitution masking paradigm. Neural activity reflecting awareness is assessed by directly comparing the neuromagnetic response elicited by effectively and ineffectively masked targets after the proportion of trials leading to masking was individually adjusted to match the proportion of trials without masking. This revealed a modulation of recurrent activity in the primary visual cortex rapidly after the onset of the feedforward sweep of processing in striate and extrastriate areas but significantly before the onset of attention-dependent recurrent modulations in V1. Our data provide direct support for the notion that (i) recurrent processing in V1 correlates with visual awareness and (ii) that attention and awareness involve distinct recurrent processing operations.

2012

The current study investigates if early visual cortical areas, V1, V2 and V3, use predictive coding to process motion information. Previous studies have reported biased visual motion responses at locations where novel visual information was presented (i.e., the motion trailing edge), which is plausibly linked to the predictability of visual input. Using high-field functional magnetic resonance imaging (fMRI), we measured brain activation during predictable versus unpreceded motion-induced contrast changes during several motion stimuli. We found that unpreceded moving dots appearing at the trailing edge gave rise to enhanced BOLD responses, whereas predictable moving dots at the leading edge resulted in suppressed BOLD responses. Furthermore, we excluded biases in directional sensitivity, shifts in cortical stimulus representation, visuo-spatial attention and classical receptive field effects as viable alternative explanations. The results clearly indicate the presence of predictive coding mechanisms in early visual cortex for visual motion processing, underlying the construction of stable percepts out of highly dynamic visual input.

Journal of Neuroscience, 2013

Current knowledge about the precise timing of visual input to the cortex relies largely on spike timings in monkeys and evoked-response latencies in humans. However, quantifying the activation onset does not unambiguously describe the timing of stimulus-feature-specific information processing. Here, we investigated the information content of the early human visual cortical activity by decoding low-level visual features from single-trial magnetoencephalographic (MEG) responses. MEG was measured from nine healthy subjects as they viewed annular sinusoidal gratings (spanning the visual field from 2 to 10°for a duration of 1 s), characterized by spatial frequency (0.33 cycles/degree or 1.33 cycles/degree) and orientation (45°or 135°); gratings were either static or rotated clockwise or anticlockwise from 0 to 180°. Time-resolved classifiers using a 20 ms moving window exceeded chance level at 51 ms (the later edge of the window) for spatial frequency, 65 ms for orientation, and 98 ms for rotation direction. Decoding accuracies of spatial frequency and orientation peaked at 70 and 90 ms, respectively, coinciding with the peaks of the onset evoked responses. Within-subject time-insensitive pattern classifiers decoded spatial frequency and orientation simultaneously (mean accuracy 64%, chance 25%) and rotation direction (mean 82%, chance 50%). Classifiers trained on data from other subjects decoded the spatial frequency (73%), but not the orientation, nor the rotation direction. Our results indicate that unaveraged brain responses contain decodable information about low-level visual features already at the time of the earliest cortical evoked responses, and that representations of spatial frequency are highly robust across individuals.

PLoS ONE, 2012

Even in V1, where neurons have well characterized classical receptive fields (CRFs), it has been difficult to deduce which features of natural scenes stimuli they actually respond to. Forward models based upon CRF stimuli have had limited success in predicting the response of V1 neurons to natural scenes. As natural scenes exhibit complex spatial and temporal correlations, this could be due to surround effects that modulate the sensitivity of the CRF. Here, instead of attempting a forward model, we quantify the importance of the natural scenes surround for awake macaque monkeys by modeling it non-parametrically. We also quantify the influence of two forms of trial to trial variability. The first is related to the neuron's own spike history. The second is related to ongoing mean field population activity reflected by the local field potential (LFP). We find that the surround produces strong temporal modulations in the firing rate that can be both suppressive and facilitative. Further, the LFP is found to induce a precise timing in spikes, which tend to be temporally localized on sharp LFP transients in the gamma frequency range. Using the pseudo R 2 as a measure of model fit, we find that during natural scene viewing the CRF dominates, accounting for 60% of the fit, but that taken collectively the surround, spike history and LFP are almost as important, accounting for 40%. However, overall only a small proportion of V1 spiking statistics could be explained (R 2 ,5%), even when the full stimulus, spike history and LFP were taken into account. This suggests that under natural scene conditions, the dominant influence on V1 neurons is not the stimulus, nor the mean field dynamics of the LFP, but the complex, incoherent dynamics of the network in which neurons are embedded.

Frontiers in Systems Neuroscience, 2009

Predictive coding has been previously introduced as a hierarchical coding framework for the visual system. At each level, activity predicted by the higher level is dynamically subtracted from the input, while the difference in activity continuously propagates further. Here we introduce modular predictive coding as a feedforward hierarchy of prediction modules without back-projections from higher to lower levels. Within each level, recurrent dynamics optimally segregates the input into novelty and familiarity components. Although the anatomical feedforward connectivity passes through the novelty-representing neurons, it is nevertheless the familiarity information which is propagated to higher levels. This modularity results in a twofold advantage compared to the original predictive coding scheme: the familiarity-novelty representation forms quickly, and at each level the full representational power is exploited for an optimized readout. As we show, natural images are successfully compressed and can be reconstructed by the familiarity neurons at each level. Missing information on different spatial scales is identified by novelty neurons and complements the familiarity representation. Furthermore, by virtue of the recurrent connectivity within each level, non-classical receptive field properties still emerge. Hence, modular predictive coding is a biologically realistic metaphor for the visual system that dynamically extracts novelty at various scales while propagating the familiarity information.