Fabio Anselmi

Istituto Italiano di Tecnologia / Italian Institute of Technology, Machine Learning, Post-Doc

Followers

Following

Co-authors

Public Views

Interests

Uploads

Papers by Fabio Anselmi

Editorial: Symmetry as a guiding principle in artificial and brain neural networks

Frontiers in Computational Neuroscience

Editorial on the Research Topic Symmetry as a guiding principle in artificial and brain neural ne... more

Download

Memo No . 63 May 26 , 2017 Symmetry Regularization

The properties of a representation, such as smoothness, adaptability, generality, equivariance/in... more The properties of a representation, such as smoothness, adaptability, generality, equivariance/invariance, depend on restrictions imposed during learning. In this paper, we propose using data symmetries, in the sense of equivalences under transformations, as a means for learning symmetryadapted representations, i.e., representations that are equivariant to transformations in the original space. We provide a sufficient condition to enforce the representation, for example the weights of a neural network layer or the atoms of a dictionary, to have a group structure and specifically the group structure in an unlabeled training set. By reducing the analysis of generic group symmetries to permutation symmetries, we devise an analytic expression for a regularization scheme and a permutation invariant metric on the representation space. Our work provides a proof of concept on why and how to learn equivariant representations, without explicit knowledge of the underlying symmetries in the dat...

Download

Symmetry-adapted representation learning

Pattern Recognition, 2019

Abstract In this paper, we propose the use of data symmetries, in the sense of equivalences under... more Abstract In this paper, we propose the use of data symmetries, in the sense of equivalences under signal transformations, as priors for learning symmetry-adapted data representations, i.e., representations that are equivariant to these transformations. We rely on a group-theoretic definition of equivariance and provide conditions for enforcing a learned representation, for example the weights in a neural network layer or the atoms in a dictionary, to have the structure of a group and specifically the group structure in the distribution of the input. By reducing the analysis of generic group symmetries to permutation symmetries, we devise a regularization scheme for representation learning algorithm, using an unlabeled training set. The proposed regularization is aimed to be a conceptual, theoretical and computational proof of concept for symmetry-adapted representation learning, where the learned data representations are equivariant or invariant to transformations, without explicit knowledge of the underlying symmetries in the data.

Data Symmetries and Learning in Fully Connected Neural Networks

IEEE Access

Symmetries in the data and how they constrain the learned weights of modern deep networks is stil... more Symmetries in the data and how they constrain the learned weights of modern deep networks is still an open problem. In this work we study the simple case of fully connected shallow non-linear neural networks and consider two types of symmetries: full dataset symmetries where the dataset X is mapped into itself by any transformation g, i.e. gX = X or single data point symmetries where gx = x, x ∈ X. We prove and experimentally confirm that symmetries in the data are directly inherited at the level of the network's learned weights and relate these findings with the common practice of data augmentation in modern machine learning. Finally, we show how symmetry constraints have a profound impact on the spectrum of the learned weights, an aspect of the so-called network implicit bias. INDEX TERMS Artificial neural networks, symmetry invariance, equivariance.

Download

Notes on Hierarchical Splines, DCLNs and i-theory

We define an extension of classical additive splines for multivariate function approximation that... more We define an extension of classical additive splines for multivariate function approximation that we call hierarchical splines. We show that the case of hierarchical, additive, piece-wise linear splines includes present-day Deep Convolutional Learning Networks (DCLNs) with linear rectifers and pooling (sum or max). We discuss how these observations together with i-theory may provide a framework for a general theory of deep networks.

Download

Convolutional neural networks on eye tracking trajectories classify patients with spatial neglect

Background and Objective : Eye-movement trajectories are rich behavioral data, providing a window... more Background and Objective : Eye-movement trajectories are rich behavioral data, providing a window on how the brain processes information. We address the challenge of characterizing signs of visuo-spatial neglect from saccadic eye trajectories recorded in brain-damaged patients with spatial neglect as well as in healthy controls during a visual search task. Methods : We establish a standardized preprocessing pipeline adaptable to other task-based eye-tracker measurements. We use a deep convolutional network, a very successful type of neural network architecture in many computer vision applications, including medical diagnosis systems, to automatically analyze eye trajectories. Results : Our algorithm can classify brain-damaged patients vs. healthy individuals with an accuracy of 86+/-5%. Moreover, the algorithm scores correlate with the degree of severity of neglect signs estimated with standardized paper-and-pencil test and with white matter tracts impairment via Diffusion Tensor Im...

Download

Invariant representation for blur and down-sampling transformations

2016 IEEE International Conference on Image Processing (ICIP), 2016

Invariant representations of images can significantly reduce the sample complexity of a classifie... more Invariant representations of images can significantly reduce the sample complexity of a classifier performing object identification or categorization as shown in a recent analysis of invariant representations for object recognition. In the case of geometric transformations of images the theory [1] shows how invariant signatures can be learned in a biologically plausible way from unsupervised observations of the transformations of a set of randomly chosen template images. Here we extend the theory to non-geometric transformations such as blur and down-sampling. The proposed algorithm achieve an invariant representation via two simple biologically-plausible steps: 1. compute normalized dot products of the input with the stored transformations of each template, and 2. for each template compute the statistics of the resulting set of values such as the histogram or moments. The performance of our system on challenging blurred and low resolution face matching tasks exceeds the previous state-of-the-art by a large margin which grows with increasing image corruption.

Download

Representation Learning in Sensory Cortex: A Theory

IEEE Access

We review and apply a computational theory based on the hypothesis that the feedforward path of t... more We review and apply a computational theory based on the hypothesis that the feedforward path of the ventral stream in visual cortex's main function is the encoding of invariant representations of images. A key justification of the theory is provided by a result linking invariant representations to small sample complexity for image recognition-that is, invariant representations allow learning from very few labeled examples. The theory characterizes how an algorithm that can be implemented by a set of ''simple'' and ''complex'' cells-a ''Hubel Wiesel module''-provides invariant and selective representations. The invariance can be learned in an unsupervised way from observed transformations. Our results show that an invariant representation implies several properties of the ventral stream organization, including the emergence of Gabor receptive filelds and specialized areas. The theory requires two stages of processing: the first, consisting of retinotopic visual areas such as V1, V2 and V4 with generic neuronal tuning, leads to representations that are invariant to translation and scaling; the second, consisting of modules in IT (Inferior Temporal cortex), with class-and object-specific tuning, provides a representation for recognition with approximate invariance to class specific transformations, such as pose (of a body, of a face) and expression. In summary, our theory is that the ventral stream's main function is to implement the unsupervised learning of ''good'' representations that reduce the sample complexity of the final supervised learning stage. INDEX TERMS Visual cortex, Hubel Wiesel model, simple and complex cells, artificial neural networks, invariance, sample complexity. I. INTRO AND BACKGROUND 18 The ventral visual stream is believed to underlie object recog-19 nition abilities in primates. Fifty years of modeling efforts, 20 beginning with the original Hubel and Wiesel proposal (HW 21 in the rest of the paper) of a hierarchical architecture iterating 22 in different layers the motif of simple and complex cells 23 in V1, have led to a series of quantitative models from 24 Fukushima [36] and Riesenhuber and Poggio [106] HMAX 25 (Hierarchical architecture with MAX pooling) to more recent 26 architectures based on contrastive [20], [139] or slow fea-27 tures [109] learning. These models are increasingly faithful 28 to biological architecture constraints and are able to mimic 29 affected not only by orthographic projection of the 3D geome-411 try but also by the process of image formation which depends 412 on the 3D geometry of the object, its reflectance properties, 413 and the relative location of the light source and viewer. 414 It turns out that the HW algorithm can still be applied to 415 non-group transformations-such as transformations of the 416 expression of a face, or of the pose of a body-to provide, 417 under certain conditions, approximate invariance. In this case 418 bounds on the invariance depend on specific details of the 419 object and the transformation: we do not have general results and suspect they may not exist. The key technical requirement 421 is that a new type of sparsity condition holds: sparsity for 422 the class of images I C with respect to the dictionary t k under 423 the transformations T r (we consider here a one parameter (r) 424 transformation) 425 693 envelope of the immature cell). Assume that all of the simple 694 cells are exposed, while in a plastic state, to a possibly 695 large set of images T = (t 1 ,. .. , t K). A specific cell at a 696 certain position in x, y, s is exposed to the set of transformed 697 templates g * T (where g * corresponds to the translation and 698 scale that transforms the ''zero'' cells to the chosen neu-699 ron in the lattice) and therefore the associated covariance 700 matrix g * TT T g T *. Thus it is possible to choose PCA as new 701 templates, and pooling over corresponding PCAs across dif-702 ferent cells is equivalent to pooling over a template and its 703 transformations. Both the invariance and selectivity result are 704 valid. Empirically, we find ([77]) that PCA of natural images 705 provides eigenvectors that are Gabor-like wavelets with a 706 random orientation for each sized receptive field. The random 707 orientation is because of the argument above, together with 708 the fact that the covariance of natural images is approximately rotation invariant. The Gabor-like shape can be qualitatively 710 explained in terms of translation invariance of the correlation 711 matrix associated with a set of natural images (and their 712 approximate scale invariance which corresponds to a ≈ 1/f 713 spectrum, see also [111], [114], and [127]). 3 Thus the Oja rule 714 acting on natural images provides ''equivalent templates'' 715 that are Gabor-like: the optimal templates, according to the 716 theory of section IIC. 717 Consider now non-retinotopic stage 2 in which transforma-718 tions are not in scale or position, such as the transformation 719 induced by a rotation of a face. Assume that a ''simple'' cell 880 table, albeit with interpolation power. The earlier case 881 is more interesting since it allows generalization from 882 a single view of a novel object. It also represent a clear 883 case of transfer learning. 884 C. DOMAIN-SPECIFIC REGIONS IN THE VENTRAL STREAM 885 As discussed by [77], there are other domain-specific 886 regions in the ventral stream besides faces and bodies. It is 887 possible that additional regions for less-common or less 888 transformation-compatible object classes will appear with 889 higher resolution imaging techniques. One example may 890 be the fruit area, discovered in macaques with high-field 891 fMRI [69]. Others include the body area and the Lat-892 eral Occipital Complex (LOC) which according to recent 893 data [85] is not really a dedicated region for general object 894 processing but a heterogeneous area of cortex containing 895 many domain-specific regions too small to be detected with 896 the resolution of fMRI (but see also [99] and [135]). The 897 Visual Word Form Area (VWFA) [19], [22] seems to repre-898 sent printed words. In addition to the generic transformations 899 that apply to all objects, printed words undergo several non-900 generic transformations that never occur with other objects. 901 For instance, our reading is rather invariant to font transfor-902 mations and can deal with handwritten text. Thus, VWFA 903 is well-accounted for by the invariance hypothesis, as words 904 are a frequently-viewed stimuli which undergo class-specific 905 transformations. 906 The justification-really a prediction-by our theory for 907 domain-specific regions in cortex is different from other 908 proposals. However, it is complementary w.r.t. some of them, 909 rather than exclusive. For instance, it would make sense that 910 the clustering depends not only on the index of compatibil-911 ity but also on the relative frequency of each object class. 912 We conjecture that a) that transformation compatibility is the 913 critical factor driving the development of domain-specific 914 regions, and b) that there are separate modules for object 915 classes that transform differently from one another. 916 972 neural circuits underlying tuning of cells. This is unlike most 973 other models or theories. 974 Predictions. From the point of view of neuroscience, the 975 theory makes a number of predictions, some obvious, some 976 less so. One of the main predictions is that simple and com-977 plex cells should be found in all visual and auditory areas, not 978 only in V1. Our definition of simple cells and complex cells 979 is different from the traditional ones used by physiologists; for example, we propose a broader interpretation of complex 981 cells, which in the theory represent invariant measurements 982 associated with histograms (or moments of their values) of 983 the outputs of simple cells. The theory implies that, under 984 some conditions, exact or approximate invariance to all geo-985 metric image transformations can be learned, either during 986 development or in adult life. It is, however, also consistent 987 with the possibility that basic invariances may be genetically 988 encoded by evolution and possibly refined and maintained 989 by unsupervised visual experience. A single cell model for 990 simple complex cells follows from the theory as an interest-991 ing possibility. Our theory also makes predictions about the 992 architecture of the ventral stream: 993 • the output of V2, V4, and PIT should access memory 994 either via connections that bypass higher areas or indi-995 rectly via equivalent neurons in higher areas (because of 996 the argument in a previous section about clutter). 997 • areas V1, V2, V4 and possibly PIT are mainly dedicated 998 to computing signatures that are invariant to translation, 999 scale and their combinations-as experienced in past visual experience.

Download

Using Physics to Explain and Interpret Convolutional Neural Network Solutions of Quantum Spin Problems

Robust deep learning object recognition models rely on low frequency information in natural images

ABSTRACTMachine learning models have difficulty generalizing to data outside of the distribution ... more ABSTRACTMachine learning models have difficulty generalizing to data outside of the distribution they were trained on. In particular, vision models are usually vulnerable to adversarial attacks or common corruptions, to which the human visual system is robust. Recent studies have found that regularizing machine learning models to favor brain-like representations can improve model robustness, but it is unclear why. We hypothesize that the increased model robustness is partly due to the low spatial frequency preference inherited from the neural representation. We tested this simple hypothesis with several frequency-oriented analyses, including the design and use of hybrid images to probe model frequency sensitivity directly. We also examined many other publicly available robust models that were trained on adversarial images or with data augmentation, and found that all these robust models showed a greater preference to low spatial frequency information. We show that preprocessing by b...

Download

Ca 2 + Modulation of Mechanotransduction in Hair Cells

The inner ear contains delicate sensory receptors that have adapted to detect the minutest mechan... more The inner ear contains delicate sensory receptors that have adapted to detect the minutest mechanical disturbances. Ca 2+ ions are implicated in all steps of the transduction process, as well as in its regulation by an impressive ensemble of finely tuned feedback control mechanisms. Recent studies have unveiled some of the key players, but things do not sound quite right yet.

Download

ATP acts as a paracrine signaling molecule to reduce the motility of T cells

As you can see below there is an interest in the paper, but the referees also bring up important ... more As you can see below there is an interest in the paper, but the referees also bring up important concerns with the paper, both conceptual and technical. In particular, issues are raised regarding the role of ATP on T cell motility. It is unclear if the raised issues can be addressed and if you can provide more data to support the conclusions, which is what is needed for publication here. Given this, I think it would be most productive if you could provided me with a detailed point-by-point response of what you could potentially add in a revision upfront and I would then use that as a base to make my decision.

Local Convolutions Cause an Implicit Bias towards High Frequency Adversarial Examples

Despite great efforts, neural networks are still prone to adversarial attacks. Recent work has sh... more Despite great efforts, neural networks are still prone to adversarial attacks. Recent work has shown that adversarial perturbations typically contain high-frequency features, but the root cause of this phenomenon remains unknown. Inspired by the theoretical work in linear full-width convolutional models [1], we hypothesize that the nonlinear local (i.e. bounded-width) convolutional models used in practice are implicitly biased to learn high frequency features, and that this is the root cause of high frequency adversarial examples. To test this hypothesis, we analyzed the impact of different choices of linear and nonlinear architectures on the implicit bias of the learned features and the adversarial perturbations, in both spatial and frequency domains. We find that the high-frequency adversarial perturbations are critically dependent on the convolution operation in two ways: (i) the translation invariance of the convolution induces an implicit bias towards sparsity in the frequency ...

Download

Genuine Personality Recognition from Highly Constrained Face Images

Image Analysis and Processing – ICIAP 2019, 2019

People are able to accurately estimate personality traits, merely on the basis of "passport"-styl... more People are able to accurately estimate personality traits, merely on the basis of "passport"-style neutral faces and, thus, cues must exist that allow for such estimation. However, up to date, there has been little progress in identifying the form and location of these cues. In this paper we address the problem of inferring true personality traits in highly constrained images using state of art machine learning techniques, in particular, deep networks and class activation maps analysis. The novelty of our work consists in that, differently from the vast majority of the current and past approaches (that refer to the problem of consensus personality rating prediction) we predict the genuine personality based on highly constrained images: the targets are self ratings on a validated personality inventory and we restrict to passport-like photos, in which so-called controllable cues are minimized. Our results show that self-reported personality traits can be accurately evaluated from facial features. A preliminar analysis on the features activation maps shows promising results for a deeper understanding on relevant facial cues for traits estimation.

Download

A computational model for grid maps in neural populations

Journal of Computational Neuroscience, 2020

Grid cells in the entorhinal cortex, together with head direction, place, speed and border cells,... more Grid cells in the entorhinal cortex, together with head direction, place, speed and border cells, are major contributors to the organization of spatial representations in the brain. In this work we introduce a novel theoretical and algorithmic framework able to explain the emergence of hexagonal grid-like response patterns from head direction cells' responses. We show that this pattern is a result of minimal variance encoding of neurons. The novelty lies into the formulation of the encoding problem through the modern Frame Theory language, specifically that of equiangular Frames, providing new insights about the optimality of hexagonal grid receptive fields. The model proposed overcomes some crucial limitations of the current attractor and oscillatory models. It is based on the well-accepted and tested hypothesis of Hebbian learning, providing a simplified cortical-based framework that does not require the presence of theta velocity-driven oscillations (oscillatory model) or translational symmetries in the synaptic connections (attractor model). We moreover demonstrate that the proposed encoding mechanism naturally explains axis alignment of neighbor grid cells and maps shifts, rotations and scaling of the stimuli onto the shape of grid cells' receptive fields, giving a straightforward explanation of the experimental evidence of grid cells remapping under transformations of environmental cues.

Download

Understanding robustness and generalization of artificial neural networks through Fourier masks

ArXiv, 2022

Despite the enormous success of artificial neural networks (ANNs) in many disciplines, the charac... more Despite the enormous success of artificial neural networks (ANNs) in many disciplines, the characterization of their computations and the origin of key properties such as generalization and robustness remain open questions. Recent literature suggests that robust networks with good generalization properties tend to be biased towards processing low frequencies in images. To explore the frequency bias hypothesis further, we develop an algorithm that allows us to learn modulatory masks highlighting the essential input frequencies needed for preserving a trained network’s performance. We achieve this by imposing invariance in the loss with respect to such modulations in the input frequencies. We first use our method to test the low-frequency preference hypothesis of adversarially trained or data-augmented networks. Our results suggest that adversarially robust networks indeed exhibit a low-frequency bias but we find this bias is also dependent on directions in frequency space. However, t...

Download

Unsupervised Learning of Invariant Representations in Hierarchical Architectures

The present phase of Machine Learning is characterized by supervised learning algorithms relying ... more The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples (n →∞). The next phase is likely to focus on algorithms capable of learning from very few labeled examples (n → 1), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a "good" representation for supervised learning, characterized by small sample complexity (n). We consider the case of visual object recognition though the theory applies to other domains. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translations, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and unique (discriminative) signature can be computed for each image patch, I, in terms of empirical distributions of the dot-products between I and a...

Download

Magic Materials: a theory of deep hierarchical architectures

Does invariant recognition predict tuning of neurons in sensory cortex ?

Tuning properties of simple cells in cortical V1 can be described in terms of a “universal shape”... more Tuning properties of simple cells in cortical V1 can be described in terms of a “universal shape” characterized by parameter values which hold across different species ([12], [33], [22]). This puzzling set of findings begs for a general explanation grounded on an evolutionarily important computational function of the visual cortex. We ask here whether these properties are predicted by the hypothesis that the goal of the ventral stream is to compute for each image a “signature” vector which is invariant to geometric transformations as postulated in [30] – with the the additional assumption that the mechanism for continuously learning and maintaining invariance consists of the memory storage of a sequence of neural images of a few objects undergoing transformations (such as translation, scale changes and rotation) via Hebbian synapses. For V1 simple cells the simplest version of this hypothesis is the online Oja rule which implies that the tuning of neurons converges to the eigenvecto...

Download

Memo No . 35 August 5 , 2015 Deep Convolutional Networks are Hierarchical Kernel Machines

In i-theory a typical layer of a hierarchical architecture consists of HW modules pooling the dot... more In i-theory a typical layer of a hierarchical architecture consists of HW modules pooling the dot products of the inputs to the layer with the transformations of a few templates under a group. Such layers include as special cases the convolutional layers of Deep Convolutional Networks (DCNs) as well as the non-convolutional layers (when the group contains only the identity). Rectifying nonlinearities – which are used by present-day DCNs – are one of the several nonlinearities admitted by i-theory for the HW module. We discuss here the equivalence between group averages of linear combinations of rectifying nonlinearities and an associated kernel. This property implies that present-day DCNs can be exactly equivalent to a hierarchy of kernel machines with pooling and nonpooling layers. Finally, we describe a conjecture for theoretically understanding hierarchies of such modules. A main consequence of the conjecture is that hierarchies of trained HW modules minimize memory requirements ...

Download

Fabio Anselmi

Uploads

Papers by Fabio Anselmi

Log In