An entropic associative memory

International Journal, 2010

Networks. Without memory, Neural Network can not be learned itself. One of the primary concepts of memory in neural networks is Associative neural memories. A survey has been made on associative neural memories such as Simple associative memories (SAM), Dynamic associative memories (DAM), Bidirectional Associative memories (BAM), Hopfield memories, Context Sensitive Auto-associative memories (CSAM) and so on. These memories can be applied in various fields to get the effective outcomes. We present a study on these associative memories in artificial neural networks.

IEEE Transactions on Neural Networks, 2000

In contrast to conventional feedback bidirectional associative Memory (BAM) network models, a feedforward BAM network is developed based on a one-shot design algorithm of (2 (+)) computational complexity, where is the number of prototype pairs and are the dimensions of the input/output bipolar vectors. The feedforward BAM is an-three-layer network of McCulloch-Pitts neurons with storage capacity 2 min and guaranteed perfect bidirectional recall. The overall network design procedure is fully scalable in the sense that any number 2 min of bidirectional associations can be implemented. The prototype patterns may be arbitrarily correlated. With respect to inference performance, it is shown that the Hamming attractive radius of each prototype reaches the maximum possible value. Simulation studies and comparisons illustrate and support these theoretical developments.

2007

The progress in information technologies enables applications of artificial neural networks even in areas like navigation and geographical information systems, telecommunications, etc. This kind of problems requires robust recall of-possibly incomplete-spatial data. In this context, spatial patterns can be geographic maps, room plans, etc. Strategies for processing of spatial patterns can be based on associative memories. However, the performance of traditional associative memories (usually Hopfield-like networks) is very sensitive to the number of stored patterns and their mutual correlations. In this thesis, we design and implement the so-called Hierarchical Associative Memory (HAM) model in order to avoid limitations imposed by processing of a large number of mutually correlated patterns. The proposed model consists of several associative memories grouped into layers. A suitable strategy applied during the network dynamics allows reliable processing of large amounts of spatial data. To evaluate the performance of the proposed HAM model, extensive simulations are carried out. The storage and recall abilities of the HAM model are studied experimentally and thoroughly compared with several other models of associative memories.

Journal of Statistical Physics, 2001

We study a two-layer neural network made of N and M(N) neurons, producing a two-way association search for a family of p(N) patterns, where each pattern is a pair of two independent sub-categories of information having respectively N and M(N) components. In terms of the ratio γ=limN→∞M(N)/N, we study the retrieval capability of this network and show that there exists, at least, three regimes of association for which we determine the evolution of the threshold α c (γ) of the storage capacity α=limN→∞p(N)/N.

1992

This thesis explores partially connected networks in associative memory tasks. The storage capacity of the fully connected associative net is reviewed. Simu¬ lation results demonstrate that the performance of this architecture was worse than that predicted by earlier analyses. New analysis is presented that accu¬ rately describes the behaviour of this net. A characterization of the storage capacity and information efficiency of the partially connected associative net is presented. In partially connected nets, one of the key problems is how to set the unit thresholds in such a way that the units which should not fire are quiet and those that should fire do so. New thresholding strategies are developed which enable this architecture to function well as an associative content-addressable memory. An important result is that the best thresholding strategies are functions of the firing history of an output unit and the number of active inputs impinging on it. Analysis is presented that pr...

2013 IEEE Information Theory Workshop (ITW), 2013

We propose a novel architecture to design a neural associative memory that is capable of learning a large number of patterns and recalling them later in presence of noise. It is based on dividing the neurons into local clusters and parallel plains, very similar to the architecture of the visual cortex of macaque brain. The common features of our proposed architecture with those of spatially-coupled codes enable us to show that the performance of such networks in eliminating noise is drastically better than the previous approaches while maintaining the ability of learning an exponentially large number of patterns. Previous work either failed in providing good performance during the recall phase or in offering large pattern retrieval (storage) capacities. We also present computational experiments that lend additional support to the theoretical analysis.

1999

Abstract. The model of Hopfield for a neural network with associative memory is modified by the introduction of a maximum value for the synaptic strength; in this way old patterns are automatically forgotten and the memory recalls only the most recent ones. If the parameters are correctly chosen, the memory never goes into the state of total confusion characteristic of the Hopfield model.

IEEE Transactions on Systems, Man, and Cybernetics, 1994

promising model for heteroassociative memories. However, its applications are severely restricted to networks with logical symmetry of interconnections and pattern orthogonality or small pattern size. Although the restrictions on pattern orthogonality and pattern size can be relaxed to a certain extent, all previous efforts are at the cost of increase in connection complexity. In this paper, a new modification on the BAM is made and a new model named Asymmetric Bidirectional Associative Memory (ABAM) is proposed. This model not only can cater for the logical asymmetry of interconnections but also is capable of accommodating a larger number of non-orthogonal patterns. Furthermore, all these properties of the ABAM are achieved without increasing the connection complexity of the network. Theoretical analysis and simulation results all demonstrate that the ABAM indeed outperforms the BAM and its existing variants in all aspects of storage capacity, error-correcting capability and convergence.

Computational properties of use to biological organisms or to the construction of computers can emerge as collective properties of systems -having a large number of simple equivalent components (or neurons). The physical meaning ofcontent-addressable memory is described by an appropriate phase space flow of the state of a system. A model of such a system is given, based on aspects of neurobiology but readily adapted to integrated circuits. The collective properties of this model produce a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size. The algorithm for the time evolution of the state of the system is based on asynchronous parallel processing. Additional emergent collective properties include some capacity for generalization, familiarity recognition, categorization, error correction, and time sequence retention. The collective properties are only weakly sensitive to details ofthe modeling or the failure of individual devices.

Physica D Nonlinear Phenomena, 2008

A fundamental part of a computational system is its memory, which is used to store and retrieve data. Classical computer memories rely on the static approach and are very different from human memories. Neural network memories are based on auto-associative attractor dynamics and thus provide a high level of pattern completion. However, they are not used in general computation since there are practically no algorithms to load an arbitrary landscape of attractors into them. In this sense neural network memory models cannot communicate well with symbolic and prior knowledge. We propose the design of a new memory based on localist attractor dynamics with reconsolidation called Reconsolidation Attractor Network (RAN). RAN combines symbolic and subsymbolic features in a very attractive way: it is based on attractors; enables pattern classification under missing data; and demonstrates dynamic reconsolidation, which is very useful for tracking changing concepts. The perception RAN enables is somewhat reminiscent of human perception due to its context sensitivity. Furthermore, it enables an immediate and clear interface with symbolic memories, including loading of attractors by means of trivial wiring, updating attractors, and retrieving them faster without waiting for full convergence. It also scales to any number of concepts. This provides a useful counterpoint to more conventional memory systems, such as random access memory and auto-associative neural networks.