Cognitive Computation

Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems, 1999

This work argues that it should be possible to combine pulse-based VLSI techniques with the relatively simple training rules of the Helmholtz Machine stochastic neural architecture, in order to build an analogue probabilistic hardware model of the latter. An overview of the necessary components is presented, as well as a design for a pulsewidth modulation oscillator, capable of transforming a current input (which represents the squashed, post-synaptic signal processed by a particular neuron) into the probability associated with the binary state of that neuron. A CMOS hardware prototype has been designed and fabricated, and precautions were taken during the design and simulation stages in order to prevent the oscillators on the same chip from locking together. Apart from testing the hardware prototype, future plans involve the hardware implementation of other modules, such as the synapse, the squashing function and weight changing circuitry.

Proceedings of the …, 2005

As Si CMOS devices are scaled down into the nanoscale regime, current computer architecture approaches are reaching their practi-cal limits. Future nano-architectures will confront devices and in-terconnections with a large number of inherent defects, which mo-tivates ...

1999

This paper describes two stochastic neural networks and how they might be instantiated as analogue VLSI circuits. We explain why stochastic neural networks have not been well represented in the hardware field in the past and define a new approach.

2010

A short survey is provided about our recent explorations of the young topic of noisebased logic. After outlining the motivation behind noise-based computation schemes, we present a short summary of our ongoing efforts in the introduction, development and design of several noise-based deterministic multivalued logic schemes and elements. In particular, we describe classical, instantaneous, continuum, spike and random-telegraphsignal based schemes with applications such as circuits that emulate the brain's functioning and string verification via a slow communication channel.

IEEE Transactions on Emerging Topics in Computing, 2016

We present an architecture and a compilation toolchain for stochastic machines dedicated to Bayesian inferences. These machines are not Von Neumann and code information with stochastic bitstreams instead of using floating point representations. They only rely on stochastic arithmetic and on Gibbs sampling to perform approximate inferences. They use banks of binary random generators which capture the prior knowledge on which the inference is built. The output of the machine is devised to continuously sample the joint probability distribution of interest. While the method is explained on a simple example, we show that our machine computes a good approximation of the solution to a problem intractable in exact inference.

As the physical limits of Moore's law are being reached, a research effort is launched to achieve further performance improvements by exploring computation paradigms departing from standard approaches. The BAMBI project (Bottom-up Approaches to Machines dedicated to Bayesian Inference) aims at developing hardware dedicated to probabilistic computation, which extends logic computation realised by boolean gates in current computer chips. Such probabilistic computing devices would allow to solve faster and at a lower energy cost a wide range of Artificial Intelligence applications, especially when decisions need to be taken from incomplete data in an uncertain environment. This paper describes an architecture where very simple operators compute on a time coding of probability values as stochastic signals. Simulation tests and a reconfigurable logic hardware implementation demonstrated the feasibility and performances of the proposed inference machine. Hardware results show this architecture can quickly solve Bayesian sensor fusion problems and is very efficient in terms of energy consumption.

Stochastic computing (SC) was proposed in the 1960s as a low-cost alternative to conventional binary computing. It is unique in that it represents and processes information in the form of digitized probabilities. SC employs very low-complexity arithmetic units which was a primary design concern in the past. Despite this advantage and also its inherent error tolerance, SC was seen as impractical because of very long computation times and relatively low accuracy. However, current technology trends tend to increase uncertainty in circuit behavior, and imply a need to better understand, and perhaps exploit, probability in computation. This paper surveys SC from a modern perspective where the small size, error resilience, and probabilistic features of SC may compete successfully with conventional methodologies in certain applications. First, we survey the literature and review the key concepts of stochastic number representation and circuit structure. We then describe the design of SC-based circuits and evaluate their advantages and disadvantages. Finally, we give examples of the potential applications of SC, and discuss some practical problems that are yet to be solved.

Scientific Reports, 2016

Belief networks represent a powerful approach to problems involving probabilistic inference, but much of the work in this area is software based utilizing standard deterministic hardware based on the transistor which provides the gain and directionality needed to interconnect billions of them into useful networks. This paper proposes a transistor like device that could provide an analogous building block for probabilistic networks. We present two proof-of-concept examples of belief networks, one reciprocal and one non-reciprocal, implemented using the proposed device which is simulated using experimentally benchmarked models.

Nature Electronics

In recent years, a considerable research effort has shown the energy benefits of implementing neural networks with memristors or other emerging memory technologies. However, for extreme-edge applications with high uncertainty, access to reduced amounts of data, and where explainable decisions are required, neural networks may not provide an acceptable form of intelligence. Bayesian reasoning can solve these concerns, but it is computationally expensive and, unlike neural networks, does not translate naturally to memristor-based architectures. In this work, we introduce, demonstrate experimentally on a fully fabricated hybrid CMOS-memristor system, and analyze a Bayesian machine designed for highly-energy efficient Bayesian reasoning. The architecture of the machine is obtained by writing Bayes' law in a way making its implementation natural by the principles of distributed memory and stochastic computing, allowing the circuit to function using solely local memory and minimal data movement. Measurements on a fabricated small-scale Bayesian machine featuring 2,048 memristors and 30,080 transistors show the viability of this approach and the possibility of overcoming the challenges associated with its design: the inherent imperfections of memristors, as well as the need to distribute very locally higher-than-nominal supply voltages. The design of a scaled-up version of the machine shows its outstanding energy efficiency on a real-life gesture recognition task: a gesture can be recognized using 5,000 times less energy than using a microcontroller unit. The Bayesian machine also features several desirable features, e.g., instant on/off operation, compatibility with low supply voltages, and resilience to single-event upsets. These results open the road for Bayesian reasoning as an attractive way for energy-efficient, robust, and explainable intelligence at the edge.

IEEE Design & Test

Probabilistic spin logic (PSL) based on networks of binary stochastic neurons (or p-bits) has been shown to provide a viable framework for many functionalities including Ising computing, Bayesian inference, invertible Boolean logic and image recognition. This paper presents a hardware building block for the PSL architecture, consisting of an embedded MTJ and a capacitive voltage adder of the type used in neuMOS. We use SPICE simulations to show how identical copies of these building blocks (or weighted p-bits) can be interconnected with wires to design and solve a small instance of the NP-complete Subset Sum Problem fully in hardware.