Skip to main content

Eric Chalmers

Followers

0

Following

1

Public Views

Interests

Uploads

Papers by Eric Chalmers

Four attributes of intelligence, a thousand questions

Biological Cybernetics, Dec 6, 2023

Jeff Hawkins is one of those rare individuals who speaks the languages of both AI and neuroscienc... more Jeff Hawkins is one of those rare individuals who speaks the languages of both AI and neuroscience. In his recent book, "A Thousand Brains: A New Theory of Intelligence", Hawkins proposes that current learning algorithms lack four attributes which will be necessary for true machine intelligence. Here we demonstrate that a minimal learning system which satisfies all four points can be constructed using only simple, classical machine learning techniques. We illustrate that such a system falls short of biological intelligence in some important ways. We suggest that Hawkins' list is a useful model, but the "recipe" for true intelligence-if there is one-may not be so easily defined.

Reinforcement Learning with Brain-Inspired Modulation can Improve Adaptation to Environmental Changes

arXiv (Cornell University), May 19, 2022

Developments in reinforcement learning (RL) have allowed algorithms to achieve impressive perform... more Developments in reinforcement learning (RL) have allowed algorithms to achieve impressive performance in highly complex, but largely static problems. In contrast, biological learning seems to value efficiency of adaptation to a constantly-changing world. Here we build on a recently-proposed neuronal learning rule that assumes each neuron can optimize its energy balance by predicting its own future activity. That assumption leads to a neuronal learning rule that uses presynaptic input to modulate prediction error. We argue that an analogous RL rule would use action probability to modulate reward prediction error. This modulation makes the agent more sensitive to negative experiences, and more careful in forming preferences. We embed the proposed rule in both tabular and deep-Q-network RL algorithms, and find that it outperforms conventional algorithms in simple, but highly-dynamic tasks. We suggest that the new rule encapsulates a core principle of biological intelligence; an important component for allowing algorithms to adapt to change in a human-like way. Preprint. Under review.

Biologically-inspired neuronal adaptation improves learning in neural networks

arXiv (Cornell University), Apr 8, 2022

Since humans still outperform artificial neural networks on many tasks, drawing inspiration from ... more Since humans still outperform artificial neural networks on many tasks, drawing inspiration from the brain may help to improve current machine learning algorithms. Contrastive Hebbian Learning (CHL) and Equilibrium Propagation (EP) are biologically plausible algorithms that update weights using only local information (without explicitly calculating gradients) and still achieve performance comparable to conventional backpropagation. In this study, we augmented CHL and EP with Adjusted Adaptation, inspired by the adaptation effect observed in neurons, in which a neuron's response to a given stimulus is adjusted after a short time. We add this adaptation feature to multilayer perceptrons and convolutional neural networks trained on MNIST and CIFAR-10. Surprisingly, adaptation improved the performance of these networks. We discuss the biological inspiration for this idea and investigate why Neuronal Adaptation could be an important brain mechanism to improve the stability and accuracy of learning.

0731 What is HSAT Missing? A Comparison of Respiratory Events and OSA Diagnosis Across Type 2 and Type 3 Studies

Sleep

Introduction The need for having in-home sleep testing has grown due to the COVID-19 pandemic. Wh... more Introduction The need for having in-home sleep testing has grown due to the COVID-19 pandemic. While Type 3 Home Sleep Apnea Tests (HSAT) are frequently used, their accuracy remains questionable. This study aimed to compare respiratory events and diagnosis of obstructive sleep apnea between Type 2 and Type 3 studies. Methods 550 participants completed overnight Type 2 sleep studies using the Cerebra Sleep System. Files were autoscored as a type 2 acquisition and were manually edited by a RPSGT. On a second auto-score, mapped file channels were reduced to nasal cannula, chest belt, SpO2, position, heart rate, and audio channels to simulate a Type 3 study. The respiratory disturbance index (RDI) in the Type 2 tests was compared to the apnea-hypopnea index (AHI) in the simulated Type 3 files using a 4% desaturation threshold. Diagnosis of severity of OSA was classified based on indices of <5 as “None”, 5-14.99 as “Mild”, 15-29.99 as “Moderate”, and above 30 as “Severe”. Results 5 re...

0439 A Polysomnographic Marker of Sleep Quality using Odds Ratio Product are Associated with Insomnia Symptoms in a Dose-Response Relationship

Sleep

Introduction Insomnia is primarily diagnosed with subjective complaints because objective biomark... more Introduction Insomnia is primarily diagnosed with subjective complaints because objective biomarkers are limited. In the current study, we assessed whether a polysomnographic marker of sleep quality (odds ratio product; ORP) was associated with different types of insomnia symptoms. Methods 416 participants (age 46.2±12.5; 176 females) recorded their sleep with an in-home Type 2 PSG test using the Cerebra Sleep System and filled out sleep questionnaires. Sleep quality was measured using odds ratio product (ORP) derived from frontal EEG channels across the duration of the recording (ORPTRT). On the questionnaires, participants reported whether they experienced any of the following insomnia symptoms at least three times a week: trouble falling asleep, trouble staying asleep, and waking up earlier than desired. Participants were then classified based on the total number of insomnia symptoms reported. An ANOVA, controlling for age, was performed to compare the effect of the number of ins...

Hippocluster: an efficient, hippocampus-inspired algorithm for graph clustering

arXiv (Cornell University), May 19, 2022

Random walks can reveal communities or clusters in networks, because they are more likely to stay... more Random walks can reveal communities or clusters in networks, because they are more likely to stay within a cluster than leave it. Thus, one family of community detection algorithms uses random walks to measure distance between pairs of nodes in various ways, and then applies K-Means or other generic clustering methods to these distances. Interestingly, information processing in the brain may suggest a simpler method of learning clusters directly from random walks. Drawing inspiration from the hippocampus, we describe a simple two-layer neural learning framework. Neurons in one layer are associated with graph nodes and simulate random walks. These simulations cause neurons in the second layer to become tuned to graph clusters through simple associative learning. We show that if these neuronal interactions are modelled a particular way, the system is essentially a variant of K-Means clustering applied directly in the walk-space, bypassing the usual step of computing node distances/similarities. The result is an efficient graph clustering method. Biological information processing systems are known for high efficiency and adaptability. In tests on benchmark graphs, our framework demonstrates this high data-efficiency, low memory use, low complexity, and real-time adaptation to graph changes, while still achieving clustering quality comparable to other algorithms.

Hippocluster: An efficient, hippocampus-inspired algorithm for graph clustering

Information Sciences

Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework

Frontiers in Computational Neuroscience

Backpropagation (BP) has been used to train neural networks for many years, allowing them to solv... more Backpropagation (BP) has been used to train neural networks for many years, allowing them to solve a wide variety of tasks like image classification, speech recognition, and reinforcement learning tasks. But the biological plausibility of BP as a mechanism of neural learning has been questioned. Equilibrium Propagation (EP) has been proposed as a more biologically plausible alternative and achieves comparable accuracy on the CIFAR-10 image classification task. This study proposes the first EP-based reinforcement learning architecture: an Actor-Critic architecture with the actor network trained by EP. We show that this model can solve the basic control tasks often used as benchmarks for BP-based models. Interestingly, our trained model demonstrates more consistent high-reward behavior than a comparable model trained exclusively by BP.

Combining Backpropagation with Equilibrium Propagation to improve an Actor-Critic RL framework

Backpropagation has been used to train neural networks for many years, allowing them to solve a w... more Backpropagation has been used to train neural networks for many years, allowing them to solve a wide variety of tasks like image classification, speech recognition, and reinforcement learning tasks. But the biological plausibility of backpropagation as a mechanism of neural learning has been questioned. Equilibrium Propagation (EP) has been proposed as a more biologically plausible alternative and achieves comparable accuracy on the CIFAR-10 image classification task. This study proposes the first EP-based reinforcement learning architecture: an actor-critic architecture with the actor network trained by EP. We show that this model can solve the basic control tasks often used as benchmarks for BP-based models. Interestingly, our trained model demonstrates more consistent high-reward behavior than a comparable model trained exclusively by backpropagation.

A Deep Convolutional Auto-Encoder with Pooling – Unpooling Layers in Caffe

International Journal of Computing, 2019

This paper presents the development of several models of a deep convolutional auto-encoder in the... more This paper presents the development of several models of a deep convolutional auto-encoder in the Caffe deep learning framework and their experimental evaluation on the example of MNIST dataset. We have created five models of a convolutional auto-encoder which differ architecturally by the presence or absence of pooling and unpooling layers in the auto-encoder’s encoder and decoder parts. Our results show that the developed models provide very good results in dimensionality reduction and unsupervised clustering tasks, and small classification errors when we used the learned internal code as an input of a supervised linear classifier and multi-layer perceptron. The best results were provided by a model where the encoder part contains convolutional and pooling layers, followed by an analogous decoder part with deconvolution and unpooling layers without the use of switch variables in the decoder part. The paper also discusses practical details of the creation of a deep convolutional au...

Machine Learning with Certainty: A Requirement for Intelligent Process Automation

2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), 2018

Many dull but non-trivial tasks in the financial industry are performed by human staff, but could... more Many dull but non-trivial tasks in the financial industry are performed by human staff, but could be automated by intelligent systems that observe staff and learn the correct processes. Such systems present unique design challenges, because errors are so costly they are simply unacceptable (referring a case to humans for manual processing would be better than processing it incorrectly). In this paper the system is cast as a classifier which learns to trigger an action under the right conditions. Several design constraints for the classifier are proposed, including interpretability and the absolute need to avoid any false positives. A gap is identified in current literature with respect to these constraints. We present "SureTree" - a simple meta-learning algorithm for training decision trees with minimal risk of false positives - and compare its performance to several conventional alternatives.

Learning to Predict Consequences as a Method of Knowledge Transfer in Reinforcement Learning

IEEE transactions on neural networks and learning systems, Jan 17, 2017

The reinforcement learning (RL) paradigm allows agents to solve tasks through trial-and-error lea... more The reinforcement learning (RL) paradigm allows agents to solve tasks through trial-and-error learning. To be capable of efficient, long-term learning, RL agents should be able to apply knowledge gained in the past to new tasks they may encounter in the future. The ability to predict actions' consequences may facilitate such knowledge transfer. We consider here domains where an RL agent has access to two kinds of information: agent-centric information with constant semantics across tasks, and environment-centric information, which is necessary to solve the task, but with semantics that differ between tasks. For example, in robot navigation, environment-centric information may include the robot's geographic location, while agent-centric information may include sensor readings of various nearby obstacles. We propose that these situations provide an opportunity for a very natural style of knowledge transfer, in which the agent learns to predict actions' environmental conseq...

Computational Properties of the Hippocampus Increase the Efficiency of Goal-Directed Foraging through Hierarchical Reinforcement Learning

Frontiers in Computational Neuroscience, 2016

The mammalian brain is thought to use a version of Model-based Reinforcement Learning (MBRL) to g... more The mammalian brain is thought to use a version of Model-based Reinforcement Learning (MBRL) to guide "goal-directed" behavior, wherein animals consider goals and make plans to acquire desired outcomes. However, conventional MBRL algorithms do not fully explain animals' ability to rapidly adapt to environmental changes, or learn multiple complex tasks. They also require extensive computation, suggesting that goal-directed behavior is cognitively expensive. We propose here that key features of processing in the hippocampus support a flexible MBRL mechanism for spatial navigation that is computationally efficient and can adapt quickly to change. We investigate this idea by implementing a computational MBRL framework that incorporates features inspired by computational properties of the hippocampus: a hierarchical representation of space, "forward sweeps" through future spatial trajectories, and context-driven remapping of place cells. We find that a hierarchical abstraction of space greatly reduces the computational load (mental effort) required for adaptation to changing environmental conditions, and allows efficient scaling to large problems. It also allows abstract knowledge gained at high levels to guide adaptation to new obstacles. Moreover, a context-driven remapping mechanism allows learning and memory of multiple tasks. Simulating dorsal or ventral hippocampal lesions in our computational framework qualitatively reproduces behavioral deficits observed in rodents with analogous lesions. The framework may thus embody key features of how the brain organizes model-based RL to efficiently solve navigation and other difficult tasks.

Context-switching and adaptation: Brain-inspired mechanisms for handling environmental changes

2016 International Joint Conference on Neural Networks (IJCNN), 2016

Reinforcement learning (RL) allows an intelligent agent to learn optimal behavior as it interacts... more Reinforcement learning (RL) allows an intelligent agent to learn optimal behavior as it interacts with its environment. Conventional model-based RL algorithms learn rapidly, but can be slow to adapt to sudden changes in the environment. Animals' brains, however, are thought to employ model-based RL mechanisms for learning, but are able to adapt to changes with relative ease. By employing "transfer learning", they can recycle previously learned information to solve new problems with minimal new learning. We developed two brain-inspired methods that can allow model-based RL to cope with changes to the underlying process being learned: hierarchical state abstraction, and context-switching. Hierarchical state abstraction allows a previously-learned model to be efficiently adapted for use in a new task, while context switching allows learned models to be saved and recalled at the appropriate times. We test these mechanisms using grid-world simulations in which the goal remains constant, but contingencies for reaching it frequently change. These mechanisms allow an agent to significantly outperform a conventional model-based RL algorithm in the task.

How does pressure configuration affect Cobb angle during AIS brace casting?

Scoliosis, 2015

This work measured the magnitude and distance between pressures applied to the torso of AIS patie... more This work measured the magnitude and distance between pressures applied to the torso of AIS patients by an orthotist during brace casting. The instantaneous effect of the pressure configuration on Cobb angle was investigated using ultrasound. The objective was to use real-time measurements to confirm that pressure configuration affects Cobb angle predictably.

Predicting the outcome of AIS brace treatment using expert judgement and a fuzzy model

Scoliosis, 2015

This work measured how accurately the outcome of brace treatment can be predicted, using only inf... more

Predicting success or failure of brace treatment for adolescents with idiopathic scoliosis

Medical & biological engineering & computing, Jan 23, 2015

Adolescent idiopathic scoliosis (AIS) is a three-dimensional spinal deformity. Brace treatment is... more Adolescent idiopathic scoliosis (AIS) is a three-dimensional spinal deformity. Brace treatment is a common non-surgical treatment, intended to prevent progression (worsening) of the condition during adolescence. Estimating a braced patient's risk of progression is an essential part of planning treatment, so method for predicting this risk would be a useful decision support tool for practitioners. This work attempts to discover whether failure of brace treatment (progression) can be predicted at the start of treatment. Records were obtained for 62 AIS patients who had completed brace treatment. Subjects were labeled as "progressive" if their condition had progressed despite brace treatment and "non-progressive" otherwise. Wrapper-based feature selection selected two useful predictor variables from a list of 14 clinical measurements taken from the records. A logistic regression model was trained to classify patients as "progressive" or "non-progr...

Prescriptive analytics applied to brace treatment for AIS: a pilot demonstration

Scoliosis, 2015

Prescriptive analytics is a concept combining statistical and computer sciences to prescribe an o... more Prescriptive analytics is a concept combining statistical and computer sciences to prescribe an optimal course of action, based on predictions of possible future events. In this simulation study we investigate using prescriptive analytics to recommend optimal in-brace corrections for braced Adolescent Idiopathic Scoliosis (AIS) patients. The objectives were to estimate the efficacy of these recommendations, ultimately working toward improved brace design protocols. Data was obtained for 90 AIS patients who had finished brace treatment at our center (60 full-time and 30 nighttime braces). Rates of ≥6 degree progression were 53% for daytime and 30% for nighttime braces. A modeling technique previously developed by our group was used to predict these patients' likely treatment outcomes given a range of in-brace corrections - the model was blinded to the true outcomes during this process. Each patient's 'recommended' correction was identified as the least aggressive corr...

Human Experts’ and a Fuzzy Model's Predictions of Outcomes of Scoliosis Treatment: A Comparative Analysis

IEEE Transactions on Biomedical Engineering, 2015

Brace treatment is the most commonly used nonsurgical treatment for adolescents with idiopathic s... more Brace treatment is the most commonly used nonsurgical treatment for adolescents with idiopathic scoliosis. However, brace treatment is not always successful and the factors influencing its success are not completely clear. This makes treatment outcome difficult to predict. A computer model which can accurately predict treatment outcomes could potentially provide valuable treatment recommendations. This paper describes a fuzzy system that includes a prediction model and a decision support engine. The model was constructed using conditional fuzzy c-means clustering to discover patterns in retrospective patient data. The model's ability to predict treatment outcome was compared to the ability of eight Scoliosis experts. The model and experts each predicted treatment outcome retrospectively for 28 braced patients, and these predictions were compared to the actual outcomes. The model outperformed all but one expert individually and performed similarly to the experts as a group. These results suggest that the fuzzy model is capable of providing meaningful treatment recommendations. This study offers the first model for this application whose performance has been shown to be at or above the human expert level.

An advanced compliance monitor for patients undergoing brace treatment for idiopathic scoliosis

Medical Engineering & Physics, 2015

Adolescent idiopathic scoliosis is a spinal deformity affecting 2-3% of adolescents. Brace treatm... more Adolescent idiopathic scoliosis is a spinal deformity affecting 2-3% of adolescents. Brace treatment, the most common non-surgical treatment, uses a hard plastic orthotic shell to prevent progression of the deformity. Previous studies have found association between treatment outcome and patients' compliance with the prescribed brace-wear regimen. However, the exact relationship between compliance and treatment outcome has yet to be elucidated. Current compliance monitoring techniques may not be providing enough information about patients' brace-wear habits. Building on previous work, we present a new compliance monitor which records both temperature and force applied to the patient's body. The combination of temperature and force readings shows both how often and how tightly the brace is worn. The new monitor is designed for minimal size and power consumption, measuring 5.2 cm × 2.5 cm × 0.8 cm, with a battery life of approximately one year. Seven patients wore the monitor in this pilot study. The temperature-based compliance estimate differed significantly from the force-based estimate in four out of seven cases. This suggests that some patients may wear their braces very loosely, and that existing temperature-only or force-only compliance monitors may not be giving a complete picture of brace-wear habits.