Hierarchical Temporal Memory

Data Drift is the phenomenon where the generating model behind the data changes over time. Due to data drift, any model built on the past training data becomes less relevant and inaccurate over time. Thus, detecting and controlling for data drift is critical in machine learning models. Hierarchical Temporal Memory (HTM) is a machine learning model developed by Jeff Hawkins, inspired by how the human brain processes information. It is a biologically inspired model of memory that is similar in structure to the neocortex, and whose performance is claimed to be comparable to state of the art models in detecting anomalies in time series data. Another unique benefit of HTMs is its independence from training and testing cycle; all the learning takes place online with streaming data and no separate training and testing cycle is required. In sequential learning paradigm, Sequential Probability Ratio Test (SPRT) offers some unique benefit for online learning and inference. This paper proposes a novel hybrid framework combining HTM and SPRT for real-time data drift detection and anomaly identification. Unlike existing data drift methods, our approach eliminates frequent retraining and ensures low false positive rates. HTMs currently work with one dimensional or univariate data. In a second study, we also propose an application of HTM in multidimensional supervised scenario for anomaly detection by combining the outputs of multiple HTM columns, one for each dimension of the data, through a neural network. Experimental evaluations demonstrate that the proposed method outperforms conventional drift detection techniques like the Kolmogorov-Smirnov (KS) test, Wasserstein distance, and Population Stability Index (PSI) in terms of accuracy, adaptability, and computational efficiency. Our experiments also provide insights into optimizing hyperparameters for real-time deployment in domains such as Telecom.

Power system faults are highly undesirable but unavoidable property of any physical power system infrastructure due to the imperfection in the design, manufacture and operation of the associated physical equipment such as the relays, transmission lines and circuit breakers. In particular, the transmission lines serve the critical function of interconnecting various power system networks and any fault on it can be disastrous if not identified on time. Thus, steps have to be taken to mitigate such issues earlier on by following a real-time perspective. The problem of determining in advance, the likelihood of a fault in advance is introduced as the incipient fault detection problem. In this paper, a systematic review of the research on the important topic of incipient fault detection on power transmission lines is presented. The review addresses some of the popular Machine Learning (ML) algorithms or techniques applied both traditional and emerging. It further seeks to examine the problem of incipient fault detection and seeks to provide insight into the demographics and type of ML in use. It also specifically provides a succinct discussion of some of the emerging detection approaches based on the use of continual learning property. The benefits of the continual learning are equally highlighted.

In this paper, a new software tool for HTM Optimal Power Flow (HTM-OPF) studies based on an emerging artificial intelligence technology called Hierarchical Temporal Memory (HTM) is presented. The HTM is based on the Cortical Learning Algorithm which is a significant improvement on the artificial neuron-based networks (ANN's) models and offers meta-cognition capabilities building more advanced neural model solutions with both active and predictive output states. HTM explores the best features of the neo-cortex by providing a set of column-like feed-forward activators with good inferential qualities to solve a specific set of generic problems. For the OPF problem, cost minimization prediction and classification were achieved using a collection of HTM cells working concurrently in order to compensate for energy losses and reduce overall systems costs based on data-driven technique. The tool was applied to a sample data obtained from three different plants to validate its predictive efficacy considering classification accuracies and percent variation in training data. The simulation results showed that increasing training data leads to higher classification accuracies but this may impact the computational run-times.

Neuromorphic systems that learn and predict from streaming inputs hold significant promise in pervasive edge computing and its applications. In this paper, a neuromorphic system that processes spatio-temporal information on the edge is proposed. Algorithmically, the system is based on hierarchical temporal memory that inherently offers online learning, resiliency, and fault tolerance. Architecturally, it is a full custom mixed-signal design with an underlying digital communication scheme and analog computational modules. Therefore, the proposed system features reconfigurability, real-time processing, low power consumption, and low-latency processing. The proposed architecture is benchmarked to predict on real-world streaming data. The network's mean absolute percentage error on the mixed-signal system is 1.129X lower compared to its baseline algorithm model. This reduction can be attributed to device non-idealities and probabilistic formation of synaptic connections. We demonstrate that the combined effect of Hebbian learning and network sparsity also plays a major role in extending the overall network lifespan. We also illustrate that the system offers 3.46X reduction in latency and 77.02X reduction in power consumption when compared to a custom CMOS digital design implemented at the same technology node. By employing specific low power techniques, such as clock gating, we observe 161.37X reduction in power consumption.

The Hierarchical Temporal Memory Cortical Learning Algorithm (HTM CLA) is an algorithm inspired by the biological functioning of the neo-cortex, which combines spatial pattern recognition and temporal sequence learning. It organizes neurons in layers of column-like units built from many neurons such that the units are connected into structures called regions (areas). Layers can be hierarchically organized and can further be connected into more complex networks, which would allow to implement higher cognitive capabilities like invariant representations. However, a complex topology and a potentially high number of neurons would require more computing power than a single machine even with multiple cores or a GPU could provide. This paper aims to improve the HTM CLA by enabling it to run on multiple nodes in a highly distributed system of processors; to achieve this we use the Actor Programming Model. The proposed concept also makes use of existing cloud and server less technology and it enables easy setup and operation of cortical algorithms in a distributed environment. The proposed model is based on a mathematical theory and computation model, which targets massive concurrency. Using this model drives different reasoning about concurrent execution and should enable flexible distribution of cortical computation logic across multiple physical nodes. This work is the first one about the parallel HTM Spatial Pooler on multiple nodes with named computational model. With the increasing popularity of cloud computing and serverless architectures, this work is the first step towards proposing interconnected independent HTM CLA units in an elastic cognitive network. Thereby it can provide an alternative to deep neuronal networks, with theoretically unlimited scale in a distributed cloud environment. This paper specifically targets the redesign of a single Spatial Pooler unit.

This study was conducted to proposea hierarchical temporal memory (HTM) approach for fault detection in the Onitsha-Alaoji transmission line in Nigeria. Using a mixed research method, the study employed the Hawkins HTM model with two objectives and their corresponding research questions. The study gathered primary and secondary data to detect and evaluate faults in the Onitsha-Alaoji transmission line in Nigeria using HTM and compares its efficacy to current fault detection methods. With the use of simulation and descriptive methods of data analysis, results showed that partial discharge (PD) is the fault type that is being detected and it is commonly found as a fault leading to transmission line errors. More so, fault detection simulations were conducted at 40 km using typical power spectral density analysis. The first fundamental shifted from about 10 kHz to roughly 13 kHz during a fault. The HTM model outperformed sequence learning methods, resulting in a 90% mean test classification accuracy (CA) over extreme learning machine(ELM) and online sequential learning-extreme learning machine (OS-ELM), with OS-ELM performing poorly.The study concluded and recommended that the proposed HTM model be used to identify various PD fault types that plague the Onitsha-Alaoji transmission line in Nigeria. With the increased efficacy and reliability of the proposed model compared to existing methods, it is recommended for future implementation in this transmission line and potentially other fault-prone power transmission lines in Nigeria.

This study was conducted to propose a hierarchical temporal memory (HTM) approach for fault detection in the Onitsha-Alaoji transmission line in Nigeria. Using a mixed research method, the study employed the Hawkins HTM model with two objectives and their corresponding research questions. The study gathered primary and secondary data to detect and evaluate faults in the Onitsha-Alaoji transmission line in Nigeria using HTM and compares its efficacy to current fault detection methods. With the use of simulation and descriptive methods of data analysis, results showed that partial discharge (PD) is the fault type that is being detected and it is commonly found as a fault leading to transmission line errors. More so, fault detection simulations were conducted at 40 km using typical power spectral density analysis. The first fundamental shifted from about 10 kHz to roughly 13 kHz during a fault. The HTM model outperformed sequence learning methods, resulting in a 90% mean test classification accuracy (CA) over extreme learning machine(ELM) and online sequential learning-extreme learning machine (OS-ELM), with OS-ELM performing poorly.The study concluded and recommended that the proposed HTM model be used to identify various PD fault types that plague the Onitsha-Alaoji transmission line in Nigeria. With the increased efficacy and reliability of the proposed model compared to existing methods, it is recommended for future implementation in this transmission line and potentially other fault-prone power transmission lines in Nigeria.

Dynamic trust models and replication approaches that adequately address the security needs of wireless cognitive ad hoc networks (MANET) have proven quite difficult to develop. The inherent decentralized nature of ad hoc networks and their collaborative services give rise to a series of vulnerabilities that can be exploited by malicious entities. This problem is exacerbated by the fact that risk assessment models show high levels of complexity, which suggest significant domain constraints and lack of true human-like reasoning. Recent work in the area of HTM demonstrates the ability to mimic higher level cognitive skills. This paper discusses the potential of using HTM to improve human-like reasoning in the replication of trust information in cognitive MANETs.

In the decade since Jeff Hawkins proposed Hierarchical Temporal Memory (HTM) as a model of neocortical computation, the theory and the algorithms have evolved dramatically. This paper presents a detailed description of HTM's Cortical Learning Algorithm (CLA), including for the first time a rigorous mathematical formulation of all aspects of the computations. Prediction Assisted CLA (paCLA), a refinement of the CLA is presented, which is both closer to the neuroscience and adds significantly to the computational power. Finally, we summarise the key functions of neocortex which are expressed in paCLA implementations.

In this article, we evaluate the ability of Hierarchical Temporal Memories (HTM) to process values coming from sensor chains. We present a study on the impact of the HTM parameterization on its ability to predict input values and its robustness to sensor faults. The HTM is evaluated on simulated multivariate time series comprising several causal relations between variables. The results show the ability of HTM to predict future values of multivariate time series and to be robust to sensor faults. We then present which parameters most impact HTM prediction performance and its robustness to faults.

When we apply Image Retrieval techniques to large image Databases .It provides restriction of search space to provide adequate response time. This restriction can be done minimized by using Clustering technique to partition the image dataset into subspaces of similar elements .In this article we will apply different clustering algorithms on large image database and then evaluate and analyse the performance of these algorithms to determine which algorithm is best for image retrieval.

Hierarchical Temporal Memory (HTM) is a biomimetic machine learning algorithm imbibing the structural and algorithmic properties of the neocortex. Two main functional components of HTM that enable spatio-temporal processing are the spatial pooler and temporal memory. In this research, we explore a scalable hardware realization of the spatial pooler closely coupled with the mathematical formulation of spatial pooler. This class of neuromorphic algorithms are advantageous in solving a subset of the future engineering problems by extracting nonintuitive patterns in complex data. The proposed architecture, Non-volatile HTM (NVHTM), leverages large-scale solid state flash memory to realize a optimal memory organization, area and power envelope. A behavioral model of NVHTM is evaluated against the MNIST dataset, yielding 91.98% classification accuracy. A full custom layout is developed to validate the design in a TSMC 180nm process. The area and power profile of the spatial pooler are 30....

In these days an image retrieval system has become a challenging task. Many systems based on the text based retrieval but the need of image based retrieval system that takes an image as the input query and retrieves images based on image content is more complicated task. Content Based Image Retrieval is an approach for retrieving semantically-relevant images from an image database based on automatically-derived image features. The exclusive aspect of the system is the utilization of hierarchical and multilayer network which includes RBFN . The proposed procedure consists of two stages. First, here we are going to filter most of the images in the hierarchical clustering and then apply the clustered images to RBFN network, so that we can get better favored image results.

The aim of this paper is to report on a pilot application of a bio-inspired intelligent network model, called Hierarchical Temporal Memory (HTM), for recognition (detection) of untrustworthy manipulation with an Automatic Teller Machine (ATM). HTM was used as a crucial part of an anomaly detection system to recognize hard-to-identifiable faces, i.e., faces with a mask, covered with a scarf, or wearing sunglasses. Those types of face occlusion can be a good indicator of potentialy malicious intentions of an ATM user. In the presented system, the Kinect camera was used for acquisition of video image sequences. The Kinect's depth output along with skeleton tracking was used as a basis of the color image segmentation. To test the proposed system, experiments have been carried out in which several participants performed normal and untrustworthy actions using an ATM simulator. The output of the face classification system can assist a security personnel in surveillance tasks.

Hierarchical temporal memory (HTM) is a biologically inspired framework that can be used to learn invariant representations of patterns in a wide range of applications. Classical HTM learning is mainly unsupervised, and once training is completed, the network structure is frozen, thus making further training (i.e., incremental learning) quite critical. In this letter, we develop a novel technique for HTM (incremental) supervised learning based on gradient descent error minimization. We prove that error backpropagation can be naturally and elegantly implemented through native HTM message passing based on belief propagation. Our experimental results demonstrate that a two-stage training approach composed of unsupervised pretraining and supervised refinement is very effective (both accurate and efficient). This is in line with recent findings on other deep architectures.

Hierarchical Temporal Memory (HTM) is a neuromorphic algorithm that emulates sparsity, hierarchy and modularity resembling the working principles of neocortex. Feature encoding is an important step to create sparse binary patterns. This sparsity is introduced by the binary weights and random weight assignment in the initialization stage of the HTM. We propose the alternative deterministic method for the HTM initialization stage, which connects the HTM weights to the input data and preserves natural sparsity of the input information. Further, we introduce the hardware implementation of the deterministic approach and compare it to the traditional HTM and existing hardware implementation. We test the proposed approach on the face recognition problem and show that it outperforms the conventional HTM approach.

Detecting anomalies in time series data plays a vital role in the various applications of diagnosis systems. The importance of anomaly detection is increased by its ability to detect abnormalities in Electrocardiogram (ECG) signals to generate alerts for cardiac health problems. An ECG is a time series that provides essential information about the electrical activity of the heart and is used in the diagnosis of numerous heart diseases. An accurate ECG streaming analytics approach requires continuous learning and adaptation in changing data behaviors. We aim to diagnose ECG by investigating healthy ECG and ECG with cardiological disorders by detecting anomalies in ECG signals. The main objective of this paper is to develop an efficient unsupervised diagnosing system for ECG streaming data based on an online sequence memory algorithm called Hierarchical Temporal Memory (HTM). The HTM is based on neural network and machine learning algorithm for continuous learning tasks. The proposed customization of the HTM algorithm based on our problem domain provides a significant performance results of detection of anomalies in the ECG signals.

When we apply Image Retrieval techniques to large image Databases .It provides restriction of search space to provide adequate response time. This restriction can be done minimized by using Clustering technique to partition the image dataset into subspaces of similar elements .In this article we will apply different clustering algorithms on large image database and then evaluate and analyse the performance of these algorithms to determine which algorithm is best for image retrieval.

Recent discoveries clarified the role of Layers 2, 3 and 4 of the neocortex as recognizers and predictors of feedforward patterns. However, the roles of Layers 5 and 6 are still unclear. In this paper, I propose that the function of Layer 5 is to recognize patterns representing *needs* for manipulations: actions that can be both physical (e.g., motor actions) and mental (e.g., cognitive tasks such as mental rotations). Such manipulations are used to help us survive (e.g. fight, find food) and to disambiguate the local context of feedforward input (i.e., understanding to which object do sensorial stimuli belong to).

Abstract- Content-based image retrieval. (CBIR) aims at developing techniques that support effective searching and browsing of large image digital libraries based on automatically derived image features. In this Image retrieval system, query results are a set of images sorted by feature similarities with respect to the query. However images with high feature similarities to the query may be very different from the query in terms of semantics. In this paper we proposed a K-means clustering techniques so that we can get better favored image result.

Content-based image retrieval. (CBIR) aims at developing techniques that support effective searching and browsing of large image digital libraries based on automatically derived image features. In this Image retrieval system, query results are a set of images sorted by feature similarities with respect to the query. However images with high feature similarities to the query may be very different from the query in terms of semantics. In this paper we proposed a K-means clustering techniques so that we can get better favored image result. KeywordsImage Retrieval, CBIR, Text annotation, Karhunen-Loeve Transform (KLT), column-wise clustering.

In the paper, we propose an alternative approach to the temporal pooling of the Hierarchical Temporal Memory (HTM) - a biologically inspired large-scale model of the neocortex. The novel method is compared with the conventional temporal pooling algorithm based on a smooth traversal of training images and their efficiency is demonstrated on a problem of the position, scale, and rotation-invariant recognition of simple geometric objects. Results have shown that the proposed method provides significantly faster convergence to the theoretical maximum classification accuracy than the conventional one.

Hierarchical Temporal Memory (HTM) is a biomimetic machine learning algorithm imbibing the structural and algorithmic properties of the neocortex. Two main functional components of HTM that enable spatio-temporal processing are the spatial pooler and temporal memory. In this research, we explore a scalable hardware realization of the spatial pooler closely coupled with the mathematical formulation of spatial pooler. This class of neuromorphic algorithms are advantageous in solving a subset of the future engineering problems by extracting nonintuitive patterns in complex data. The proposed architecture, Non-volatile HTM (NVHTM), leverages large-scale solid state flash memory to realize a optimal memory organization, area and power envelope. A behavioral model of NVHTM is evaluated against the MNIST dataset, yielding 91.98% classification accuracy. A full custom layout is developed to validate the design in a TSMC 180nm process. The area and power profile of the spatial pooler are 30....

When we apply Image Retrieval techniques to large image Databases .It provides restriction of search space to provide adequate response time. This restriction can be done minimized by using Clustering technique to partition the image dataset into subspaces of similar elements .In this article we will apply different clustering algorithms on large image database and then evaluate and analyse the performance of these algorithms to determine which algorithm is best for image retrieval.

In this paper, the author examines the differences in handling complexity between the human brain and artificial intelligences. It will be shown how the current modularity and tendency to reduce complexity of most current AIs is a limit to their potential to reach artificial general intelligence (AGI). Finally, techniques to address these limitations and to properly address complexity are addressed.

Hierarchical Temporal Memory (HTM) is a biomimetic machine learning algorithm imbibing the structural and algorithmic properties of the neocortex. Two main functional components of HTM that enable spatio-temporal processing are the spatial pooler and temporal memory. In this research, we explore a scalable hardware realization of the spatial pooler closely coupled with the mathematical formulation of spatial pooler. This class of neuromorphic algorithms are advantageous in solving a subset of the future engineering problems by extracting nonintuitive patterns in complex data. The proposed architecture, Non-volatile HTM (NVHTM), leverages large-scale solid state flash memory to realize a optimal memory organization, area and power envelope. A behavioral model of NVHTM is evaluated against the MNIST dataset, yielding 91.98% classification accuracy. A full custom layout is developed to validate the design in a TSMC 180nm process. The area and power profile of the spatial pooler are 30....

Food is a requirement for living, and traded in enormous amounts everyday. The globalization has led to optimization of the supermarkets and that a lot of stores have introduced self-scanning systems at check out and payment. When articles such as vegetables and fruits are traded the process becomes slower because the packages usually do not wear barcodes, which have to be added manually. This is a problem and the purpose with self-scanning drops out. In this thesis, a recognition system is built with the purpose to be used in self-scanning systems. The system thresholds the original image into a binary image. The binary image is sent to an advanced type of Neural Network called Hierarchical Temporal Memory. Such a network is independent of color, size, spatial space and rotations. These properties make it suitable for the given task. Two sorts of fruits were tested and the algorithm gave the accurate prediction in 97.5 % when tested on previously unseen images.

In these days an image retrieval system has become a challenging task. Many systems based on the text based retrieval but the need of image based retrieval system that takes an image as the input query and retrieves images based on image content is more complicated task. Content Based Image Retrieval is an approach for retrieving semantically-relevant images from an image database based on automatically-derived image features. The exclusive aspect of the system is the utilization of hierarchical and multilayer network which includes RBFN . The proposed procedure consists of two stages. First, here we are going to filter most of the images in the hierarchical clustering and then apply the clustered images to RBFN network, so that we can get better favored image results.

Food is a requirement for living, and traded in enormous amounts everyday. The globalization has led to optimization of the supermarkets and that a lot of stores have introduced self-scanning systems at check out and payment. When articles such as vegetables and fruits are traded the process becomes slower because the packages usually do not wear barcodes, which have to be added manually. This is a problem and the purpose with self-scanning drops out. In this thesis, a recognition system is built with the purpose to be used in self-scanning systems. The system thresholds the original image into a binary image. The binary image is sent to an advanced type of Neural Network called Hierarchical Temporal Memory. Such a network is independent of color, size, spatial space and rotations. These properties make it suitable for the given task. Two sorts of fruits were tested and the algorithm gave the accurate prediction in 97.5 % when tested on previously unseen images.

Neuromorphic systems that learn and predict from streaming inputs hold significant promise in pervasive edge computing and its applications. In this paper, a neuromorphic system that processes spatio-temporal information on the edge is proposed. Algorithmically, the system is based on hierarchical temporal memory that inherently offers online learning, resiliency, and fault tolerance. Architecturally, it is a full custom mixed-signal design with an underlying digital communication scheme and analog computational modules. Therefore, the proposed system features reconfigurability, real-time processing, low power consumption, and low-latency processing. The proposed architecture is benchmarked to predict on real-world streaming data. The network's mean absolute percentage error on the mixed-signal system is 1.129X lower compared to its baseline algorithm model. This reduction can be attributed to device non-idealities and probabilistic formation of synaptic connections. We demonstrate that the combined effect of Hebbian learning and network sparsity also plays a major role in extending the overall network lifespan. We also illustrate that the system offers 3.46X reduction in latency and 77.02X reduction in power consumption when compared to a custom CMOS digital design implemented at the same technology node. By employing specific low power techniques, such as clock gating, we observe 161.37X reduction in power consumption.

In this paper a pattern classification and object recognition approach based on bio-inspired techniques is presented. It exploits the Hierarchical Temporal Memory (HTM) topology, which imitates human neocortex for recognition and categorization tasks. The HTM comprises a hierarchical tree structure that exploits enhanced spatiotemporal modules to memorize objects appearing in various orientations. In accordance with HTM's biological inspiration, human vision mechanisms can be used to preprocess the input images. Therefore, the input images undergo a saliency computation step, revealing the plausible information of the scene, where a human might fixate. The adoption of the saliency detection module releases the HTM network from memorizing redundant information and augments the classification accuracy. The efficiency of the proposed framework has been experimentally evaluated in the ETH-80 dataset, and the classification accuracy has been found to be greater than other HTM systems.

This paper presents a deep-learning method for pattern classification and object recognition. The proposed methodology is based on an optimised version of Hierarchical Temporal Memory algorithm (HTM) and it preserves its basic structure, along with a tree structure of connected nodes. The tree structured scheme is inspired by the human neocortex, which gives us great capabilities for recognition and categorisation. The proposed method is enriched with more representative quantisation centres using an adaptive Neural Gas algorithm, and a more accurate and dense grouping by applying a graph clustering technique. Sparse representation using L1 norm minimisation is embedded as a liaison between the quantisation centres and their grouping, reinforcing the proposed technique with advantages, such as a natural discrimination capability. The proposed work is experimentally compared with the aforementioned techniques as well as with state of the art algorithms, presenting a better classification performance.

A large number of real world applications, like user support systems, can still not be performed easily by conventional algorithms in comparison with the human brain. Recently, such intelligence has often been reached by using probability based systems. This paper presents results on the implementation of one such user support system, namely an intention estimation information appliance system, on a Bayesian Network as well as Hierarchical Temporal Memory. The latter is a new and quite promising soft computing platform modelling the human brain, though currently only available as a software model. A second part of the paper therefore focuses on a possible VLSI architecture for Hierarchical Temporal Memory. Since it models the human brain, communication as well as memory are of high importance for this VLSI architecture.

A large number of real world applications, such as image recognition and understanding, can still not be performed easily by conventional algorithms in comparison with the human brain. Implementing applications that require such intelligence, might therefore require a different approach, for which Hierarhical Temporal Memory (HTM) seems a promising framework. Currently HTM exists as a software model. However, this software implementation has its performance limitations and a distinct learning and operating mode. This paper proposes a possible architecture for VLSI implementation that also allows learning during normal operation.

Hierarchical Temporal Memory (HTM)-Spatial Pooler (SP) is a Learning Algorithm for learning of spatial patterns inspired by the neo-cortex. It is designed to learn the pattern in a few iteration steps and to generate the Sparse Distributed Representation (SDR) of the input. It encodes spatially similar inputs into the same or similar SDRs memorized as a population of active neurons organized in groups called micro-columns. Findings in this research show that produced SDRs can be forgotten during the training progress, which causes the SP to learn the same pattern again and converts into the new SDR. This work shows that instable learning behaviour of the SP is caused by the internal boosting algorithm inspired by the homeostatic plasticity mechanism. Previous findings in neurosciences show that this mechanism is only active during the development of newborn mammals and later deactivated or shifted from cortical layer L4, where the SP is supposed to be active. The same mechanism was used in this work. The SP algorithm was extended with the new homeostatic plasticity component that controls the boosting and deactivates it after entering the stable state. Results show that learned SDRs remain stable during the lifetime of the Spatial Pooler.

Hierarchical Temporal Memory (HTM)-Spatial Pooler (SP) is a Learning Algorithm for learning of spatial patterns inspired by the neo-cortex. It is designed to learn the pattern in a few iteration steps and to generate the Sparse Distributed Representation (SDR) of the input. It encodes spatially similar inputs into the same or similar SDRs memorized as a population of active neurons organized in groups called micro-columns. Findings in this research show that produced SDRs can be forgotten during the training progress, which causes the SP to learn the same pattern again and converts into the new SDR. This work shows that instable learning behaviour of the SP is caused by the internal boosting algorithm inspired by the homeostatic plasticity mechanism. Previous findings in neurosciences show that this mechanism is only active during the development of newborn mammals and later deactivated or shifted from cortical layer L4, where the SP is supposed to be active. The same mechanism was used in this work. The SP algorithm was extended with the new homeostatic plasticity component that controls the boosting and deactivates it after entering the stable state. Results show that learned SDRs remain stable during the lifetime of the Spatial Pooler.

The Hierarchical Temporal Memory Cortical Learning Algorithm (HTM CLA) is an algorithm inspired by the biological functioning of the neo-cortex, which combines spatial pattern recognition and temporal sequence learning. It organizes neurons in layers of column-like units built from many neurons such that the units are connected into structures called regions (areas). Layers can be hierarchically organized and can further be connected into more complex networks, which would allow to implement higher cognitive capabilities like invariant representations. However, a complex topology and a potentially high number of neurons would require more computing power than a single machine even with multiple cores or a GPU could provide. This paper aims to improve the HTM CLA by enabling it to run on multiple nodes in a highly distributed system of processors; to achieve this we use the Actor Programming Model. The proposed concept also makes use of existing cloud and server less technology and it enables easy setup and operation of cortical algorithms in a distributed environment. The proposed model is based on a mathematical theory and computation model, which targets massive concurrency. Using this model drives different reasoning about concurrent execution and should enable flexible distribution of cortical computation logic across multiple physical nodes. This work is the first one about the parallel HTM Spatial Pooler on multiple nodes with named computational model. With the increasing popularity of cloud computing and serverless architectures, this work is the first step towards proposing interconnected independent HTM CLA units in an elastic cognitive network. Thereby it can provide an alternative to deep neuronal networks, with theoretically unlimited scale in a distributed cloud environment. This paper specifically targets the redesign of a single Spatial Pooler unit.

Nigerian Stock Exchange is the institution empowered by the Nigerian law to Manage and regulates the activities of the Nigerian capital market and therefore a key player role as they set the rules of the game despite the volatile and unpredictable nature of the market. The use of machine learning algorithms to predict the future values and other variables in the market through the use of time series data has proven to be of immense advantage to market players globally in the last two decades. In this work, the time serial movement of stock prices over a period of time extracted from daily official list of Nigeria Stock Exchange is used to predict the future values of other variables through the use of time series data and moving average methodology. Digital signal processing based on a biological neural network called Hierarchical Temporal Memory (HTM) was used for stock price data encoding and predictions and show high performance accuracy.

In this paper, the author examines the differences in handling complexity between the human brain and artificial intelligences.
It will be shown how the current modularity and tendency to reduce complexity of most current AIs is a limit to their potential to reach artificial general intelligence (AGI).
Finally, techniques to address these limitations and to properly address complexity are ad- dressed.

A current bottleneck that prevents Machine Learning (ML) from being successful outside of a few restricted fields such as chess playing and highway driving is its impairment in ap- propriately using context to infer the meaning of what it is observing. This paper describes techniques to allow ML systems to derive meaning from context, derived from how the human cortex works.
In particular, this paper shows how the multiplication a horizontal vector representing a Sparse Distributed Representation (SDR) of patterns in sensory data by a vertical vector representing a SDR of patterns in context data followed by a pattern recognition operation on the result- ing matrix results in the integration of relevant context and in the output of data containing meaning.

Recent works demonstrated the usefulness of temporal coherence to regularize supervised training or to learn invariant features with deep architectures. In particular, enforcing smooth output changes while presenting temporally-closed frames from video sequences, proved to be an effective strategy. In this paper we prove the efficacy of temporal coherence for semi-supervised incremental tuning. We show that a deep architecture, just mildly trained in a supervised manner, can progressively improve its classification accuracy, if exposed to video sequences of unlabeled data. The extent to which, in some cases, a semi-supervised tuning allows to improve classification accuracy (approaching the supervised one) is somewhat surprising. A number of control experiments pointed out the fundamental role of temporal coherence.

In this paper an optimized classification method for object recognition is presented. The proposed method is based on the Hierarchical Temporal Memory (HTM), which stems from the memory prediction theory of the human brain. As in HTM, this method comprises a tree structure of connected computational nodes, whilst utilizing different rules to memorize objects appearing in various orientations. These rules involve both the spatial and the temporal module. As HTM is inspired from brain activity, its input should also comply with the human vision system. Thus, for the representation of the input images the logpolar was given preference to the Cartesian one. As compared to the original HTM method, experimental results exhibit performance enhancements with this approach, in recognition and categorization applications. Results obtained prove that the proposed method is more accurate and faster in training, whilst retaining the network robustness in multiple orientation variations.

In recent years, there has been a cross-fertilization of ideas between computational neuroscience models of the operation of the neocortex and artificial intelligence models of machine learning. Much of this work has focussed on the mammalian visual cortex, treating it as a hierarchically-structured pattern recognition machine that exploits statistical regularities in retinal input.

In this paper a pattern classification and object recognition approach based on bio-inspired techniques is presented. It exploits the Hierarchical Temporal Memory (HTM) topology, which imitates human neocortex for recognition and categorization tasks. The HTM comprises a hierarchical tree structure that exploits enhanced spatiotemporal modules to memorize objects appearing in various orientations. In accordance with HTM's biological inspiration, human vision mechanisms can be used to preprocess the input images. Therefore, the input images undergo a saliency computation step, revealing the plausible information of the scene, where a human might fixate. The adoption of the saliency detection module releases the HTM network from memorizing redundant information and augments the classification accuracy. The efficiency of the proposed framework has been experimentally evaluated in the ETH-80 dataset, and the classification accuracy has been found to be greater than other HTM systems.

In this paper an optimized classification method for object recognition is presented. The proposed method is based on the Hierarchical Temporal Memory (HTM), which stems from the memory prediction theory of the human brain. As in HTM, this method comprises a tree structure of connected computational nodes, whilst utilizing different rules to memorize objects appearing in various orientations. These rules involve both the spatial and the temporal module. As HTM is inspired from brain activity, its input should also comply with the human vision system. Thus, for the representation of the input images the logpolar was given preference to the Cartesian one. As compared to the original HTM method, experimental results exhibit performance enhancements with this approach, in recognition and categorization applications. Results obtained prove that the proposed method is more accurate and faster in training, whilst retaining the network robustness in multiple orientation variations.