On a learning model with growing memory

Mark  Trakhtenbrot

doi:10.1007/BF01068453

Outline

On a learning model with growing memory

Mark Trakhtenbrot

1974, Cybernetics

https://doi.org/10.1007/BF01068453

visibility

…

description

10 pages

link

1 file

Sign up for access to the world's latest research

checkGet notified about relevant papers

checkSave papers to use in your research

checkJoin the discussion with peers

checkTrack your impact

Abstract

In this article we present an example of a growing structure that consists at each instant of time of a specified automaton, i.e., a head and finitely many finite automata (elements) of the same type. This structure interacts with a pair of Bernoullian random generators that yield ones and zeros with probabilities unknown to the structure (which interrogates one of the generators during each interval). The principal result is as follows. For any initial state of the structure and any generators, the limiting average frequency of interrogation of the generator that yields a "one" with a higher probability is equal to 1. In this sense the structure realizes an optimal training algorithm. The algorithm consists of alternating (socalled) learning and operating cycles, in Sec. i we shall formulate a sufficient condition of optimality of such algorithms. In Sec. 2 we shall describe the automata forming the structure, and in Sec. 3 we shall show that the algorithm realized by it satisfies this sufficient condition. Each element of the structure has three states, and the minimum possible number of states is not smaller than two in any case. In Sec. 4 we present bounds related to a probability-theoretical scheme whose simulation underlies the operation of the structure; these bounds are perhaps of an intrinsic interest.

maher maherben said

2010

This paper explores the role of memory in decision making in dynamic environments. We examine the inference problem faced by an agent with bounded memory who receives a sequence of signals from a hidden Markov model. We show that the optimal symmetric memory rule may be deterministic. This result contrasts sharply with Hellman and Cover (1970) and Wilson (2004) and

downloadDownload free PDF View PDFchevron_right

Efficient Learning of Typical Finite Automata from Random Walks (Extended Abstract)

Ronitt Rubinfeld

1996

This paper describes new and e cient algorithms for learning deterministic nite automata. Our approach is primarily distinguished by two features: (1) the adoption of an average-case setting to model the \typical" labeling of a nite automaton, while retaining a worst-case model for the underlying graph of the automaton, along with (2) a learning model in which the learner is not provided with the means to experiment with the machine, but rather must learn solely by observing the automaton's output behavior on a random input sequence. The main contribution of this paper is in presenting the rst e cient algorithms for learning non-trivial classes of automata in an entirely passive learning model.

downloadDownload free PDF View PDFchevron_right

Colonies of learning automata

Ann Nowé

IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), 2002

Originally, learning automata (LAs) were introduced to describe human behavior from both a biological and psychological point of view. In this paper, we show that a set of interconnected LAs is also able to describe the behavior of an ant colony, capable of finding the shortest path from their nest to food sources and back.

downloadDownload free PDF View PDFchevron_right

Recursion theoretic models of learning: Some results and intuitions

William Gasarch

Annals of Mathematics and Artificial Intelligence, 1995

To implement a program that somehow "learns", it is necessary to fix a set of concepts to be learned and to develop a representation for the concepts and examples of the concepts. In order to investigate general properties of machine learning, it is necessary to work in as representation-independent fashion as possible. In this work, we consider machines that learn programs for recursive (total effectively computable) functions. Several authors have argued that such studies are general enough to include a wide array of learning situations .

downloadDownload free PDF View PDFchevron_right

A New Class of Learning Automata: The Probabilistically-Switch-Action-on-Failure Automaton Part 2 – Simulations

Lev Levitin

2022

In this part of the two-part paper, simulations of Probabilistically-Switch-Action-on-Failure learning automaton (PSAFA) are presented in various stationary and non-stationary environments. The PSAFA is a novel fixed structure stochastic automaton (FSSA) framework, and its analytical model is presented in detail in Part 1 of this paper set. The key differentiating feature of this automaton is that it allows action switching in every state. We anticipate that this feature attributes PSAFA dynamic properties that make certain aspects of its performance superior to other FSSA that do not possess this property. In this paper, simulations of the PSAFA in comparison with other FSSA are considered in two types of environments: a stationary environment (with fixed penalty probabilities) and a non-stationary environment, where the penalty probabilities are changing in time periodically as a sinusoidal function. In both cases the simulation demonstrates a dramatic difference in performance for these types of learning automata. The PSAFA shows its huge advantage in adaptability that leads to a better performance for the length of the simulation up to 30,000-150,000 steps. Only for very long stationary conditions Tsetlin automata outperforms PSAFA. In the case of sinusoidal modulations, the PSAFA tremendously outperforms other types of FSSA for all modulation frequencies and for all depths D>3. The performance of PSAFA does not deteriorate with increasing modulation frequency, while other FSSA are not resilient to that increase.

downloadDownload free PDF View PDFchevron_right

A Mathematical Framework for Cellular Learning Automata

Hamid Beigy

Advances in Complex Systems, 2004

The cellular learning automata, which is a combination of cellular automata, and learning automata, is a new recently introduced model. This model is superior to cellular automata because of its ability to learn and is also superior to a single learning automaton because it is a collection of learning automata which can interact with each other. The basic idea of cellular learning automata, which is a subclass of stochastic cellular learning automata, is to use the learning automata to adjust the state transition probability of stochastic cellular automata. In this paper, we first provide a mathematical framework for cellular learning automata and then study its convergence behavior. It is shown that for a class of rules, called commutative rules, the cellular learning automata converges to a stable and compatible configuration. The numerical results also confirm the theoretical investigations.

downloadDownload free PDF View PDFchevron_right

Efficient Learning of Typical Finite Automata from Random Walks

Ronitt Rubinfeld

Information and Computation, 1997

downloadDownload free PDF View PDFchevron_right

Learning with bounded memory

Jaideep Roy

Nº.: UC3M Working papers. Economics 2000-72-26, 2000

The paper studies infinite repetition of finite strategic form games. Players use a learning behavior and face bounds on their cognitive capacities. We show that for any given beliefprobability over the set of possible outcomes where players have no experience. games can be payoff classified and there always exists a stationary state in the space of action profiles. In particular, if the belief-probability assumes all possible outcomes without experience to be equally likely, in one class of Prisoners' Dilemmas where the average defecting payoff is higher than the cooperative payoff and the average cooperative payoff is lower than the defecting payoff. play converges in the long run to the static Nash equilibrium while in the other class of Prisoners' Dilemmas where the reserve holds, play converges to cooperation. Results are applied to a largc class of 2 x 2 games.

downloadDownload free PDF View PDFchevron_right

Incremental learning with temporary memory

was wie76

Theoretical Computer Science, 2010

In the inductive inference framework of learning in the limit, a variation of the bounded example memory (B em) language learning model is considered. Intuitively, the new model constrains the learner's memory not only in how much data may be retained, but also in how long that data may be retained. More specifically, the model requires that, if a learner commits an example x to memory in some stage of the learning process, then there is some subsequent stage for which x no longer appears in the learner's memory. This model is called temporary example memory (T em) learning. In some sense, it captures the idea that memories fade.

downloadDownload free PDF View PDFchevron_right

Recent Advances in Learning Automata

Alireza Rezvanian

Studies in Computational Intelligence, 2018

The series "Studies in Computational Intelligence" (SCI) publishes new developments and advances in the various areas of computational intelligence-quickly and with a high quality. The intent is to cover the theory, applications, and design methods of computational intelligence, as embedded in the fields of engineering, computer science, physics and life sciences, as well as the methodologies behind them. The series contains monographs, lecture notes and edited volumes in computational intelligence spanning the areas of neural networks, connectionist systems, genetic algorithms, evolutionary computation, artificial intelligence, cellular automata, self-organizing systems, soft computing, fuzzy systems, and hybrid intelligent systems. Of particular value to both the contributors and the readership are the short publication timeframe and the worldwide distribution, which enable both wide and rapid dissemination of research output.

downloadDownload free PDF View PDFchevron_right

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (4)

M~ L~ Tsetlin, Studies in Automata Theory and Simulation of Biological Systems [in Russian], Nauka, Moscow (1969).
M. E. Hellman and Th~ M~ Cover, "Comments on automata in random media," Problemy Peredachi Informatsii, _6, No. 2 (1970)~
Ho Robbins, "Some aspects of the sequential design of experiments," Bull. Amer. Math. Soc., 58, 529-532 (1952)~
W~ Feller, An Introduction to Probability Theory and Its Applications, Vol~ 1, Wiley, New York {1950)~

Lev Levitin

2022

We present the novel concept of Probabilistically-Switch-Action-on-Failure learning automaton (PSAFA). The PSAFA is a fixed structure stochastic automaton (FSSA), characterized by a fan-shaped state transition diagram where each branch of the state space is a chain of D states, and is associated with a particular action. The first states of all chains form a circle of initial states. The PSAFA can switch from a present state in any chain to the initial state of the next chain in the circle, on each failure, with some finite probability. This probability, which plays the role of an action-switching probability, is a function of the distance of the present state from initial state of its branch. The learning behavior of PSAFA is determined by the dependence of the action switching probability on the distance from the initial state. The probabilistic action-switching capability distinguishes PSAFA from conventional FSSA that have deterministic action selection at each state, and only some states transit to states with a different action. This action-switching capability at any time is also typical for conventional variable structure stochastic automata (VSSA) but it comes with added computational complexity. VSSA are more adaptive than FSSA in non-stationary environments because of this action-switching capability. We believe that the addition of this capability should also make the PSAFA more adaptive in non-stationary environments than classical FSSA while preserving the simplified computational complexity of FSSA. The effectiveness of the proposed framework is demonstrated through the theoretical analysis of optimality of the PSAF learning automaton in stationary environments in part 1 of this 2-part paper.

downloadDownload free PDF View PDFchevron_right

A learning strategy for a class of probabilistic automata

Lyle Reibling

1983

downloadDownload free PDF View PDFchevron_right

Asymptotic behavior of a hierarchical system of learning automata

Kandethody Ramachandran

Information Sciences, 1985

Learning automata arranged in a twolevel hierarchy are considered. The automata operate in a stationary random environment and update their action probabilities according to the linear-reward-a-penalty algorithm at each level. Unlike some hierarchical systems previously proposed, no information transfer exists from one level to another, and yet the hierarchy possesses good convergence properties. Using weak-convergence concepts it is shown that for large time and small values of parameters in the algorithm, the evolution of the optimal path probability can be represented by a diffusion whose parameters can be computed explicitly.

downloadDownload free PDF View PDFchevron_right

Inference and Learning in Stochastic Automata

rehman ahmad

International Journal of Pure and Apllied Mathematics, 2017

Machine learning provides algorithms that can learn from data and make inferences or predictions on data. Stochastic automata are a class of input/output devices which can model components in machine learning scenarios. In this paper, we provide an inference algorithm for stochastic automata which is related to the Viterbi algorithm. Moreover, we specify a learning algorithm using the expectation-maximization technique and describe a more efficient implementation which is related to the Baum-Welch algorithm.

downloadDownload free PDF View PDFchevron_right

Varieties of learning automata: an overview

M. Thathachar

IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), 2002

Automata models of learning systems introduced in the 1960s were popularized as learning automata (LA) in a survey paper in 1974 [1]. Since then, there have been many fundamental advances in the theory as well as applications of these learning models. In the past few years, the structure of LA has been modified in several directions to suit different applications. Concepts such as parameterized learning automata (PLA), generalized learning automata (GLA), and continuous action-set learning automata (CALA) have been proposed, analyzed, and applied to solve many significant learning problems. Furthermore, groups of LA forming teams and feedforward networks have been shown to converge to desired solutions under appropriate learning algorithms. Modules of LA have been used for parallel operation with consequent increase in speed of convergence. All of these concepts and results are relatively new and are scattered in technical literature. An attempt has been made in this paper to bring together the main ideas involved in a unified framework and provide pointers to relevant references.

downloadDownload free PDF View PDFchevron_right

Results on memory-limited U-shaped learning

Lorenzo Carlucci

Information and Computation/information and Control, 2007

U-shaped learning is a learning behaviour in which the learner first learns something, then unlearns it and finally relearns it. Such a behaviour, observed by psychologists, for example, in the learning of past-tenses of English verbs, has been widely discussed among psychologists and cognitive scientists as a fundamental example of the non-monotonicity of learning. Previous theory literature has studied whether or not Ushaped learning, in the context of Gold's formal model of learning languages from positive data, is necessary for learning some tasks. It is clear that human learning involves memory limitations. In the present paper we consider, then, this question of the necessity of U-shaped learning for some learning models featuring memory limitations. Our results show that the question of the necessity of Ushaped learning in this memory-limited setting depends on delicate tradeoffs between the learner's ability to remember its own previous conjecture, to store some values in its longterm memory, to make queries about whether or not items occur in previously seen data and on the learner's choice of hypothesis space.

downloadDownload free PDF View PDFchevron_right

Group actions and learning for a family of automata

Pierre Baldi

Journal of Computer and System Sciences, 1988

We study properties of a family of neural network type of automata which are generalizations of the Hoptield model for content addressable memory using a novel approach based on the action of groups of isometries of the hypercube. We review storage algorithms and show that the classical sum of outerproducts scheme and several of its variations are invariant under such action. As a consequence, we prove that the space of states is partitioned into orbits of points having the same dynamical behavior. We find minimal sets of conditions forcing this property and discuss applications for the structure of the stable points. Calculations for a simple orthogonal case are given as an example. 0 1988 Academic Press. Inc.

downloadDownload free PDF View PDFchevron_right

A theoretical comparison of batch-mode, on-line, cyclic, and almost-cyclic learning

Tom Heskes

IEEE Transactions on Neural Networks, 2000

We study and compare di erent neural network learning strategies: batch-mode learning, on-line learning, cyclic learning, and almost cyclic learning. Incremental learning strategies require less storage capacity than batch-mode learning. However, due to the arbitrariness in the presentation order of the training patterns, incremental learning is a stochastic process, whereas batch-mode learning is deterministic. In zeroth order, i.e., as the learning parameter tends to zero, all learning strategies approximate the same ordinary di erential equation, for convenience referred to as the \ideal behavior". Using stochastic methods valid for small learning parameters , we derive di erential equation describing the evolution of the lowest order deviations from this ideal behavior. We compute how the asymptotic misadjustment, measuring the average asymptotic distance from a stable xed point of the ideal behavior, scales as a function of the learning parameter and the number of training patterns. Knowing the asymptotic misadjustment, we calculate the typical number of learning steps necessary to generate a weight within order of this xed point, both with xed and time-dependent learning parameters. We conclude that almost cyclic learning (learning with random cycles) is a better alternative for batch-mode learning than cyclic learning (learning with a xed cycle).

downloadDownload free PDF View PDFchevron_right

Learning the global maximum with parameterized learning automata

M. Thathachar

IEEE Transactions on Neural Networks, 1995

A feedforward network composed of units of teams of parameterized learning automata is considered as a model of a reinforcement learning system. The internal state vector of each learning automaton is updated using an algorithm consisting of a gradient following term and a random perturbation term. It is shown that the algorithm weakly converges to a solution of the Langevin equation implying that the algorithm globally maximizes an appropriate function. The algorithm is decentralized, and the units do not have any information exchange during updating. Simulation results on common payoff games and pattern recognition problems show that reasonable rates of convergence can be obtained.

downloadDownload free PDF View PDFchevron_right

Adaptively learning probabilistic deterministic automata from data streams

Jorge Castro

Machine Learning, 2013

Markovian models with hidden state are widely-used formalisms for modeling sequential phenomena. Learnability of these models has been well studied when the sample is given in batch mode, and algorithms with PAC-like learning guarantees exist for specific classes of models such as Probabilistic Deterministic Finite Automata (PDFA). Here we focus on PDFA and give an algorithm for inferring models in this class in the restrictive data stream scenario: Unlike existing methods, our algorithm works incrementally and in one pass, uses memory sublinear in the stream length, and processes input items in amortized constant time. We also present extensions of the algorithm that (1) reduce to a minimum the need for guessing parameters of the target distribution and (2) are able to adapt to changes in the input distribution, relearning new models when needed. We provide rigorous PAC-like bounds for all of the above. Our algorithm makes a key usage of stream sketching techniques for reducing memory and processing time, and is modular in that it can use different tests for state equivalence and for change detection in the stream.

downloadDownload free PDF View PDFchevron_right

On a learning model with growing memory

Sign up for access to the world's latest research

Abstract

Related papers

References (4)

Related papers

Related topics