Computational evolution of decision-making strategies

The generation of complex entities with advantageous behaviours beyond our manual design capability requires long-term incremental evolution with continuing emergence. In this paper, we argue that artificial selection models, such as traditional genetic algorithms, are fundamentally inadequate for this goal. Existing natural selection systems are evaluated, revealing both significant achievements and pitfalls. Thus, some requirements for the perpetuation of evolutionary emergence are established. An (artificial) environment containing simple virtual autonomous organisms with neural controllers has been created to satisfy these requirements and to aid in the development of an accompanying theory of evolutionary emergence. Resulting behaviours are reported alongside their neural correlates. In a particular example, the collective behaviour of one species provides a selective force which is overcome by another species, demonstrating the incremental evolutionary emergence of advantageous behaviours via naturally-arising coevolution. While the results fall short of the ultimate goal, experience with the system has provided some useful lessons for the perpetuation of emergence towards increasingly complex advantageous behaviours.

Chaos: An Interdisciplinary Journal of Nonlinear Science, 2015

We integrate dual-process theories of human cognition with evolutionary game theory to study the evolution of automatic and controlled decision-making processes. We introduce a model where agents who make decisions using either automatic or controlled processing compete with each other for survival. Agents using automatic processing act quickly and so are more likely to acquire resources, but agents using controlled processing are better planners and so make more effective use of the resources they have. Using the replicator equation, we characterize the conditions under which automatic or controlled agents dominate, when coexistence is possible, and when bistability occurs. We then extend the replicator equation to consider feedback between the state of the population and the environment. Under conditions where having a greater proportion of controlled agents either enriches the environment or enhances the competitive advantage of automatic agents, we find that limit cycles can occur, leading to persistent oscillations in the population dynamics. Critically, however, these limit cycles only emerge when feedback occurs on a sufficiently long time scale. Our results shed light on the connection between evolution and human cognition, and demonstrate necessary conditions for the rise and fall of rationality.

Intelligent Decision Technologies, 2009

This work concerns the problem of modelling evolving prospective agent systems. Inasmuch a prospective agent [1] looks ahead a number of steps into the future, it is confronted with the problem of having several different possible courses of evolution, and therefore needs to be able to prefer amongst them to decide the best to follow as seen from its present state. First it needs a priori preferences for the generation of likely courses of evolution. Subsequently, this being one main contribution of this paper, based on the historical information as well as on a mixture of quantitative and qualitative a posteriori evaluation of its possible evolutions, we equip our agent with so-called evolution-level preferences mechanism, involving three distinct types of commitment. In addition, one other main contribution, to enable such a prospective agent to evolve, we provide a way for modelling its evolving knowledge base, including environment and course of evolution triggering of all active goals (desires), context-sensitive preferences and integrity constraints. We exhibit several examples to illustrate the proposed concepts.

Evolutionary Computation, 2000

Holland's Adaptation in Natural and Artificial Systems largely dealt with how systems, comprised of many self-interested entities, can and should adapt as a whole. This seminal book led to the last 25 years of work in genetic algorithms (GAs), and related forms of evolutionary computation (EC). In recent years, the expansions of the Internet, other telecommunications technologies, and other large scale networks, have led to a world where large numbers of semi-autonomous software entities (i.e., agents) will be interacting in an open, universal system. This development cast the importance of Holland's legacy in a new light. This paper argues that Holland's fundamental arguments, and the years of developments that have followed, have a direct impact on systems of general network agents, regardless of whether they explicitly exploit EC. However, it also argues that the techniques and theories of EC cannot be directly transferred to the world of general (rather than EC-specific) agents, without examination of effects that are embodied in general software agents. This paper introduces a framework for EC interchanges between general-purpose software agents. Preliminary results are shown that illustrate the EC effects of asynchronous actions of agents within this framework. Building on this framework, coevolutionary agents that interact in a simulated producer/consumer economy are introduced. Using these preliminary results as illustrations, areas for future investigation of embodied EC software agents are discussed.

2004

: A developmental perspective is useful to understand how intelligent human behavior comes to be performed because it combines insight of evolutionary factors that enable dynamic genetic-environmental interactions within individual humans. Such developmental adaptations may now be studied experimentally using developmental and epigenetic robots. Resulting insights is a useful step toward more complete, valid understanding of intelligent behavior, its adaptive nature and its structural roots. Taken together these broaden the concept of "engineering mind" to include the larger concept of "development". This paper overviews recent work of evolutionary and developmental psychology, epigenetic robots and cognitive science. A synthesis of these suggest means by which the fluid nature of adaptive knowledge arises developmentally within a heterogeneous architecture adapted for adaptation itself as part of a rational-empirical process. At this top-level of intelligence, s...

2003

Organisms in nature are both proximate (operating by rules of thumb) and adapted (the rules influenced by natural selection). We introduce new methods that can be used to study in silico versions of organisms behaving according to proximate adapted rules. Our approach goes beyond neural networks and offers an alternative to optimization methods. It is based on the idea that organisms receive signals from the environment, that the signals are modified by internal (state-dependent) factors to create feelings (which we refer to as hedonic tones), and that behavioural processes (decisions) are a response to the hedonic tones. We illustrate these ideas through a model of a fish moving in a vertically structured environment, subject to predation and competition from conspecifics. The fish in our model responds to food, light, temperature and conspecifics, without any reference to current or future fitness. We use a combination of hedonic modelling to process the response, and genetic algorithms to modify the response via natural selection, according to internal needs and evolutionary history. We show that many different combinations of genes can lead to similar fitnesses, so that this approach generates genetic diversity. We compare our results with those of a variety of empirical studies and show that our approach can lead to new links between empirical and simulation studies.

Proceedings of the 11th Annual conference on Genetic and evolutionary computation - GECCO '09, 2009

Integrating different kinds of micro-theories of cognition in intelligent systems when a huge amount of variables are changing continuously, with increasing complexity, is a very exhaustive and complicated task. Our approach proposes a hybrid cognitive architecture that relies on the integration of emergent and cognitivist approaches using evolutionary strategies, in order to combine implicit and explicit knowledge representations necessary to develop cognitive skills. The proposed architecture includes a cognitive level controlled by autopoietic machines and artificial immune systems based on genetic algorithms, giving it a significant degree of plasticity. Furthermore, we propose an attention module which includes an evolutionary programming mechanism in charge of orchestrating the hierarchical relations among specialized behaviors, taking into consideration the global workspace theory for consciousness. Additionally, a coevolutionary mechanism is proposed to propagate knowledge among cognitive agents on the basis of memetic engineering. As a result, several properties of self-organization and adaptability emerged when the proposed architecture was tested in an animat environment, using a multi-agent platform.

Engineering Letters, 2007

In this work, an hybrid, self-configurable, multilayered and evolutionary subsumption architecture for cognitive agents is developed. Each layer of the multilayered architecture is modeled by one different Machine Learning

Frontiers in Psychology

In this paper, we show that a neurally implemented a cognitive architecture with evolutionary dynamics can solve the four-tree problem. Our model, called Darwinian Neurodynamics, assumes that the unconscious mechanism of problem solving during insight tasks is a Darwinian process. It is based on the evolution of patterns that represent candidate solutions to a problem, and are stored and reproduced by a population of attractor networks. In our first experiment, we used human data as a benchmark and showed that the model behaves comparably to humans: it shows an improvement in performance if it is pretrained and primed appropriately, just like human participants in Kershaw et al. (2013)'s experiment. In the second experiment, we further investigated the effects of pretraining and priming in a two-by-two design and found a beginner's luck type of effect: solution rate was highest in the condition that was primed, but not pretrained with patterns relevant for the task. In the third experiment, we showed that deficits in computational capacity and learning abilities decreased the performance of the model, as expected. We conclude that Darwinian Neurodynamics is a promising model of human problem solving that deserves further investigation.

Studies in Computational Intelligence, 2010

We study a model of evolving populations of self-learning agents and analyze the interaction between learning and evolution. We consider agent-brokers that predict stock price changes and use these predictions for selecting actions. Each agent is equipped with a neural network adaptive critic design for behavioral adaptation. We discuss three cases in which either learning, or evolution, or both, are active in our model. We show that the Baldwin effect can be observed in our model, viz., originally acquired adaptive policy of best agent-brokers becomes inherited over the course of the evolution. Additionally, we analyze influence of neural network structure of adaptive critic design on learning processes.