Autonomous Mental Development

We propose an architecture for self-motivated agents allowing them to construct their own knowledge of objects and of geometrical properties of space through interaction with their environment. Self-motivation is defined here as a tendency to experiment and to respond to behavioral opportunities afforded by the environment. Interactions have predefined valences that specify inborn behavioral preferences. The long-term goal is to design agents that construct their own knowledge of their environment through experience, rather than exploiting pre-coded knowledge. Over time, the agent learns relations between elements of the environment that afford its interactions, and its perception of these elements, in the form of data structures called signatures of interactions. These signatures allow the agent to attribute a low level semantics to elements that constitute its environment based on valences of interactions, without predefined knowledge about these elements and regardless of the number of element types. Signatures of interaction are then used to localize elements in space and to construct data structures that characterize spatial properties of space, called signatures of places and signatures of presence. Signatures of place and of presence characterize space using interactions rather than geometrical or topological properties. The agent uses these structures to maintain an egocentric representation of affordances of the surrounding environment, without any preconception about the elements that compose the environment, and without using notions of geometrical space. Experiments with simulated agents show that they learn to behave in their environment, taking into account multiple surrounding objects, reaching or avoiding objects according to the valence of the interactions that they afford.

TWIG ("Transportable Word Intension Generator") is a system that allows a robot to learn compositional meanings for new words that are grounded in its sensory capabilities. The system is novel in its use of logical semantics to infer which entities in the environment are the referents (extensions) of unfamiliar words; its ability to learn the meanings of deictic ("I," "this") pronouns in a real sensory environment; its use of decision trees to implicitly contrast new word definitions with existing ones, thereby creating more complex definitions than if each word were treated as a separate learning problem; and its ability to use words learned in an unsupervised manner in complete grammatical sentences for production, comprehension, or referent inference. In an experiment with a physically embodied robot, TWIG learns grounded meanings for the words "I" and "you," learns that "this" and "that" refer to objects of varying proximity, that "he" is someone talked about in the third person, and that "above" and "below" refer to height differences between objects. Follow-up experiments demonstrate the system's ability to learn different conjugations of "to be"; show that removing either the extension inference or implicit contrast components of the system results in worse definitions; and demonstrate how decision trees can be used to model shifts in meaning based on context in the case of color words.

Building a smart home is a multidisciplinary and challenging problem. Our goal is to build an agent that can propose context aware services to the users. High variability of users' needs and the uniqueness of every home are difficult to handle using "Classical AI". We propose an alternative approach inspired by Developmental Artificial Intelligence and Constructivism Theory. Being constructivist means that the agent builds its knowledge in situ through user's interactions. This continuous interaction process enables the user to customize or bring up the system to meet his personal needs. We have made a first experiment by learning schemas from a simulated two-weeks home scenario. This preliminary experiment gives us indications that Constructivism is a promising approach for ambient intelligence.

TWIG ("Transportable Word Intension Generator") is a system that allows a robot to learn compositional meanings for new words that are grounded in its sensory capabilities. The system is novel in its use of logical semantics to infer which entities in the environment are the referents (extensions) of unfamiliar words; its ability to learn the meanings of deictic ("I," "this") pronouns in a real sensory environment; its use of decision trees to implicitly contrast new word definitions with existing ones, thereby creating more complex definitions than if each word were treated as a separate learning problem; and its ability to use words learned in an unsupervised manner in complete grammatical sentences for production, comprehension, or referent inference. In an experiment with a physically embodied robot, TWIG learns grounded meanings for the words "I" and "you," learns that "this" and "that" refer to objects of varying proximity, that "he" is someone talked about in the third person, and that "above" and "below" refer to height differences between objects. Follow-up experiments demonstrate the system's ability to learn different conjugations of "to be"; show that removing either the extension inference or implicit contrast components of the system results in worse definitions; and demonstrate how decision trees can be used to model shifts in meaning based on context in the case of color words.

This paper presents a computational theory of developmental mental architectures for artificial and natural systems, motivated by neuroscience. The work is an attempt to approximately model biological mental architectures using mathematical tools. Six types of architecture are presented, beginning with the observation-driven Markov decision process as Type-1. From Type-1 to Type-6, the architecture progressively becomes more complete toward the necessary functions of autonomous mental development. Properties of each type are presented. Experiments are discussed with emphasis on their architectures.

This paper presents a computational theory of developmental mental architectures for artificial and natural systems, motivated by neuroscience. The work is an attempt to approximately model biological mental architectures using mathematical tools. Six types of architecture are presented, beginning with the observation-driven Markov decision process as Type-1. From Type-1 to Type-6, the architecture progressively becomes more complete toward the necessary

The paper proposes a novel research framework for building probabilistic computational neurogenetic models (pCNGM). The pCNGM is a multilevel modeling framework inspired by the multilevel information processes in the brain. The framework comprises a set of several dynamic models, namely low (molecular) level models, a more abstract dynamic model of a protein regulatory network (PRN) and a probabilistic spiking neural network model (pSNN), all linked together. Genes/proteins from the PRN control parameters of the pSNN and the spiking activity of the pSNN provides feedback to the PRN model. The overall spatio-temporal pattern of spiking activity of the pSNN is interpreted as the highest level state of the pCNGM. The paper demonstrates that this framework can be used for modeling both artificial cognitive systems and brain processes. In the former application, the pCNGM utilises parameters that correspond to sensory elements and neuromodulators. In the latter application a pCNGM uses data obtained from relevant genes/proteins to model their dynamic interaction that matches data related to brain development, higher-level brain function or disorder in different scenarios. An exemplar case study on Alzheimer's Disease is presented. Future applications of pCNGM are discussed.

We present a framework for intrinsically motivated developmental learning of abstract skill hierarchies by reinforcement learning agents in structured environments. Long-term learning of skill hierarchies can drastically improve an agent's efficiency in solving ensembles of related tasks in a complex domain. In structured domains composed of many features, understanding the causal relationships between actions and their effects on different features of the environment can greatly facilitate skill learning. Using Bayesian network structure-learning techniques and structured dynamic programming algorithms, we show that reinforcement learning agents can learn incrementally and autonomously both the causal structure of their environment and a hierarchy of skills that exploit this structure. Furthermore, we present a novel active learning scheme that employs intrinsic motivation to maximize the efficiency with which this structure is learned. As new structure is acquired using an agent's current set of skills, more complex skills are learned, which in turn allow the agent to discover more structure, and so on. This bootstrapping property makes our approach a developmental learning process that results in steadily increasing domain knowledge and behavioral complexity as an agent continues to explore its environment.