Visual learning by imitation with motor representations

Adaptive Behavior, 2004

This paper examines the interest of a developmental approach applied to the design of autonomous robots and the understanding of adaptive behaviors, such as imitation. The proposed model is a neural network architecture that learns and uses associations between vision and arm movements, even if the problem is ill posed (in the case of mapping problems between the visual space and the joints space of the arm). The central part of the model is a visuo-motor map able to represent the arm end point's position in an ego-centered space (constrained by the vision) according to motor information (the proprioception). Sensorimotor behaviors such as tracking, pointing, spontaneous imitating, and sequences learning can then be obtained as the consequence of different internal dynamics computed on neural fields triggered by the visuo-motor map. The readout mechanism also explains how an apparently complex behavior can be generated and controlled from one simple internal dynamics and how at the same time the learning problems can be simplified. While highlighting the generic aspect of our model, we show that our robot can autonomously imitate and learn more complex sequences of gestures after the online learning of the visual and proprioceptive control of its hand extremity. Finally, we defend the idea of a co-development of imitative and sensorimotor capabilities, allowing the acquisition and the building of increasingly complex behavioral capabilities.

2014

This work describes the necessary aspects for the programming of a robot through the paradigm of learning by demonstration and it proposes a system that keeps concordance with what is known about the functioning of biological systems. Four gestures performed by the arm of different human demonstrators were used in the imitation performed by a simulated arm. To achieve these gesture imitations, a Visual-Motor map was created which allows the robot to give its own interpretation of the observed gesture. The imitation performance was evaluated for its quality and quantity. New indicators are presented for the four gestures that make possible the imitation evaluation beyond the following of the trajectory usually performed in order to define the success of the imitation. The imitation was successful in 3 of the 4 gestures and scored an average of 62% for the poll and 78.6% for the indicators.

2005

We report results of an interdisciplinary project which aims at endowing a real robot system with the capacity for learning by goal-directed imitation. The control architecture is biologically inspired as it reflects recent experimental findings in action observation/execution studies. We test its functionality in variations of an imitation paradigm in which the artefact has to reproduce the observed or inferred end state of a grasping-placing sequence displayed by a human model.

Object recognition is a key problem of artificial vision; in robotics, it is strongly connected to that of grasping. In fact, there is so far no general solution to either problem. Traditionally, visual features are evaluated from camera images and statistical methods are then trained on huge visual datasets in order to obtain a robust object classifier. The knowledge so obtained is then used to choose a model to perform a grasping action. Inspired, among others, by the neuroscientific framework of mirror neurons, we hereby propose to enhance the model of an object by adding to its visual features a probabilistic description of the grasps chosen by human subjects to grasp it. Since in a standard setting the grasps are not directly available to the system, they must be reconstructed from the visual features, and then used to augment the recognition system's input space. We achieve this by building a map from visual to motor features, which we call a Visuo-Motor Map (VMM), practically enforced via regression on a human grasping database. We experimentally show that such a technique improves the recognition rate of a standard object classifier: in case the original motor features are used, the improvement is dramatic, whereas when we reconstruct them via the VMM we still obtain a statistically significant improvement.

Brain Research Bulletin, 2008

Being at the crux of human cognition and behavior, imitation has become the target of investigations ranging from experimental psychology and neurophysiology to computational sciences and robotics. It is often assumed that the imitation is innate, but it has more recently been argued, both theoretically and experimentally, that basic forms of imitation could emerge as a result of self-observation. Here, we tested this proposal on a realistic experimental platform, comprising an associative network linking a 16 degrees of freedom robotic hand and a simple visual system. We report that this minimal visuomotor association is sufficient to bootstrap basic imitation. Our results indicate that crucial features of human imitation, such as generalization to new actions, may emerge from a connectionist associative network. Therefore, we suggest that a behavior as complex as imitation could be, at the neuronal level, founded on basic mechanisms of associative learning, a notion supported by a recent proposal on the developmental origin of mirror neurons. Our approach can be applied to the development of realistic cognitive architectures for humanoid robots as well as to shed new light on the cognitive processes at play in early human cognitive development.

Brain and Cognition, 2008

Previous research has indicated a potential discontinuity between monkey and human ventral premotor-parietal mirror systems, namely that monkey mirror systems process only transitive (object-directed) actions, whereas human mirror systems may also process intransitive (non-object-directed) actions. The present study investigated this discontinuity by seeking evidence of automatic imitation of intransitive actions-hand opening and closing-in humans using a simple reaction time (RT), stimulus-response compatibility paradigm. Left-right and up-down spatial compatibility were controlled by ensuring that stimuli were presented and responses executed in orthogonal planes, and automatic imitation was isolated from simple and complex orthogonal spatial compatibility by varying the anatomical identity of the stimulus hand and response hemispace, respectively. In all conditions, action compatible responding was faster than action incompatible responding, and no effects of spatial compatibility were observed. This experiment therefore provides evidence of automatic imitation of intransitive actions, and support for the hypothesis that human and monkey mirror systems differ with respect to the processing of intransitive actions.

Lecture Notes in Computer Science, 2002

In biological systems vision is always in the context of a particular body and tightly coupled to action. Therefore it is natural to consider visuo-motor methods (rather than vision alone) for learning about objects in the world. Indeed, initially it may be necessary to act on something to learn that it is an object! Learning to act involves not only learning the visual consequences of performing a motor action, but also the other direction, i.e. using the learned association to determine which motor action will bring about a desired visual condition. In this paper we show how a humanoid robot uses its arm to try some simple pushing actions on an object, while using vision and proprioception to learn the effects of its actions. We show how the robot learns a mapping between the initial position of its arm and the direction the object moves in when pushed, and then how this learned mapping is used to successfully position the arm to push/pull the target object in a desired direction.

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018

2004

We propose an approach that allows a robot to learn a task through imitation, using motor representations, as suggested by recent findings in neuroscience. The robot relies on a visuomotor map to convert visual information into motor data. Then, by observing and imitating other agents, the robot can learn a set of elementary motions (motor vocabulary), that will eventually be used to compose more complex actions, for each specific task domain. We illustrate the approach in a mobile robotics task. Egomotion estimation is used as a visuomotor map, that allows the robot to learn a motor vocabulary for topological mapping and navigation.The approach can be extended to different robots and applications. Encouraging results are presented and discussed.