Papers by Thierry Chaminade
PloS one, Mar 21, 2024
We present an analytical framework aimed at predicting the local brain activity in uncontrolled e... more We present an analytical framework aimed at predicting the local brain activity in uncontrolled experimental conditions based on multimodal recordings of participants' behavior, and its application to a corpus of participants having conversations with another human or a conversational humanoid robot. The framework consists in extracting high-level features from the raw behavioral recordings and applying a dynamic prediction of binarized fMRIrecorded local brain activity using these behavioral features. The objective is to identify behavioral features required for this prediction, and their relative weights, depending on the brain area under investigation and the experimental condition. In order to validate our framework, we use a corpus of uncontrolled conversations of participants with a human or a robotic agent, focusing on brain regions involved in speech processing, and more generally in social interactions. The framework not only predicts local brain activity significantly better than random, it also quantifies the weights of behavioral features required for this prediction, depending on the brain area under investigation and on the nature of the conversational partner. In the left Superior Temporal Sulcus, perceived speech is the most important behavioral feature for predicting brain activity, regardless of the agent, while several features, which differ between the human and robot interlocutors, contribute to the prediction in regions involved in social cognition, such as the TemporoParietal Junction. This framework therefore allows us to study how multiple behavioral signals from different modalities are integrated in individual brain regions during complex social interactions.
HAL (Le Centre pour la Communication Scientifique Directe), Nov 28, 2023
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific r... more HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific r... more HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Human behaviors from tool-making to language are thought to rely on a uniquely evolved capacity f... more Human behaviors from tool-making to language are thought to rely on a uniquely evolved capacity for hierarchical action sequencing. Testing this idea will require objective, generalizable methods for measuring the structural complexity of real-world behavior. Here we present a data-driven approach for extracting action grammars from basic ethograms, exemplified with respect to the evolutionarilyrelevant behavior of stone tool-making. We analyzed sequences from the experimental replication of ~2.5 Mya Oldowan vs. ~0.5 Mya Acheulean tools, finding that, while using the same "alphabet" of elementary actions, Acheulean sequences are quantifiably more complex and Oldowan grammars are a subset of Acheulean grammars. We illustrate the utility of our complexity measures by re-analyzing data from an fMRI study of stone tool-making to identify brain responses to structural complexity. Beyond specific implications regarding the co-evolution of language and technology, this exercise illustrates the general applicability of our method to investigate naturalistic human behavior and cognition.
Assessing the social competence of anthropomorphic artificial agents developed to produce engagin... more Assessing the social competence of anthropomorphic artificial agents developed to produce engaging social interactions with humans has become of primary importance to effectively compare various appearances and/or behaviours. Here we attempt to objectify the social competence of artificial agents, across different dimensions, using human brain neurophysiology. Whole brain activity is recorded with functional Magnetic Resonance Imaging (fMRI) while participants, naïve to the real purpose of the experiment, discuss either with a human confederate or with an artificial agent, presently the robotic conversational head Furhat controlled with a Wizard of Oz procedure. This allows a direct comparison of local brain responses, not only at the cortical level but also in subcortical structures associated with motivational drive and impossible to investigate with non-invasive neurophysiology techniques such as surface recordings. The present data (n=21 participants) demonstrates the feasibility of this approach, and results confirm an increased activity in subcortical structures, in particular the amygdala involved in emotional processing and the hypothalamus, known to secrete, among others, the
Philosophical Transactions of the Royal Society B, Mar 11, 2019
We present a novel functional magnetic resonance imaging paradigm for second-person neuroscience.... more We present a novel functional magnetic resonance imaging paradigm for second-person neuroscience. The paradigm compares a human social interaction (human-human interaction, HHI) to an interaction with a conversational robot (human-robot interaction, HRI). The social interaction consists of 1 min blocks of live bidirectional discussion between the scanned participant and the human or robot agent. A final sample of 21 participants is included in the corpus comprising physiological (blood oxygen leveldependent, respiration and peripheral blood flow) and behavioural (recorded speech from all interlocutors, eye tracking from the scanned participant, face recording of the human and robot agents) data. Here, we present the first analysis of this corpus, contrasting neural activity between HHI and HRI. We hypothesized that independently of differences in behaviour between interactions with the human and robot agent, neural markers of mentalizing (temporoparietal junction (TPJ) and medial prefrontal cortex) and social motivation (hypothalamus and amygdala) would only be active in HHI. Results confirmed significantly increased response associated with HHI in the TPJ, hypothalamus and amygdala, but not in the medial prefrontal cortex. Future analysis of this corpus will include fine-grained characterization of verbal and non-verbal behaviours recorded during the interaction to investigate their neural correlates. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human -robot interaction'.
Supplementary material from "Brain activity during reciprocal social interaction investigated using conversational robots as control condition
We present a novel functional magnetic resonance imaging paradigm for second-person neuroscience.... more We present a novel functional magnetic resonance imaging paradigm for second-person neuroscience. The paradigm compares a human social interaction (human–human interaction, HHI) to an interaction with a conversational robot (human–robot interaction, HRI). The social interaction consists of 1 min blocks of live bidirectional discussion between the scanned participant and the human or robot agent. A final sample of 21 participants is included in the corpus comprising physiological (blood oxygen level-dependent, respiration and peripheral blood flow) and behavioural (recorded speech from all interlocutors, eye tracking from the scanned participant, face recording of the human and robot agents) data. Here, we present the first analysis of this corpus, contrasting neural activity between HHI and HRI. We hypothesized that independently of differences in behaviour between interactions with the human and robot agent, neural markers of mentalizing (temporoparietal junction (TPJ) and medial prefrontal cortex) and social motivation (hypothalamus and amygdala) would only be active in HHI. Results confirmed significantly increased response associated with HHI in the TPJ, hypothalamus and amygdala, but not in the medial prefrontal cortex. Future analysis of this corpus will include fine-grained characterization of verbal and non-verbal behaviours recorded during the interaction to investigate their neural correlates.This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human–robot interaction'.
How do artificial agents think?
Anthropomorphic artificial agents, computed characters or humanoid robots, can be sued to investi... more Anthropomorphic artificial agents, computed characters or humanoid robots, can be sued to investigate human cognition. They are intrinsically ambivalent. They appear and act as humans, hence we should tend to consider them as human, yet we know they are machine designed by humans, and should not consider them as humans. Reviewing a number of behavioral and neurophysiological studies provides insights into social mechanisms that are primarily influenced by the appearance of the agent, and in particular its resemblance to humans, and other mechanisms that are influenced by the knowledge we have about the artificial nature of the agent. A significant finding is that, as expected, humans don’t naturally adopt an intentional stance when interacting with artificial agents.
BIO web of conferences, 2011
Robotic devices, thanks to the controlled variations in their appearance and behaviors, provide u... more Robotic devices, thanks to the controlled variations in their appearance and behaviors, provide useful tools to test hypotheses pertaining to social interactions. These agents were used to investigate one theoretical framework, resonance, which is defined, at the behavioral and neural levels, as an overlap between first-and third-person representations of mental states such as motor intentions or emotions. Behaviorally, we found a reduced, but significant, resonance towards a humanoid robot displaying biological motion, compared to a human. Using neuroimaging, we've reported that while perceptual processes in the human occipital and temporal lobe are more strongly engaged when perceiving a humanoid robot than a human action, activity in areas involved in motor resonance depends on attentional modulation for artificial agent more strongly than for human agents. Altogether, these studies using artificial agents offer valuable insights into the interaction of bottom-up and top-down processes in the perception of artificial agents.
Anthropomorphic bias found in typically developing children is not found in children with autistic spectrum disorder
Autism, Dec 17, 2013
The anthropomorphic bias describes the finding that the perceived naturalness of a biological mot... more The anthropomorphic bias describes the finding that the perceived naturalness of a biological motion decreases as the human-likeness of a computer-animated agent increases. To investigate the anthropomorphic bias in autistic children, human or cartoon characters were presented with biological and artificial motions side by side on a touchscreen. Children were required to touch one that would grow while the other would disappear, implicitly rewarding their choice. Only typically developing controls depicted the expected preference for biological motion when rendered with human, but not cartoon, characters. Despite performing the task to report a preference, children with autism depicted neither normal nor reversed anthropomorphic bias, suggesting that they are not sensitive to the congruence of form and motion information when observing computer-animated agents’ actions.
Neuropsychologia, 2007
The appearance of the first intentionally modified stone tools over 2.5 million years ago marked ... more The appearance of the first intentionally modified stone tools over 2.5 million years ago marked a watershed in human evolutionary history, expanding the human adaptive niche and initiating a trend of technological elaboration that continues to the present day. However, the cognitive foundations of this behavioral revolution remain controversial, as do its implications for the nature and evolution of modern human technological abilities. Here we shed new light on the neural and evolutionary foundations of human tool making skill by presenting functional brain imaging data from six inexperienced subjects learning to make stone tools of the kind found in the earliest archaeological record. Functional imaging of this complex, naturalistic task was accomplished through positron emission tomography with the slowly decaying radiological tracer 18 flouro-2-deoxyglucose. Results show that simple stone tool making is supported by a mosaic of primitive and derived parietofrontal perceptual-motor systems, including recently identified human specializations for representation of the central visual field and perception of three-dimensional form from motion. In the naïve tool makers reported here, no activation was observed in prefrontal executive cortices associated with strategic action planning or in inferior parietal cortex thought to play a role in the representation of everyday tool use skills. We conclude that uniquely human capacities for sensorimotor adaptation and affordance perception, rather than abstract conceptualization and planning, were central factors in the initial stages of human technological evolution.
We report on a functional magnetic resonance imaging (fMRI) study of the perception of human and ... more We report on a functional magnetic resonance imaging (fMRI) study of the perception of human and artificial agents. Participants viewed videos of familiar body movements enacted by the android Repliee Q2, the human after whom it was modeled, and the "skinned" version of Q2 revealing its mechanical parts. We used a neural adaptation (repetition suppression) analysis to reveal brain areas sensitive to body movements, and explored whether the identity of the perceived agents modulated these responses. We found significantly higher activity in a distributed network of brain areas for the android, most notably in anterior intraparietal cortex. The responses for the human and the robot with the mechanical appearance resembled each other. We interpret these results within the framework of predictive coding and suggest that the "uncanny valley" phenomenon may have its roots in processing conflicts within the brain's action perception system.
European Journal of Neuroscience, Aug 1, 2003
Structural equation modelling was used to study the change of connectivity during a visual task w... more Structural equation modelling was used to study the change of connectivity during a visual task with continuous variation of the attention load. The model was based on areas defined by the haemodynamic responses described elsewhere A neural network elicited by parametric manipulation of the attention load. Neuroreport, 13, 2331-2334], including occipitotemporal, parietal, temporal and prefrontal (lateral and medial areas) cortices. We have studied stationary-(which does not depend on the attentional load) and attention-related coupling between areas. This allowed the segregation of two subsystems. The first could reflect a system performing the integration step of the visual signal and the second a system participating in response selection. The major finding is the mutual negative influence between the lateral and medial parts of the prefrontal cortex. This negative influence between these two brain regions increased with the attention load. This is interpreted as a modification of the balance between integration and decision processes that are needed for the task to be efficiently completed.
Neuroreport, Oct 1, 2002
Agency is the sense that I am the one generating an action. In this neuroimaging experiment, subj... more Agency is the sense that I am the one generating an action. In this neuroimaging experiment, subjects controlled a circle with a mouse while requested either to lead another circle (i.e., being the agent) or to follow it (i.e., being acted upon). Clusters within the right intraparietal sulcus were associated with following for the most rostral and leading for the most caudal ones. Bilateral activity in the inferior parietal lobule in conditions involving confusion about the origin of the action con¢rmed its role in agency. A lateralization e¡ect was also found in these conditions, the response being stronger in the left inferior parietal lobule when subjects were not the agent of the performed action, and in the right when they were. NeuroReport13:1975^1978
Neurophysiologie cognitive de l'imitation
L'imitation est un phenomene naturel chez l'homme qui consiste a reproduire une action no... more L'imitation est un phenomene naturel chez l'homme qui consiste a reproduire une action nouvelle demontree par un autre agent. Nous proposons un modele cognitif de l'imitation a deux niveaux representationnels. La "resonance motrice", implementee par les boucles parietopremotrices de controle moteur, associe une representation sensorielle et une representation motrice de l'action, et le "code evenementiel" repose sur l'activite du cortex prefrontal. Trois etudes d'imitation reciproque en neuroimagerie ont permis d'etudier le sens de l'agentivite, et deux experiences ont utilise des plans factoriels pour distinguer les substrats neuronaux impliques dans l'imitation des differents elements d'une action transitive ou intransitive. L'imitation d'actions transitives confirme notre modele cognitif a deux niveaux de representation. Le lobule parietal inferieur intervient dans la resonance motrice, avec une specialisation de l'hemisphere gauche pour imiter et du droit dans le sens de l'agentivite.
Characterizing precisely neurophysiological activity involved in natural conversations remains a ... more Characterizing precisely neurophysiological activity involved in natural conversations remains a major challenge. We explore in this paper the relationship between multimodal conversational behavior and brain activity during natural conversations. This is challenging due to Functional Magnetic Resonance Imaging (fMRI) time resolution and to the diversity of the recorded multimodal signals. We use a unique corpus including localized brain activity and behavior recorded during a fMRI experiment when several participants had natural conversations alternatively with a human and a conversational robot. The corpus includes fMRI responses as well as conversational signals that consist of synchronized raw audio and their transcripts, video and eye-tracking recordings. The proposed approach includes a first step to extract discrete neurophysiological time-series from functionally well defined brain areas, as well as behavioral time-series describing specific behaviors. Then, machine learning models are applied to predict neurophysiological timeseries based on the extracted behavioral features. The results show promising prediction scores, and specific causal relationships are found between behaviors and the activity in functional brain areas for both conditions, i.e., human-human and humanrobot conversations.
Motor resonance and MOSAIC
HAL (Le Centre pour la Communication Scientifique Directe), May 8, 2018
We introduce here a new experimental set-up that provides temporally aligned linguistic and behav... more We introduce here a new experimental set-up that provides temporally aligned linguistic and behavioral data together with physiological activity time-series recorded during social interactions. It brings the experimental approach closer to ecological social interaction. Such endeavour requires the aggregation of linguistic, physiological and neuro-cognitive information. Compared to measurement of activity grounded on existing linguistic material our setting presents some additional challenges as we are dealing with conversations. In addition to present the rationale, set-up and preliminary analyses, we discuss (i) the challenges caused by the spontaneous and interactional nature of the activity recorded ; (ii) the problem of balancing experimental set-up between the technical needs and the desire to keep some level of naturalness in the task ; and (iii) the difficulties in relating in a temporal way linguistic events with physiological signals that have their own biological dynamics.
HAL (Le Centre pour la Communication Scientifique Directe), Jul 22, 2017
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific r... more HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Uploads
Papers by Thierry Chaminade