A context dependent attention system for a humanoid robot

arXiv (Cornell University), 2021

To enhance human-robot social interaction, it is essential for robots to process multiple social cues in a complex real-world environment. However, incongruency of input information across modalities is inevitable and could be challenging for robots to process. To tackle this challenge, our study adopted the neurorobotic paradigm of crossmodal conflict resolution to make a robot express human-like social attention. A behavioural experiment was conducted on 37 participants for the human study. We designed a round-table meeting scenario with three animated avatars to improve ecological validity. Each avatar wore a medical mask to obscure the facial cues of the nose, mouth, and jaw. The central avatar shifted its eye gaze while the peripheral avatars generated sound. Gaze direction and sound locations were either spatially congruent or incongruent. We observed that the central avatar's dynamic gaze could trigger crossmodal social attention responses. In particular, human performances are better under the congruent audiovisual condition than the incongruent condition. Our saliency prediction model was trained to detect social cues, predict audiovisual saliency, and attend selectively for the robot study. After mounting the trained model on the iCub, the robot was exposed to laboratory conditions similar to the human experiment.

2006 6th IEEE-RAS International Conference on Humanoid Robots, 2006

This paper presents a framework for a visual search behavior of a 3D object in a 3D environment performed by HRP-2 humanoid robot. Object search is a sensor planning problem in which appropriate sensor configuration must be selected in order to allow a proper recognition. The sensor planning is formulated as an optimization problem in which the goal is to maximize the target detection probability while minimizing the energy/distance and time to achieve the task. The knowledge of the target position is encoded in a probability distribution map which will be updated after each sensing operation. Specificities of the humanoid robot as well as the characteristics of the recognition system are taken into account to restrict the sensor configuration space to meet heavy time constraints on the behavior in order to have a reactive and near human like reaction time.

2014 IEEE International Conference on Robotics and Automation (ICRA), 2014

For smooth interaction between human and robot, the robot should have an ability to manipulate human attention and behaviors. In this study, we developed a visual attention model for manipulating human attention by a robot. The model consists of two modules, such as the saliency map generation module and manipulation map generation module. The saliency map describes the bottom-up effect of visual stimuli on human attention and the manipulation map describes the top-down effect of face, hands and gaze. In order to evaluate the proposed attention model, we measured human gaze points during watching a magic video, and applied the attention model to the video. Based on the result of this experiment, the proposed attention model can better explain human visual attention than the original saliency map.

IEEE Transactions on Human-Machine Systems, 2015

We propose a human-robot interaction approach for social robots that attracts and controls the attention of a target person depending on her/his current visual focus of attention. The system detects the person's current task (attention) and estimates the level by using the "task-related contextual cues" and "gaze pattern." The attention level is used to determine the suitable time to attract the target person's attention toward the robot. The robot detects the interest or willingness of the target person to interact with it. Then, depending on the level of interest of the target person, the robot generates awareness and establishes a communication channel with her/him. To evaluate the performance, we conducted an experiment using our static robot to attract the target human's attention when she/he is involved in four different tasks: reading, writing, browsing, and viewing paintings. The proposed robot determines the level of attention of the current task and considers the situation of the target person. Questionnaire measures confirmed that the proposed robot outperforms a simple attention control robot in attracting participants' attention in an acceptable way. It also causes less disturbance and establishes effective eye contact. We implemented the system into a commercial robotic platform (Robovie-R3) to initiate interaction between visitors and the robot in a museum scenario. The robot determined the visitors' gaze points and established a successful interaction with a success rate of 91.7%. Index Terms-Gaze pattern, human-robot interaction, taskrelated contextual cues, visual focus of attention (VFOA).

2005

The Problem: If a robot is intended to interact with people, it needs an active vision system that can serve both a perceptual and communicative function.

In this paper, we introduce the main components comprising the action-perception loop of an overarching framework implementing artificial attention, designed to fulfil the requirements of social interaction (i.e., reciprocity, and awareness), with strong inspiration on current theories in functional neuroscience. We demonstrate the potential of our framework, by showing how it exhibits coherent behaviour without any inbuilt prior expectations regarding the experimental scenario. Current research in cognitive systems for social robots has suggested that automatic attention mechanisms are essential to social interaction. In fact, we hypothesise that enabling artificial cognitive systems with middleware implementing these mechanisms will empower robots to perform adaptively and with a higher degree of autonomy in complex and social environments. However, this type of assumption is yet to be convincingly and systematically put to the test. The ultimate goal will be to test our working hypothesis and the role of attention in adaptive, social robotics.

Lecture Notes in Computer Science, 2003

Social robots are receiving much interest in the robotics community. The most important goal for such robots lies in their interaction capabilities. An attention system is crucial, both as a filter to center the robot's perceptual resources and as a mean of letting the observer know that the robot has intentionality. In this paper a simple but flexible and functional attentional model is described. The model, which has been implemented in an interactive robot currently under development, fuses both visual and auditive information extracted from the robot's environment, and can incorporate knowledge-based influences on attention.

Sensors

A robust perception system is crucial for natural human–robot interaction. An essential capability of these systems is to provide a rich representation of the robot’s environment, typically using multiple sensory sources. Moreover, this information allows the robot to react to both external stimuli and user responses. The novel contribution of this paper is the development of a perception architecture, which was based on the bio-inspired concept of endogenous attention being integrated into a real social robot. In this paper, the architecture is defined at a theoretical level to provide insights into the underlying bio-inspired mechanisms and at a practical level to integrate and test the architecture within the complete architecture of a robot. We also defined mechanisms to establish the most salient stimulus for the detection or task in question. Furthermore, the attention-based architecture uses information from the robot’s decision-making system to produce user responses and rob...

2011

In biological vision systems, the attention mechanism is responsible for selecting the relevant information from the sensed field of view. In robotics, this ability is specially useful because of the restrictions in computational resources which are necessary to simultaneously perform different tasks. An emerging area in robotics is developing social robots which are capable to navigate and to interact with humans and with their environment by perceiving the real world in a similar way that people do. In this proposal, we focus on the development of an object-based attention mechanism for a social robot. It consists of three main modules. The first one (preattentive stage) implements a concept of saliency based on "proto-objects." In the second stage (semiattentive), significant items according to the tasks to accomplish are identified and tracked. Finally, the attentive stage fixes the field of attention to the most salient object depending on the current task.