An Active Vision System for a Social Robot

This short paper discusses the importance of human-like gaze behaviors for humanoid robots with physical eyes. It gives a brief overview of the functions gaze fulfills in human-human interactions from a social evolutionary perspective. In the second part we describe how human-like gaze has been implemented in robot in the research field of social robotics in the last years. The last part of the paper briefly introduces an architecture for a conversational gaze controller (CGC). The parameters of this gaze controller are based on the analysis of a large corpus of gaze tracking data collected during human-human conversations. We describe our experimental approach for obtaining this data and discuss the implications of endowing robots with human-like behaviors and enabling them to engage in face to face social interactions.

IEEE Transactions on Human-Machine Systems, 2015

We propose a human-robot interaction approach for social robots that attracts and controls the attention of a target person depending on her/his current visual focus of attention. The system detects the person's current task (attention) and estimates the level by using the "task-related contextual cues" and "gaze pattern." The attention level is used to determine the suitable time to attract the target person's attention toward the robot. The robot detects the interest or willingness of the target person to interact with it. Then, depending on the level of interest of the target person, the robot generates awareness and establishes a communication channel with her/him. To evaluate the performance, we conducted an experiment using our static robot to attract the target human's attention when she/he is involved in four different tasks: reading, writing, browsing, and viewing paintings. The proposed robot determines the level of attention of the current task and considers the situation of the target person. Questionnaire measures confirmed that the proposed robot outperforms a simple attention control robot in attracting participants' attention in an acceptable way. It also causes less disturbance and establishes effective eye contact. We implemented the system into a commercial robotic platform (Robovie-R3) to initiate interaction between visitors and the robot in a museum scenario. The robot determined the visitors' gaze points and established a successful interaction with a success rate of 91.7%. Index Terms-Gaze pattern, human-robot interaction, taskrelated contextual cues, visual focus of attention (VFOA).

This paper presents part of an on-going project to integrate perception, attention, drives, emotions, behavior arbitration, and expressive acts for a robot designed to interact socially with humans. We present the design of a visual attention system based on a model of human visual search behavior from Wolfe (1994). The attention system integrates perceptions (motion detection, color saliency, and face popouts) with habituation effects and influences from the robot's motivational and behavioral state to create a context-dependent attention activation map. This activation map is used to direct eye movements and to satiate the drives of the motivational system.

IEEE Transactions on Cognitive and Developmental Systems

Guest Editorial A Sense of Interaction in Humans and Robots: From Visual Perception to Social Cognition H UMAN ability to interact with one another is substantially strengthened by vision, with several visual processes tuned to support prosocial behaviors since early infancy. A key challenge of robotics research is to provide artificial agents with similar advanced visual perception skills, with the ultimate goal of designing machines able to recognize and interpret both explicit and implicit communication cues embedded in human behaviors. This special issue addresses this challenge, with a focus both on understanding human perception supporting interaction abilities and on the implementation perspective, considering new algorithms and modeling efforts brought forward to improve current robotics. This multidisciplinary effort aims to bring innovations not only in human-machine interaction but also in domains such as developmental psychology and cognitive rehabilitation. I. SCOPE OF THIS SPECIAL ISSUE Since early infancy, the ability of humans to interact with one another has been substantially strengthened by vision, with several visual processes tuned to support prosocial behaviors. For instance, a natural predisposition to look at human faces or to detect biological motion is present at birth [items 1) and 2) in the Appendix]. More refined abilities-as the understanding and anticipation of others' actions and intentions [items 3) and 4) in the Appendix]progressively develop with age, leading, in a few years, to a full capability of interaction based on mutual understanding, joint coordination, and collaboration. Today, a key challenge of robotics research is to provide artificial agents with similar advanced visual perception skills, with the ultimate goal of designing machines able to recognize and interpret both explicit and implicit communication cues embedded in human behaviors [items 5)-9) in the Appendix]. These achievements pave the way for the large-scale use of human-robot interaction applications in a variety of contexts, ranging from the design of personal robots, to physical, social, and cognitive rehabilitation. Understanding how efficient and seamless collaborations can be achieved among human partners and which of them are the explicit and implicit cues intuitively

SN Applied Sciences

Empowering a robot to direct its attention to the most appropriate target at all times during multi-party interactions is an interesting and useful challenge to establish natural communication between the robot and users. In this paper, implementing a social gaze control system suitable for multi-person interactions with a RASA social robot is discussed. This system takes some important verbal and non-verbal social cues into account, and at each moment enables the robot to decide socially at which human it should direct its gaze. The algorithm for the target selection has been enhanced, compared to past studies, by quantitating the effects of distance and orientation on grabbing humans' attention in addition to the inherent importance of each cue in communications based on the gaze behavior of a group of human participants. After this was completed, another group of volunteers were employed to examining the performance of the RASA robot equipped with this system. Their average gaze pattern was compared with the targets selected by the robot in a real situation, and their opinions on the sociability and intelligence of the system were recorded. We indicated that the gaze generated by the robotic system matched the average gaze pattern of the participants 76.9% in an 80-s real-life scenario. Moreover, the results of the questionnaire showed us that ~ 90% of the subjects felt that at times RASA was really looked at them with a quite high average score of 4.33 out of 5.

Concepts, Methodologies, Tools, and Applications

Computer vision is essential to develop a social robotic system capable to interact with humans. It is responsible to extract and represent the information around the robot. Furthermore, a learning mechanism, to select correctly an action to be executed in the environment, pro-active mechanism, to engage in an interaction, and voice mechanism, are indispensable to develop a social robot. All these mechanisms together provide a robot emulate some human behavior, like shared attention. Then, this chapter presents a robotic architecture that is composed with such mechanisms to make possible interactions between a robotic head with a caregiver, through of the shared attention learning with identification of some objects.

In this paper, we introduce the main components comprising the action-perception loop of an overarching framework implementing artificial attention, designed to fulfil the requirements of social interaction (i.e., reciprocity, and awareness), with strong inspiration on current theories in functional neuroscience. We demonstrate the potential of our framework, by showing how it exhibits coherent behaviour without any inbuilt prior expectations regarding the experimental scenario. Current research in cognitive systems for social robots has suggested that automatic attention mechanisms are essential to social interaction. In fact, we hypothesise that enabling artificial cognitive systems with middleware implementing these mechanisms will empower robots to perform adaptively and with a higher degree of autonomy in complex and social environments. However, this type of assumption is yet to be convincingly and systematically put to the test. The ultimate goal will be to test our working hypothesis and the role of attention in adaptive, social robotics.

Current computational models of visual attention focus on relevant information and overlook the current situation other than its focus. However, studies in visual cognition show that humans use context to facilitate object detection in natural scenes by directing their attention or eyes to diagnostic regions. Top-down attention is knowledge driven approach where an idea, or decision is controlled or directed from the highest level of internally driven attention. Using visual and verbal affective intent goals of a responsive and social agent can be achieved. Developing a social agent to perceive things on the basis of merging the visionary system with verbal patterns and facial patterns including attention level and stress level can lead to the required result. Essential consequences can be described as sensing a stimuli using auditory and visual system, then make a perception (processing of stimulus) and decision using top down approach to interact with human beings. In interaction with human beings, background knowledge and previous experience will influence on perception. This influence is a key factor to achieve goal. The top-down component uses accumulated statistical knowledge of the visual features of the desired search target and background clutter, to optimally tune the bottom-up maps. Testing on an artificial and natural scene shows that the model's predictions are consistent with a large body of available literature on human psycho physics of visual search. These results suggest that our model may provide good approximation of how humans combine bottom-up and top-down cues such as to optimize target detection and efficiently performing speed.

IEEE Transactions on Cognitive and Developmental Systems, 2016

Robot's perception is essential for performing highlevel tasks such as understanding, learning, and in general, human-robot interaction (HRI). For this reason, different perception systems have been proposed for different robotic platforms in order to detect high-level features such as facial expressions and body gestures. However, due to the variety of robotics software architectures and hardware platforms, these highly customized solutions are hardly interchangeable and adaptable to different HRI contexts. In addition, most of the developed systems have one issue in common: they detect features without awareness of the real-world contexts (e.g., detection of environmental sound assuming that it belongs to a person who is speaking, or treating a face printed on a sheet of paper as belonging to a real subject). This paper presents a novel social perception system (SPS) that has been designed to address the previous issues. SPS is an outof-the-box system that can be integrated into different robotic platforms irrespective of hardware and software specifications. SPS detects, tracks and delivers in real-time to robots, a wide range of human-and environment-relevant features with the awareness of their real-world contexts. We tested SPS in a typical scenario of HRI for the following purposes: to demonstrate the system capability in detecting several high-level perceptual features as well as to test the system capability to be integrated into different robotics platforms. Results show the promising capability of the system in perceiving real world in different social robotics platforms, as tested in two humanoid robots i.e., FACE and ZENO.

Advances in Human-Computer Interaction, 2014

Making eye contact is a most important prerequisite function of humans to initiate a conversation with others. However, it is not an easy task for a robot to make eye contact with a human if they are not facing each other initially or the human is intensely engaged his/her task. If the robot would like to start communication with a particular person, it should turn its gaze to that person and make eye contact with him/her. However, such a turning action alone is not enough to set up an eye contact phenomenon in all cases. Therefore, the robot should perform some stronger actions in some situations so that it can attract the target person before meeting his/her gaze. In this paper, we proposed a conceptual model of eye contact for social robots consisting of two phases: capturing attention and ensuring the attention capture. Evaluation experiments with human participants reveal the effectiveness of the proposed model in four viewing situations, namely, central field of view, near per...