ROBOT AUDITION: ITS RISE AND PERSPECTIVES

2000

In this paper, we present an active audition system for humanoid robot "SIG the humanoid". The audition system of the highly intelligent humanoid requires localization of sound sources and identification of meanings of the sound in the auditory scene. The active audition reported in this paper focuses on improved sound source tracking by integrating audition, vision, and motor movements. Given the multiple sound sources in the auditory scene, SIG actively moves its head to improve localization by aligning microphones orthogonal to the sound source and by capturing the possible sound sources by vision. However, such an active head movement inevitably creates motor noise. The system must adaptively cancel motor noise using motor control signals. The experimental result demonstrates that the active audition by integration of audition, vision, and motor control enables sound source tracking in variety of conditions.

2000

Perception for humanoid should be active, e.g., by moving its body or by controlling parameters of sensors such as cameras or microphones, to perceive environments better. Active vision is one of the common capabilities of a humanoid. Active perception usually causes sounds of actuators, such sounds make audition processing more difficult. A conventional solution of this problem in audition processing is the "stop-perceiveact" principle; that is, humanoid keeps still to listen to sounds, extracts features from input sounds, and makes a plan to act. In this paper, we propose active audition to overcome the shortage of the "stop-perceive-act" principle. Its main issues contain (1) the suppression of internal sounds caused by actuators to enhance external sounds in input sounds; (2) sound source separation from a mixture of sounds, since input sounds do not consist of a single sound source; (3) sensor fusion because sound source separation is an ill-posed problem. A cover of a humanoid is used to suppress its internal sounds to enhance outer sounds. Usually sound source separation assumes the influence of a humanoid head, or a Head-Related Transfer Function (HRTF), which is measured in an anechoic room for many distinct spatial positions. However, this is time-consuming and in addition, the HRTF needs to be remeasured when acoustic environments change. Therefore, we present a new method based on the epipolar geometry which does not use any HRTF. We present an active audition system for humanoid robot "SIG the humanoid". The audition system of a humanoid requires localization of sound sources and identification of meanings of the sound in the auditory scene. The active audition reported in this paper focuses on improved sound source tracking by integrating audition, vision, and motor movements. Given the multiple sound sources in the auditory scene, SIG actively moves its head to improve localization by aligning microphones orthogonal to the sound source and by capturing the possible sound sources by vision. However, such an active head movement inevitably creates motor noise. The system must adaptively cancel motor noise using motor control signals. The experimental result demonstrates that the active audition by integration of audition, vision, and motor control enables sound source tracking in variety of conditions. ¼

2007

This paper describes the implementation of a novel real time robot audition system which combines a 3D sound localisation system and a voice characterisation (VC) system. The localisation system employs a 4 microphone array and uses the time delay estimation method. Accuracy is improved through the use of a correlation confidence threshold and a median filter. The VC system, which classifies between speech, non speech and silence, uses a decision tree classifier and a feature set comprising MFCCs, mean MFCCs and variance in MFCCs. The complete system has a processing time of 0.73x real time, and a range of up to 3 m. The compact design, high accuracy, and real time processing ability makes the system and the approach well suited to robotics.

2004

We discuss biologically inspired techniques for processing sound with a view to solving the what and where tasks. Starting from an auditorynerve like representation, we discuss an onset grouping technique for determining when to compute interaural time and intensity differences, and hence find t he direction (where) of the source even in the presence of reflections. We have not yet implemented the what task, and we discuss some possible strategies.

2017

Intuitive spoken dialogues are a prerequisite for human-robot interaction. In many practical situations, robots must be able to identify and focus on sources of interest in the presence of interfering speakers. Techniques such as spatial filtering and blind source separation are therefore often used, but rely on accurate knowledge of the source location. In practice, sound emitted in enclosed environments is subject to reverberation and noise. Hence, sound source localization must be robust to both diffuse noise due to late reverberation, as well as spurious detections due to early reflections. For improved robustness against reverberation, this paper proposes a novel approach for sound source tracking that constructively exploits the spatial diversity of a microphone array installed in a moving robot. In previous work, we developed speaker localization approaches using expectation-maximization (EM) approaches and using Bayesian approaches. In this paper we propose to combine the EM and Bayesian approach in one framework for improved robustness against reverberation and noise.

2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566)

We propose a system that gives a mobile robot the ability to separate simultaneous sound sources. A microphone array is used along with a real-time dedicated implementation of Geometric Source Separation and a post-filter that gives us a further reduction of interferences from other sources. We present results and comparisons for separation of multiple non-stationary speech sources combined with noise sources. The main advantage of our approach for mobile robots resides in the fact that both the frequency-domain Geometric Source Separation algorithm and the post-filter are able to adapt rapidly to new sources and non-stationarity. Separation results are presented for three simultaneous interfering speakers in the presence of noise. A reduction of log spectral distortion (LSD) and increase of signalto-noise ratio (SNR) of approximately 10 dB and 14 dB are observed.

Robotics and Autonomous Systems, 2017

Sound source localization (SSL) in a robotic platform has been essential in the overall scheme of robot audition. It allows a robot to locate a sound source by sound alone. It has an important impact on other robot audition modules, such as source separation, and it enriches human–robot interaction by complementing the robot's perceptual capabilities. The main objective of this review is to thoroughly map the current state of the SSL field for the reader and provide a starting point to SSL in robotics. To this effect, we present: the evolution and historical context of SSL in robotics; an extensive review and classification of SSL techniques and popular tracking methodologies; different facets of SSL as well as its state-of-the-art; evaluation methodologies used for SSL; and a set of challenges and research motivations.

Frontiers in Robotics and AI, 2022

Artificial audition aims at providing hearing capabilities to machines, computers and robots. Existing frameworks in robot audition offer interesting sound source localization, tracking and separation performance, although involve a significant amount of computations that limit their use on robots with embedded computing capabilities. This paper presents ODAS, the Open embeddeD Audition System framework, which includes strategies to reduce the computational load and perform robot audition tasks on low-cost embedded computing systems. It presents key features of ODAS, along with cases illustrating its uses in different robots and artificial audition applications.

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012

To demonstrate the influence of an artificial audition system on speech recognition and dialogue management for a robot, this paper presents a case study involving soft coupling of ManyEars, a sound source localization, tracking and separation system, with the CSLU Dialogue Management system. Trials were conducted in a laboratory and a cafeteria. Results indicate that preprocessing of the audio signals by ManyEars improves speech recognition and dialogue management of the system, demonstrating the feasibility and the added flexibility provided by ManyEars for a robot to interact vocally with humans in a wide variety of contexts.

Interaction between humans involves a plethora of sensory information, both in the form of explicit communication as well as more subtle unconsciously perceived signals. In order to enable natural human-robot interaction, robots will have to acquire the skills to detect and meaningfully integrate information from multiple modalities. In this article, we focus on sound localization in the context of a multi-sensory humanoid robot that combines audio and video information to yield natural and intuitive responses to human behavior, such as directed eye-head movements towards natural stimuli. We highlight four common sound source localization algorithms and compare their performance and advantages for real-time interaction. We also briefly introduce an integrated distributed control framework called DVC, where additional modalities such as speech recognition, visual tracking, or object recognition can easily be integrated. We further describe the way the sound localization module has been integrated in our humanoid robot, CB.