The service robot that carries out task s ordered by the users through speech needs a vision sy s... more The service robot that carries out task s ordered by the users through speech needs a vision sy stem to recogniz e the obj ects appearing in the orders and a speech interf ace f or natural communication with the user. The user' s order may be ex pl icit or impl icit. The speech interf aces shoul d have a capabil ity of deal ing with ex pl icit as wel l as impl icit utterances. I n this paper, we present ' how environmental cues hel p in understanding user' s orders. ' We assume that humans usual l y put a particul ar obj ect on a smal l number of pl aces in the environment. Using the environmental k nowl edge, the robot can ef f icientl y understand and accompl ish the user' s demand with l ess vision task and user burden.
Use of Spatial Reference Systems in Interactive Object Recognition
We are developing a helper robot that carries out tasks ordered by users through speech. The robo... more We are developing a helper robot that carries out tasks ordered by users through speech. The robot needs a vision system to recognize the objects appearing in the orders. It is, however, difficult to realize vision systems that can work in various conditions. They may find many objects and cannot determine which is the target. We have proposed a method
Geometric and Photometric Analysis for Interactively Recognizing Multicolor or Partially Occluded Objects
An effective human-robot interaction is essential for wide penetration of service robots into the... more An effective human-robot interaction is essential for wide penetration of service robots into the market. Such robots need vision systems to recognize objects. It is, however, difficult to realize vision systems that can work in various conditions. More robust techniques of object recognition and image segmentation are essential. Thus, we have proposed to use the human user’s assistance for objects recognition through speech. Our previous system assumes that it can segment images without failure. However, if there are occluded objects and/or objects composed of multicolor parts, segmentation failures cannot be avoided. This paper presents an extended system that can recognize objects in occlusion and/or multicolor cases using geometric and photometric analysis of images. If the robot is not sure about the segmentation results, it asks questions of the user by appropriate expressions depending on the certainty to remove the ambiguity.
In research on human-robot interaction the interest is currently shifting from uni-modal dialog s... more In research on human-robot interaction the interest is currently shifting from uni-modal dialog systems to multi-modal interaction schemes. We present a system for human-style interaction with a robot that is integrated on our mobile robot BIRON. To model the dialog we adopt an extended grounding concept with a mechanism to handle multi-modal in-and output where object references are resolved by the interaction with an object attention system (OAS). The OAS integrates multiple input from, e.g., the object and gesture recognition systems and provides the information for a common representation. This representation can be accessed by both modules and combines symbolic verbal attributes with sensor-based features. We argue that such a representation is necessary to achieve a robust and efficient information processing.
Generation of efficient and user-friendly queries for helper robots to detect target objects
Advanced Robotics, 2006
ABSTRACT We are developing a helper robot that carries out tasks ordered by users through speech.... more ABSTRACT We are developing a helper robot that carries out tasks ordered by users through speech. The robot needs a vision system to recognize the objects appearing in the orders. However, conventional vision systems cannot recognize objects in complex scenes. They may find many objects and cannot determine which is the target. This paper proposes a method of using a conversation with the user to solve this problem. The robot asks a question to which the user can easily answer and whose answer can efficiently reduce the number of candidate objects. It considers the characteristics of features used for object identification such as the ease for humans to specify them by word, generating a user-friendly and efficient sequence of questions. Experimental results show that the robot can detect target objects by asking the questions generated by the method.
We are developing a helper robot that carries out tasks ordered by users through speech. The robo... more We are developing a helper robot that carries out tasks ordered by users through speech. The robot needs a vision system to recognize objects appearing in the orders. However, conventional vision systems cannot recognize objects in complex scenes. They may find many objects and cannot determine which is the target. This paper proposes a method of using a conversation with the user to solve this problem. The robot asks a question to which the user can easily answer and whose answer can efficiently reduce the number of candidate objects. It considers the characteristics of features used for object recognition such as the easiness for humans to specify them by word, generating a user-friendly and efficient sequence of questions. Experimental results show that the robot can detect target objects by asking the questions generated by the method.
The service robot that carries out task s ordered by the users through speech needs a vision sy s... more The service robot that carries out task s ordered by the users through speech needs a vision sy stem to recogniz e the obj ects appearing in the orders and a speech interf ace f or natural communication with the user. The user' s order may be ex pl icit or impl icit. The speech interf aces shoul d have a capabil ity of deal ing with ex pl icit as wel l as impl icit utterances. I n this paper, we present ' how environmental cues hel p in understanding user' s orders. ' We assume that humans usual l y put a particul ar obj ect on a smal l number of pl aces in the environment. Using the environmental k nowl edge, the robot can ef f icientl y understand and accompl ish the user' s demand with l ess vision task and user burden.
Object recognition through human-robot interaction by speech
Abstract We are developing a helper robot that carries out tasks ordered by users through speech.... more Abstract We are developing a helper robot that carries out tasks ordered by users through speech. The robot needs a vision system to recognize objects appearing in the orders. However, conventional vision systems cannot recognize objects in complex scenes. They ...
We are developing a helper robot that carries out tasks ordered by the user through speech. The r... more We are developing a helper robot that carries out tasks ordered by the user through speech. The robot needs a vision system to recognize the objects appearing in the orders. It is, however, difficult to realize vision systems that can work in various conditions. Thus, we have proposed to use the human user's assistance through speech. When the vision system cannot achieve a task, the robot makes a speech to the user so that the natural response by the user can give helpful information for its vision system. Our previous system assumes that it can segment images without failure. However, if there are occluded objects and/or objects composed of multicolor parts, segmentation failures cannot be avoided. This paper presents an extended system that tries to recover from segmentation failures using photometric invariance. If the system is not sure about segmentation results, the system asks the user by appropriate expressions depending on the invariant values. Experimental results show the usefulness of the system.
An effective human-robot interaction is essential for wide penetration of service robots into the... more An effective human-robot interaction is essential for wide penetration of service robots into the market. Such robot needs a vision system to recognize objects. It is, however, difficult to realize vision systems that can work in various conditions. More robust techniques of object recognition and image segmentation are essential. Thus, we have proposed to use the human user's assistance for objects recognition through speech. This paper presents a system that recognizes objects in occlusion and/or multicolor cases using geometric and photometric analysis of images. Based on the analysis results, the system makes a hypothesis of the scene. Then, it asks the user for confirmation by describing the hypothesis. If the hypothesis is not correct, the system generates another hypothesis until it correctly understands the scene. Through experiments on a real mobile robot, we have confirmed the usefulness of the system.
Recovery from Segmentation Failures Using Photometric Invariance in an Interactive Object Recognition System
We are developing a helper robot that carries out tasks ordered by the user through speech. The r... more We are developing a helper robot that carries out tasks ordered by the user through speech. The robot needs a vision system to recognize the objects appearing in the orders. It is, however, difficult to realize vision systems that can work in various conditions. Thus, we have proposed to use the human user's assistance through speech. When the vision system cannot achieve a task, the robot makes a speech to the user so that the natural response by the user can give helpful information for its vision system. Our previous system assumes that it can segment images without failure. However, if there are occluded objects and/or objects composed of multicolor parts, segmentation failures cannot be avoided. This paper presents an extended system that tries to recover from segmentation failures using photometric invariance. If the system is not sure about segmentation results, the system asks the user by appropriate expressions depending on the invariant values.
Uploads
Papers by Rahmadi Kurnia