Camera-based gesture recognition for robot control

IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2000

A gesture recognition system (GRS) is comprised of a gesture, gesture-capture device (sensor), tracking algorithm (for motion capture), feature extraction, and classification algorithm. With the impending movement toward natural communication with mechanical and software systems, it is important to examine the first apparatus that separates the human communicator and the device being controlled. Although there are numerous reviews of GRSs, a comprehensive analysis of the integration of sensors into GRSs and their impact on system performance is lacking in the professional literature. Thus, we have undertaken this effort. Determination of the sensor stimulus, context of use, and sensor platform are major preliminary design issues in GRSs. Thus, these three components form the basic structure of our taxonomy. We emphasize the relationship between these critical components and the design of the GRS in terms of its architectural functions and computational requirements. In this treatise, we consider sensors that are capable of capturing dynamic and static arm and hand gestures. Although we discuss various sensor types, our main focus is on visual sensors as we expect these to become the sensor of choice in the foreseeable future. We delineate the challenges ahead for their increased effectiveness in this application domain. We note as a special challenge, the development of sensors that take over many of the functions the GRS designer struggles with today. We believe our contribution, in this first survey on sensors for GRSs, can give valuable insights into this important research and development topic, and encourage advanced research directions and new approaches. interests include human robotic cooperative learning, robotic teleoperation, robotic visual area search, machine learning, computer vision, soft-computing applications, hand-gesture recognition, optimal gesture vocabulary design, color face and multiobject tracking, and intelligent multimodal man-machine interfaces. Dr. Stern is a member of the IEEE Computer Society.

Procedia Technology, 2012

Human Computer Interaction techniques have become a bottleneck in the effective utilization of the available information flow. The development of user interfaces influences the changes in the Human-Computer Interaction (HCI). Human hand gestures have been a mode of non verbal interaction widely used. Naturalistic and intuitiveness of the hand gesture has been a great motivating factor for the researchers in the area of HCI to put their efforts to research and develop the more promising means of interaction between human and computers. This paper designs a system for gestural interaction between a user and a computer in dynamic environment. The gesture recognition system uses image processing techniques for detection, segmentation, tracking and recognition of hand gestures for converting it to a meaningful command. The interface being proposed here can be substantially applied towards different applications like image browser, games etc.

2010 IEEE International Conference on Image Processing, 2010

ABSTRACT In everyday live head gestures such as head shaking or nodding and hand gestures like pointing gestures form important aspects of human-human interaction. Therefore, recent research considers integrating these intuitive communication cues into technical systems for improving and easing human-computer interaction. In this paper we present a vision-based system to recognize head gestures (nodding, shaking, neutral) and dynamic hand gestures (hand moving right/left/up/down, fist moving right/left) in real-time. The gestural input delivers a communication modality for a human-robot interaction scenario situated in an assistive household environment. The use of fast low-level image-feature extraction methods contributes to the real-time capability of the system and advanced classification approaches relying on Graphical Models provide high robustness. Graphical Models offer the possibility to group the input features in several sub-nodes resulting in a better classification than obtained via a traditional Hidden Markov Model classification. The applied grouping can regard interdependencies owing to, either physical constraints (like for the head gestures), or interrelations between shape and motion (like for the hand gestures).

2010

FPGA implementation of real-time human motion recognition on a reconfigurable video processing architecture Received: date / Revised: date Abstract In recent years, automatic human motion recognition has been widely researched within the computer vision and image processing communities. Here we propose a real-time embedded vision solution for human motion recognition implemented on a ubiquitous device. There are three main contributions in this paper. Firstly, we have developed a fast human motion recognition system with simple motion features and a linear Support Vector Machine(SVM) classifier. The method has been tested on a large, public human action dataset and achieved competitive performance for the temporal template (eg. “motion history image”) class of approaches. Secondly, we have developed a reconfigurable, FPGA based video processing architecture. One advantage of this architecture is that the system processing performance can be reconfigured for a particular application,...

— Gesture is a way of Human Computer Interaction. It is a technique that is mainly carried out to make impaired community communicate with normal humans. It's main aim is to convey human gestures to computing device. They are efficient for natural and intuitive human-computer interaction. To achieve this goal, computers should be able to visually recognize hand gestures from video input. We propose a new architecture to solve the problem of real-time gesture recognition. The fundamental idea is to use a combination of hand gestures to control various functionalities. This will include two parts, first is to track the movement of the hand and secondly is to track the orientation of the hand. This paper presents a robust and efficient technique for gesture recognition. The OpenCV library provides us a greatly interesting demonstration for object detection. Furthermore, it provides programs (or functions) that they used to train classifiers for their gesture detection system, called HaarTraining, so that we can create our own object classifiers using these functions. Our working environment is Visual Studio on Windows 7. The objective of this project is to develop an application for recognition of hand gestures with reasonable accuracy and thus creating windows media player compatible remote control.

2001

1. Introduction A primary goal of virtual environments (VE) is to provide natural, efficient, powerful, and flexible interaction. Gesture as an input modality can help meet these requirements. Human gestures are certainly natural and flexible, and may often be efficient and powerful, especially as compared with alternative interaction modes. This chapter will cover automatic gesture recognition, particularly computer vision based techniques that do not require the user to wear extra sensors, clothing or equipment.

AMC'98 - Coimbra. 1998 5th International Workshop on Advanced Motion Control. Proceedings (Cat. No.98TH8354)

Persons detection and tracking in a scene constitutes a very active working field due to their applicability to many areas and specially in MMI applications. The system presented in this paper is the result of the work carried out in the last years to improve the reliability and efficiency of operation in natural complex environments. Image segmentation is performed from movement detection. The recognition of moving objects such as persons, is performed by means of a very simplified multicylindrical geometrical model, thus allowing to operate with a low computational cost.

Journal of Real-Time Image Processing, 2014

The overarching goal of the pattern recognition community consists of presenting hypotheses to describe classes of objects using mathematical models, processing the information to eliminate the presence of noise, and selecting the model that best explains the given observations; nevertheless, it does not prioritize in memory and time complexity when matching models to observations. Given that we describe, explain and manipulate these objects through the perceptual system, there is an increasing need to favor those pattern recognition techniques that can explain, process and predict large volumes of visual data in realtime. Such techniques cannot be developed ''in vitro'' due to the physical constraints of the complex environment and the context in which these techniques are used. Further, these new methods need to achieve high detection, classification and recognition accuracies in real-time even when these are conflicting objectives. To make pattern recognition techniques viable for practical applications (such as surveillance, robotics and medical applications), considerations such as computational complexity reduction, hardware implementation, software optimization, and strategies for parallelizing solutions must be observed.

Hand gesture recognition, being a natural way of human computer interaction, is an area of active research in computer vision and machine learning. This is an area with many different possible applications, giving users a simpler and more natural way to communicate with robots/systems interfaces, without the need for extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them to convey information or for device control. This work intends to study and implement a solution, generic enough, able to interpret user commands, composed of a set of dynamic and static gestures, and use those solutions to build an application able to work in a real- time human-computer interaction systems. The proposed solution is composed of two modules controlled by a FSM (Finite State Machine): a real time hand tracking and feature extraction system, supported by a SVM (Support Vector Machine) model for static hand posture classification and a set of HMMs (Hidden Markov Models) for dynamic single stroke hand gesture recognition. The experimental results showed that the system works very reliably, being able to recognize the set of defined commands in real-time. The SVM model for hand posture classification, trained with the selected hand features, achieved an accuracy of 99,2%. The proposed solution as the advantage of being computationally simple to train and use, and at the same time generic enough, allowing its application in any robot/system command interface.

Real-Time Image and Video Processing 2010, 2010

Human communication relies on a large number of different communication mechanisms like spoken language, facial expressions, or gestures. Facial expressions and gestures are one of the main nonverbal communication mechanisms and pass large amounts of information between human dialog partners. Therefore, to allow for intuitive human-machine interaction, a real-time capable processing and recognition of facial expressions, hand and head gestures are of great importance. We present a system that is tackling these challenges. The input features for the dynamic head gestures and facial expressions are obtained from a sophisticated three-dimensional model, which is fitted to the user in a real-time capable manner. Applying this model different kinds of information are extracted from the image data and afterwards handed over to a real-time capable data-transferring framework, the so-called Real-Time DataBase (RTDB). In addition to the head-and facial-related features, also low-level image features regarding the human hand-optical flow, Hu-moments-are stored into the RTDB for the evaluation process of hand gestures. In general, the input of a single camera is sufficient for the parallel evaluation of the different gestures and facial expressions. The real-time capable recognition of the dynamic hand and head gestures are performed via different Hidden Markov Models, which have proven to be a quick and real-time capable classification method. On the other hand, for the facial expressions classical decision trees or more sophisticated support vector machines are used for the classification process. These obtained results of the classification processes are again handed over to the RTDB, where other processes (like a Dialog Management Unit) can easily access them without any blocking effects. In addition, an adjustable amount of history can be stored by the RTDB buffer unit.