KINECT SENSOR IMPLEMENTATION IN FANUC ROBOT MANIPULATION

This paper presents the design process of a gesture control system based on the Microsoft Kinect sensor. An environment enabling implementation of the integrated system, using a variety of equipment and software was selected and prepared. A method for integrating the sensor with the Arduino environment has also been discussed. Algorithms for remote gesture control of the given servodrive angle and the position of the robot arm gripper were prepared. The results of several experiments, which were carried out in order to determine the optimal method for starting, controlling, and stopping the drive and for assessment of the accuracy of the proposed method for the arm control, are presented.

Journal of Telecommunication, Electronic and Computer Engineering, 2018

The conventional method to control something like machines such as remote controllers or wearable sensors have its limitation and cannot cater the high demands in some scenarios. To overcome this situation, a real-time Kinect finger tracking and recognition is developed to control a servo gripper via a single board microcontroller, Arduino. This paper presents hand and finger tracking method that can determine the number of fingers and also the angle of the finger’s position. The hand gesture is captured by the Kinect and go through a series of image processing and finally, the information will send to the Arduino. The image processing method includes a detection of hand using depth sensor on the Kinect, then finds the finger and calculates the angle of the finger’s position. The convex hull algorithm is used to represent the region of the hand. The fingertips are recognized by calculating the angle of the fingertip and each angle is compared with threshold angle. The orientation an...

Technology is becoming more natural and intuitive. People already use gestures and speech to interact with their pes and other devices. In this paper, we have developed an application interface to recognize human body gestures and reflect these gestures through Kinect sensor to Lego robot. The main contribution of this paper is implementation process of the system and development of application interface. Visual Studio 2013 in c# will be used to control a Lego Robot gestures detected through API Kinect. At the end we will test the system and we will represent the results graphically.

IRJET, 2020

In this project focuses on developing a hand gesture-based quadcopter. Gesture recognition technology helps you to communicate or control any other devices. Here we use a Kinect motion sensor to control the quadcopter. The user can make these gestures in front of Microsoft Kinect devices, in the Kinect device, it has three motion cameras to track the user hand gesture.in this project, we piloting an Arduino-controlled quadcopter using a Microsoft Kinect device.

IFAC-PapersOnLine, 2019

A new approach to control robot manipulators using body gestures is presented. It implements the Microsoft Kinect sensor ability to track joint positions. The later has been used in the developed software application for gesture recognition, operating in Matlab programming environment, that maps the gestures into robot control commands. The goal is to control the movements of an industrial manipulator that shares its workspace with humans. Such a working environment is characterized with complexity and dynamism. In this way a more-effective interface with the controlled robot can been created. The experimental tests have been carried out with a Mitsubishi SCARA robot manipulator model RH-6CH7020-1D1-S15.

2015

We present a lightweight gesture recognizer utilizing Microsoft’s Kinect version 2 sensor. Our recognizer seems to be robust enough for many applications and does not require any training. Because this version of the Kinect sensor is equipped with a higher resolution than its predecessor depth camera, it can track some of the user’s fingers, and the Kinect SDK is able to provide state information of the user’s hands. Armed with this capability, we were able to design our gesture recognizer. New gestures can be specified programmatically at the moment, but we are also working on a graphical user interface (GUI) that would allow a user to define new gestures with it. We show how we built the recognizer and demonstrate its usage via two applications we designed. The first application is a simple picture manipulation application. For the second application, we designed a 3DOF robotic arm that can be controlled using

International Journal of Advanced Robotic Systems, 2015

In this paper, we present a gesture recognition system for the development of a human-robot interaction (HRI) interface. Kinect cameras and the OpenNI framework are used to obtain real-time tracking of a human skeleton. Ten different gestures, performed by different persons, are defined. Quaternions of joint angles are first used as robust and significant features. Next, neural network (NN) classifiers are trained to recognize the different gestures. This work deals with different challenging tasks, such as the real-time implementation of a gesture recognition system and the temporal resolution of gestures. The HRI interface developed in this work includes three Kinect cameras placed at different locations in an indoor environment and an autonomous mobile robot that can be remotely controlled by one operator standing in front of one of the Kinects. Moreover, the system is supplied with a people reidentification module which guarantees that only one person at a time has control of the robot. The system's performance is first validated offline, and then online experiments are carried out, proving the real-time operation of the system as required by a HRI interface.

International Journal of Advance Research in Computer Science and Management Studies [IJARCSMS] ijarcsms.com

For better interaction between Computer and Human we use gesture recognization (Microsoft's Kinect sensor). The Kinect is enlisted to construct skeletons in the 3D space using 20 body joints coordinates of an human. From this coordinates, 10 joints are required and 6 triangles have been contrived along with 6 respective centroids. The feature space correlate with the Euclidean distances between spine joint and the centroids for each frame. For ranking purpose, support vector machine is used using a kernel function. For several gestures it work effectively at the rate of 89.7%. we use RGB-D sensor for hand gesture recognition. To find the number of hands in the image the algorithm uses different method. Ten different static hand gestures are recognized, including all different composition of spread fingers. In addition six dynamic gestures are established by following the movements of an open hand. The main advantage of our approach is without the need of wearing any specific clothing or additional devices we can access freedom of the user's hands to be at any position of the image. Besides, the whole method can be executed without any initial training or calibration.Through the experiments we carried out among various users from different environments we find that the accuracy of the method which additionally can be run in real-time.

International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2022

In today's world, industry and daily routine tasks are increasingly attracted to and implemented through robotic automation. Well, this study proposes a method for replacing buttons and joysticks with a more novel technology, namely, commanding the entire Robotic Arm with the user's hand movement, motion, or gesture. The goal is to alter how people think about remote controllers for manually driven robotic arms. Pick and place robots are a type of technology used in the industrial industry to execute pick and place tasks. The system is constructed in such a way that it reduces human error and intervention, resulting in more exact work. The system is designed with a simple, flexible, and minimum control technique in mind. There are numerous domains in which human intervention is problematic, yet the process in question must be operated and regulated, which leads to the application of robots. According to the literature, pick and place robots are conceived and deployed in a variety of industries, including the bottle filling industry, surveillance to identify and eliminate bombs, and so on. The goal of the project is to create a gesture-controlled robot with a robotic arm that can perform any pick-and-place tasks. A radio frequency transmission is used to control the pick and place robot. For the gripper open and close, up and down, forward and backward, base clockwise and counterclockwise, the Robotic Hand has certain independent commands. Four Omni wheels support the robotic arm's displacement on the chassis. The implemented robotic arm has four degrees of freedom. Line follower, wall hugger, obstacle avoider, metal detector, and other capabilities can be added to this robot to increase its versatility.

International Journal of Modeling and Optimization