Grasp success prediction with quality metrics

Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453), 2003

This paper deals with visually guided grasping of unmodeled objects for robots which exhibit an adaptive behavior based on their previous experiences. Nine features are proposed to characterize three-finger grasps. They are computed from the object image and the kinematics of the hand. Real experiments on a humanoid robot with a Barrett hand are carried out to provide experimental data. This data is employed by a classification strategy, based on the k-nearest neighbour estimation rule, to predict the reliability of a grasp configuration in terms of five different performance classes. Prediction results suggest the methodology is adequate.

2007

In critical human/robotic interactions such as, e.g., teleoperation by a disabled master or with insufficient bandwidth, or intelligent prostheses, it is highly desirable to have semi-autonomous robotic artifacts interact with a human being. Semi-autonomous teleoperation, for instance, consists in having a smart slave able to guess the master's intentions and possibly take over control in order to perform the desired actions in a more skillful/timely way than with plain, pointto-point teleoperation. In this paper we investigate the possibility of building such an intelligent robotic artifact by training a machine learning system on data gathered from several human subjects while trying to grasp objects in a teleoperation setup. The idea is to have the slave "guess" when the master wants to grasp an object with the maximum possible accuracy; at the same time, the system must be light enough to be usable in an on-line scenario and flexible enough to adapt to different masters, e.g., elderly and/or slow. The outcome of the experiment is that such a system, based upon Support Vector Machines, meets all the requirements, being (a) highly accurate, (b) compact and fast, and (c) largely unaffected by the subjects' diversity. The system is, moreover, trained by something like 3.5 minutes of human data in the worst case.

The International Journal of Robotics Research, 2014

Grasping an object is usually only an intermediate goal for a robotic manipulator. To finish the task, the robot needs to know where the object is in its hand and what action to execute. This paper presents a general statistical framework to address these problems. Given a novel object, the robot learns a statistical model of grasp state conditioned on sensor values. The robot also builds a statistical model of the requirements for a successful execution of the task in terms of uncertainty in the state of the grasp. Both of these models are constructed by offline experiments. The online process then grasps objects and chooses actions to maximize likelihood of success. This paper describes the framework in detail, and demonstrates its effectiveness experimentally in placing, dropping, and insertion tasks. To construct statistical models, the robot performed over 8,000 grasp trials, and over 1,000 trials each of placing, dropping, and insertion.

2018

In this work we present an empirical approach for solving the grasp synthesis problem for anthropomorphic robots equipped with vacuum grippers. Our approach exploits a self-supervised, data-driven learning approach to estimate a suitable grasp for known and unknown objects. We employ a Convolutional Neural Network (CNN) that directly infers the grasping points and the approach angles from RGB-D images as a regression problem. In particular, we split the image into a cell grid where the CNN provides, for each cell, an estimate of a grasp along with a confidence score. We collected a training dataset composed of 4000 grasping attempts by means of an automatic trial-and-error procedure, and we trained end-to-end the CNN directly on both the grasping successes and failures. We report a set of preliminary experiments performed by using known (i.e., object included in the training dataset) and unknown objects, showing that our system is able to effectively learn good grasping configurations.

2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013

Handling objects with a single hand without dropping the object is challenging for a robot. A possible way to aid the motion planning is the prediction of the sensory results of different motions. Sequences of different movements can be performed as an offline simulation, and using the predicted sensory results, it can be evaluated whether the desired goal is achieved. In particular, the task in this paper is to roll a sphere between the fingertips of the dexterous hand of the humanoid robot TWENDY-ONE. First, a forward model for the prediction of the touch state resulting from the in-hand manipulation is developed. As it is difficult to create such a model analytically, the model is obtained through machine learning. To get real world training data, a dataglove is used to control the robot in a master-slave way. The learned model was able to accurately predict the course of the touch state while performing successful and unsuccessful in-hand manipulations. In a second step, it is shown that this simulated sequence of sensor states can be used as input for a stability assessment model. This model can accurately predict whether a grasp is stable or whether it results in dropping the object. In a final step, a more powerful grasp stability evaluator is introduced, which works for our task regardless of the sphere diameter. After grasping an object, often regrasping is necessary before the actual task can be performed, for example when grasping a pen for writing. Regrasping with one hand and without additional support is challenging: in order to achieve robust in-hand manipulation, the current touch state has to be taken into account, but modeling of the contact state is difficult. In particular, it is challenging to design an analytical

Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference, 2020

Robotic grasping of unknown objects in cluttered scenes is already well established, mainly based on advances in Deep Learning methods. A major drawback is the need for a big amount of real-world training data. Furthermore these networks are not interpretable in a sense that it is not clear why certain grasp attempts fail. To make the process of robotic grasping traceable and simplify the overall model we suggest to divide the complex task of robotic grasping into three simpler tasks to find stable grasp points. The first task is to find all grasp points where the gripper can be lowered onto the table without colliding with the object. The second task is to determine for the grasp points and gripper parameters from the first step how the object moves while the gripper is closed. Finally in the third step for all grasp points from the second step it is predicted whether the object slips out of the gripper during lifting. By this simplification it is possible to understand for each grasp point why it is stable and-just as important-why others are unstable or not feasible. In this study we focus on the second task, the prediction of the physical interaction between gripper and object while the gripper is closed. We investigate different Convolutional Neural Network (CNN) architectures and identify the architecture(s) that predict the physical interactions in image space best. We perform the experiments for training data generation in the robot and physics simulator V-REP.

2017 IEEE International Conference on Robotics and Automation (ICRA)

Grasping tasks have always been challenging for robots, despite recent innovations in vision-based algorithms and object-specific training. If robots are to match human abilities and learn to pick up never-before-seen objects, they must combine vision with tactile sensing. This paper present a novel way to improve robotic grasping: by using tactile sensors and an unsupervised feature-learning approach, a robot can find the common denominators behind successful and failed grasps, and use this knowledge to predict whether a grasp attempt will succeed or fail. This method is promising as it uses only high-level features from two tactile sensors to evaluate grasp quality, and works for the training set as well as for new objects. In total, using a total of 54 different objects, our system recognized grasp failure 83.70% of time.

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019

Grasping is a fundamental robotic task needed for the deployment of household robots or furthering warehouse automation. However, few approaches are able to perform grasp detection in real time (frame rate). To this effect, we present Grasp Quality Spatial Transformer Network (GQ-STN), a oneshot grasp detection network. Being based on the Spatial Transformer Network (STN), it produces not only a grasp configuration, but also directly outputs a depth image centered at this configuration. By connecting our architecture to an externally-trained grasp robustness evaluation network, we can train efficiently to satisfy a robustness metric via the backpropagation of the gradient emanating from the evaluation network. This removes the difficulty of training detection networks on sparsely annotated databases, a common issue in grasping. We further propose to use this robustness classifier to compare approaches, being more reliable than the traditional rectangle metric. Our GQ-STN is able to detect robust grasps on the depth images of the Dex-Net 2.0 dataset with 92.4 % accuracy in a single pass of the network. We finally demonstrate in a physical benchmark that our method can propose robust grasps more often than previous sampling-based methods, while being more than 60 times faster.

Systems, Man, and …, 2005

This paper presents and analyses twelve quality measures that characterize robotic grips according to their stability and reliability. The measures are designed to assess three-finger grips of 2D parts performed in a real environment, taking into account both theoretical aspects and unavoidable uncertainties of a grasping action. They build on the existing literature and on physical and mechanical considerations. The measures constitute a feature space that pattern recognition methods can use in order to classify robotic grips according to their quality. Six of the measures depend on the actual finger configuration of the gripper, and they have shown to be critical for a better characterization. The kinematics of the Barrett Hand has been used. As a validation step, the measures are merged in two global quality values (with different practical applicability) that can be used to rank feasible candidate grips.

IEEE Transactions on Automation Science and Engineering