3D Grasp Synthesis Based on Object Exploration

2010

In this paper a method to achieve an human-like grasp in unstructured environments is presented. The algorithm is composed of an object surface reconstruction algorithm and a local grasp planner, evolving in parallel. The former uses an elastic elliptical reconstruction surface, with axes and dimensions assigned by a preshaping process, that is let to evolve dynamically under the action of reconstruction forces. The reconstruction surface shrinks toward the object until some parts of the surface intercept the object visual hull. The latter moves the fingertips on the current available reconstruction surface towards points which are optimal (in a local sense) with respect to a certain number of indices weighting both the grasp quality and the kinematic configuration of the hand. A control module must ensure that the references given by the planner are correctly followed by the robotic hand. Experiments are presented, showing the effectiveness of the proposed algorithm.

2020

Innate morphological characteristics of objects may obfuscate the learning of robotic grasping. Even simple structures (e.g. cyclical) offer a wide range of plausible grasping orientations, creating ambiguities for neural regressors. We investigate and unfold multiple such conflicts on the challenging dataset Jacquard and derive a novel grasp map representation, suitable for pixel-wise synthesis. Our augmented maps disentangle cooccurent grasping orientations around the same point by partitioning the angle space into multiple bins. Subsequently, we propose the ORientation AtteNtive Grasp synthEsis (ORANGE) framework, that jointly addresses classification into bins and angle-value regression. The constructed bin-wise orientation maps further serve as an attention mechanism for areas with higher graspness, i.e. probability of being a true grasp point. This procedure is model-agnostic and can be embedded to any existing architecture to boost its performance. Namely, we report a new sta...

Computer Vision …, 2009

We describe a system for autonomous learning of visual object representations and their grasp affordances on a robot-vision system. It segments objects by grasping and moving 3D scene features, and creates probabilistic visual representations for object detection, recognition and pose estimation, which are then augmented by continuous characterizations of grasp affordances generated through biased, random exploration. Thus, based on a careful balance of generic prior knowledge encoded in (1) the embodiment of the system, (2) a vision system extracting structurally rich information from stereo image sequences as well as (3) a number of built-in behavioral modules on the one hand, and autonomous exploration on the other hand, the system is able to generate object and grasping knowledge through interaction with its environment.

arXiv (Cornell University), 2020

Inherent morphological characteristics in objects may offer a wide range of plausible grasping orientations that obfuscates the visual learning of robotic grasping. Existing grasp generation approaches are cursed to construct discontinuous grasp maps by aggregating annotations for drastically different orientations per grasping point. Moreover, current methods generate grasp candidates across a single direction in the robot's viewpoint, ignoring its feasibility constraints. In this paper, we propose a novel augmented grasp map representation, suitable for pixel-wise synthesis, that locally disentangles grasping orientations by partitioning the angle space into multiple bins. Furthermore, we introduce the ORientation AtteNtive Grasp synthEsis (ORANGE) framework, that jointly addresses classification into orientation bins and anglevalue regression. The bin-wise orientation maps further serve as an attention mechanism for areas with higher graspability, i.e. probability of being an actual grasp point. We report new state-of-the-art 94.71% performance on Jacquard, with a simple U-Net using only depth images, outperforming even multi-modal approaches. Subsequent qualitative results with a real bi-manual robot validate ORANGE's effectiveness in generating grasps for multiple orientations, hence allowing planning grasps that are feasible. Code is available at https: //github.com/nickgkan/orange. This project received funding from the RoboTrust and the Skills4Robots projects. Experimental computations have been conducted on an Nvidia DGX-1 at TU Darmstadt.

Intelligent Robots and …, 2007

A grasp is the beginning of any manipulation task. Therefore, an autonomous robot should be able to grasp objects it sees for the first time. It must hold objects appropriately in order to successfully perform the task. This paper considers the problem of grasping unknown objects in the same manner as humans. Based on the idea that the human brain represents objects as volumetric primitives in order to recognize them, the presented algorithm predicts grasp as a function of the object's parts assembly. Beginning with a complete 3D model of the object, a segmentation step decomposes it into single parts. Each single part is fitted with a simple geometric model. A learning step is finally needed in order to find the object component that humans choose to grasp it.

Robotics: Science and Systems XIV, 2018

This paper presents a real-time, object-independent grasp synthesis method which can be used for closed-loop grasping. Our proposed Generative Grasping Convolutional Neural Network (GG-CNN) predicts the quality and pose of grasps at every pixel. This one-to-one mapping from a depth image overcomes limitations of current deep-learning grasping techniques by avoiding discrete sampling of grasp candidates and long computation times. Additionally, our GG-CNN is orders of magnitude smaller while detecting stable grasps with equivalent performance to current state-of-the-art techniques. The lightweight and single-pass generative nature of our GG-CNN allows for closed-loop control at up to 50Hz, enabling accurate grasping in non-static environments where objects move and in the presence of robot control inaccuracies. In our real-world tests, we achieve an 83% grasp success rate on a set of previously unseen objects with adversarial geometry and 88% on a set of household objects that are moved during the grasp attempt. We also achieve 81% accuracy when grasping in dynamic clutter.

arXiv (Cornell University), 2022

This paper describes a method for generating robot grasps by jointly considering stability and other task and object-specific constraints. We introduce a three-level representation that is acquired for each object class from a small number of exemplars of objects, tasks, and relevant grasps. The representation encodes task-specific knowledge for each object class as a relationship between a keypoint skeleton and suitable grasp points that is preserved despite intra-class variations in scale and orientation. The learned models are queried at run time by a simple sampling-based method to guide the generation of grasps that balance task and stability constraints. We ground and evaluate our method in the context of a Franka Emika Panda robot assisting a human in picking tabletop objects for which the robot does not have prior CAD models. Experimental results demonstrate that in comparison with a baseline method that only focuses on stability, our method is able to provide suitable grasps for different tasks.

2009

A distinct property of robot vision systems is that they are embodied. Visual information is extracted for the purpose of moving in and interacting with the environment. Thus, different types of perception-action cycles need to be implemented and evaluated.

2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010

We study two important problems in the area of robot grasping: i) the methodology and representations for grasp selection on known and unknown objects, and ii) learning from experience for grasping of similar objects. The core part of the paper is the study of different representations necessary for implementing grasping tasks on objects of different complexity. We show how to select a grasp satisfying force-closure, taking into account the parameters of the robot hand and collisionfree paths. Our implementation takes also into account efficient computation at different levels of the system regarding representation, description and grasp hypotheses generation.

Robotics and Autonomous Systems, 2014

Humans excel when dealing with everyday manipulation tasks, being able to learn new skills, and to adapt to different complex environments. This results from a lifelong learning, and also observation of other skilled humans. To obtain similar dexterity with robotic hands, cognitive capacity is needed to deal with uncertainty. By extracting relevant multisensor information from the environment (objects), knowledge from previous grasping tasks can be generalized to be applied within different contexts. Based on this strategy, we show in this paper that learning from human experiences is a way to accomplish our goal of robot grasp synthesis for unknown objects. In this article we address an artificial system that relies on knowledge from previous human object grasping demonstrations. A learning process is adopted to quantify probabilistic distributions and uncertainty. These distributions are combined with preliminary knowledge towards inference of proper grasps given a point cloud of an unknown object. In this article, we designed a method that comprises a twofold process: object decomposition and grasp synthesis. The decomposition of objects into primitives is used, across which similarities between past observations and new unknown objects can be made. The grasps are associated with the defined object primitives, so that feasible object regions for grasping can be determined. The hand pose relative to the object is computed for the pre-grasp and the selected grasp. We have validated our approach on a real robotic platform-a dexterous robotic hand. Results show that the segmentation of the object into primitives allows to identify the most suitable regions for grasping based on previous learning. The proposed approach provides suitable grasps, better than more time consuming analytical and geometrical approaches, contributing for autonomous grasping.