An overview of 3D object grasp synthesis algorithms

2021

In recent times, object detection and pose estimation have gained significant attention in the context of robotic vision applications. Both the identification of objects of interest as well as the estimation of their pose remain important capabilities in order for robots to provide effective assistance for numerous robotic applications ranging from household tasks to industrial manipulation. This problem is particularly challenging because of the heterogeneity of objects having different and potentially complex shapes, and the difficulties arising due to background clutter and partial occlusions between objects. As the main contribution of this work, we propose a system that performs real-time object detection and pose estimation, for the purpose of dynamic robot grasping. The robot has been pre-trained to perform a small set of canonical grasps from a few fixed poses for each object. When presented with an unknown object in an arbitrary pose, the proposed approach allows the robot ...

Journal of Intelligent and Robotic Systems, 2017

Robot grasp quality metrics are used to evaluate, compare and select robotic grasp configurations. Many of them have been proposed based on a diversity of underlying principles and to assess different aspects of the grasp configurations. As a consequence, some of them provide similar information but other can provide completely different assessments. Combinations of metrics have been proposed in order to provide global indexes, but these attempts have shown the difficulties of merging metrics with different numerical ranges and even physical units. All these studies have raised the need of a deeper knowledge in order to determine independent grasp quality metrics which enable a global assessment of a grasp, and a way to combine them. This paper presents an exhaustive study in order to provide numerical evidence for these issues. Ten quality metrics are used to evaluate a set of grasps planned by a simulator for 7 different robot hands over a set of 126 object models. Three statistical analysis, namely, variability, correlation and sensitivity, are performed over this extensive database. Results and graphs presented allow to set practical thresholds for each quality metric, select independent metrics, and determine the robustness of each metric,providing a reliability indicator under pose uncertainty. The results from this paper are

Robotics, 2021

The field of robotics has been rapidly developing in recent years, and the work related to training robotic agents with reinforcement learning has been a major focus of research. This survey reviews the application of reinforcement learning for pick-and-place operations, a task that a logistics robot can be trained to complete without support from a robotics engineer. To introduce this topic, we first review the fundamentals of reinforcement learning and various methods of policy optimization, such as value iteration and policy search. Next, factors which have an impact on the pick-and-place task, such as reward shaping, imitation learning, pose estimation, and simulation environment are examined. Following the review of the fundamentals and key factors for reinforcement learning, we present an extensive review of all methods implemented by researchers in the field to date. The strengths and weaknesses of each method from literature are discussed, and details about the contribution ...

Algorithmic Foundations of Robotics XIV, 2021

Fixtures for constraining the movement of parts have been extensively investigated in robotics, since they are essential for using robots in automated manufacturing. This paper deals with the design and optimized synthesis of a special type of fixtures, which we call snapping fixtures. Given a polyhedral workpiece P with n vertices and of constant genus, which we need to hold, a snapping fixture is a semi-rigid polyhedron G, made of a palm and several fingers, such that when P and G are well separated, we can push P toward G, slightly bending the fingers of G on the way (exploiting its mild flexibility), and obtain a configuration, where G is back in its original shape and P and G are inseparable as rigid bodies. We prove the minimal closure conditions under which such fixtures can hold parts, using Helly's theorem. We then introduce an algorithm running in O(n 3) time that produces a snapping fixture, minimizing the number of fingers and optimizing additional objectives, if a snapping fixture exists. We also provide an efficient and robust implementation of a simpler version of the algorithm, which produces the fixture model to be 3D printed and runs in O(n 4) time. We describe two applications with different optimization criteria: Fixtures to hold add-ons for drones, where we aim to make the fixture as lightweight as possible, and small-scale fixtures to hold precious stones in jewelry, where we aim to maximize the exposure of the stones, namely minimize the obscuring of the workpiece by the fixture.

Intelligent Service Robotics, 2021

Intelligent Object manipulation for grasping is a challenging problem for robots. Unlike robots, humans almost immediately know how to manipulate objects for grasping due to learning over the years. A grown woman can grasp objects more skilfully than a child because of learning skills developed over years, the absence of which in the present day robotic grasping compels it to perform well below the human object grasping benchmarks. In this paper we have taken up the challenge of developing learning based pose estimation by decomposing the problem into both position and orientation learning. More specifically, for grasp position estimation, we explore three different methods-a Genetic Algorithm (GA) based optimization method to minimize error between calculated image points and predicted end-effector (EE) position, a regression based method (RM) where collected data points of robot EE and image points have been regressed with a linear model, a PseudoInverse (PI) model which has been formulated in the form of a mapping matrix with robot EE position and image points for several observations. Further for grasp orientation learning, we develop a deep reinforcement learning (DRL) model which we name as Grasp Deep Q-Network (GDQN) and benchmarked our results with Modified VGG16 (MVGG16). Rigorous experimentations show that due to inherent capability of producing very high-quality solutions for optimization problems and search problems, GA based predictor performs much better than the other two models for position estimation. For orientation learning results indicate that off policy learning through GDQN outperforms MVGG16, since GDQN architecture is specially made suitable for the reinforcement learning. Based on our proposed architectures and algorithms, the robot is capable of grasping all rigid body objects having regular shapes.

2013

In this review, we explore how teleoperation could potentially be applied to the management of humanoid robots, with an emphasis on humanoid robots that are used in assistive roles including clinical therapies and telemedicine. Since there are very few examples of teleoperation of a full humanoid, the review emphasizes those teleoperation technologies that are potentially relevant to the teleoperation of humanoids. Teleoperation of humanoid robots faces many of the same practical challenges associated with (a) traditional teleoperation, including latency and telepresence, and (b) teleoperating manipulators and other robots with high degrees of freedom. Teleoperation systems for humanoid robots must also address unique challenges triggered by strong emotional and social responses to humanoid robotics-responses of both the operator and any humans who may interact with the humanoid. These challenges trigger new opportunities for redefining teleoperation to include scripting, programming by demonstration, speech production, and Wizard of Oz interaction. The challenges also provide opportunities to specialize traditional modes of interaction and unique opportunities to develop humanoid-specific forms of teleoperation such as controlling humanoids via exoskeleton-based or inertial sensors. We include in this review a survey of enabling technologies, a taxonomy of practical uses, and a list of emerging themes and approaches to teleoperating humanoid robots.

Journal of Intelligent Manufacturing, 2022

Robotic vision plays a key role for perceiving the environment in grasping applications. However, the conventional framed-based robotic vision, suffering from motion blur and low sampling rate, may not meet the automation needs of evolving industrial requirements. This paper, for the first time, proposes an event-based robotic grasping framework for multiple known and unknown objects in a cluttered scene. With advantages of microsecond-level sampling rate and no motion blur of event camera, the model-based and model-free approaches are developed for known and unknown objects’ grasping respectively. The event-based multi-view approach is used to localize the objects in the scene in the model-based approach, and then point cloud processing is utilized to cluster and register the objects. The proposed model-free approach, on the other hand, utilizes the developed event-based object segmentation, visual servoing and grasp planning to localize, align to, and grasp the targeting object. U...

Biological Cybernetics, 2014

We investigate the role of obstacle avoidance in visually guided reaching and grasping movements. We report on a human study in which subjects performed prehensile motion with obstacle avoidance where the position of the obstacle was systematically varied across trials. These experiments suggest that reaching with obstacle avoidance is organized in a sequential manner, where the obstacle acts as an intermediary target. Furthermore, we demonstrate that the notion of workspace travelled by the hand is embedded explicitly in a forward planning scheme, which is actively involved in detecting obstacles on the way when performing reaching. We find that the gaze proactively coordinates the pattern of eye-arm motion during obstacle avoidance. This study provides also a quantitative assessment of the coupling between the eye-arm-hand motion. We show that the coupling follows regular phase dependencies and is unaltered during obstacle avoidance. These observations provide a basis for the design of a computational model. Our controller extends the coupled dynamical systems framework and provides fast and synchronous control of the eyes, the arm and the hand within a single and compact framework, mimicking similar control system found in humans. We validate our model for visuomotor control of a humanoid robot.

Journal of Intelligent & Robotic Systems

This paper presents a comprehensive review of ground agricultural robotic systems and applications with special focus on harvesting that span research and commercial products and results, as well as their enabling technologies. The majority of literature concerns the development of crop detection, field navigation via vision and their related challenges. Health monitoring, yield estimation, water status inspection, seed planting and weed removal are frequently encountered tasks. Regarding robotic harvesting, apples, strawberries, tomatoes and sweet peppers are mainly the crops considered in publications, research projects and commercial products. The reported harvesting agricultural robotic solutions, typically consist of a mobile platform, a single robotic arm/manipulator and various navigation/vision systems. This paper reviews reported development of specific functionalities and hardware, typically required by an operating agricultural robot harvester; they include (a) vision sys...

Robotics

Vision-based pose detection and grasping complex-shaped and thin objects are challenging tasks. We propose an architecture that integrates the Generative Grasping Convolutional Neural Network (GG-CNN) with depth recognition to identify a suitable grasp pose. First, we construct a training dataset with data augmentation to train a GG-CNN with only RGB images. Then, we extract a segment of the tool using a color segmentation method and use it to calculate an average depth. Additionally, we apply and evaluate different encoder–decoder models with a GG-CNN structure using the Intersection Over Union (IOU). Finally, we validate the proposed architecture by performing real-world grasping and pick-and-place experiments. Our framework achieves a success rate of over 85.6% for picking and placing seen surgical tools and 90% for unseen surgical tools. We collected a dataset of surgical tools and validated their pick and place with different GG-CNN architectures. In the future, we aim to expan...

Machine Vision and Applications

Grasping objects intelligently is a challenging task even for humans and we spend a considerable amount of time during our childhood to learn how to grasp objects correctly. In the case of robots, we can not afford to spend that much time on making it to learn how to grasp objects effectively. Therefore, in the present research we propose an efficient learning architecture based on VQVAE so that robots can be taught with sufficient data corresponding to correct grasping. However, getting sufficient labelled data is extremely difficult in the robot grasping domain. To help solve this problem, a semi-supervised learning based