Multi-Contact Grasp Interaction for Virtual Environments

2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 2019

This paper presents a simple strategy to allow the rotation of unknown objects using a robotic hand equipped with tactile sensors. The tactile and kinematic information, obtained while that the object is manipulated, is used to determine the hand configurations that change the object position. The approach was successfully tested using an anthropomorphic robotic hand (Allegro Hand), with the fingertips modified to include tactile sensors (WTS-FT). Using three fingers of the hand, everyday objects were successfully rotated without using their model for the determination of the hand movements. Index Terms-Model-free manipulation, tactile sensing, multifingered hand.

Manipulation of objects in 3D space is not always a trivial task. When dealing with touchscreens, finding the correct set of mappings and interactions in three dimensions becomes even more of an issue. Current techniques evaluate 3D interaction regardless of physical properties of our world, aiming for task optimization instead of realism. The purpose of this paper is to evaluate three different types of 3D object manipulation techniques using multi-touch surfaces when the environment performs with a realistic physics behavior. The task used in the interface evaluation consists of organized objects in a box in a pre-specified way. Our results show that users tend to leverage from the physical properties of the environment to help them manipulate objects. Hence, the interface design that best approximates real-world manipulation performed best.

2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2018

Figure 1: Demonstration of our physics-based hand interaction method for VR environments. Our approach allows countless complex interactions such as handling a box in one hand and opening it with the other (top-left), or smashing a wall with an axe held in one hand (top-right). We can even spin an object between two fingers from two different hands (bottom-left) or from the same hand (bottom-right) in a very realistic manner. The red arrows indicate the forces calculated with our method and applied to the objects. Yellow arrows depict the global hand motion depicted for demonstration purposes only.

Proc. Computer …, 1998

In this paper, we propose a multi-sensor based method of automatic grasping motion control for multiple synthetic actors. Despite the fact that it is described and implemented in our specific model, the method is general and can be applicable to other models. A heuristic method is defined to decide the different grasping strategies from object geometry, hand geometry and observation of real grasping. Inverse kinematics can derive the final posture of the arms in order to bring the hands around the object. Multi-sensor object detection decides the finger contact points on the object and determine their position and orientation. Then, a group of polynomials derived from Euler-Lagrange equation is used to interpolate between the initial and final arm postures resulting in a more realistic real-time motion than linear interpolation. We also present 3D interactive grasping examples involving multiple synthetic actors.

IEEE Robotics and Automation Letters

We present a new gripper and exploration approach that uses a finger with very low reflected inertia for probing and then grasping objects. The finger employs a transparent transmission, resulting in a light touch when contact occurs. The finger elements are stiff and mounted on precise Cartesian axes for accurate proprioceptive sensing. Experiments show that the finger can safely move faster into contacts than industrial parallel jaw grippers or even most force-controlled grippers with backdrivable transmissions. This property allows rapid proprioceptive probing of objects. Contact information is leveraged to execute grasping actions with a contact-first strategy and to reduce environment state uncertainty. We evaluate a particle filtering algorithm that inputs contact information from either proprioception, or a combination of tactile sensing and proprioception, to estimate object location. Both methods can estimate location within 2 mm; combined tactile sensing and proprioception requires fewer observations.

2009

This demonstration introduces a new 3D interaction technique called "3-Hand Manipulation", for multiuser collaborative manipulation of 3D objects. This technique relies on the use of three manipulation points that can be used simultaneously by three different "hands" of two or three users. The three translation motions of the manipulation points fully determine the resulting 6 DOF motion of an object. We describe the graphical representations of the 3-Hand Manipulation and an illustration of its use by two or three users on an insertion task.

2009

In this paper we introduce a new 3D interaction technique called "3-Hand Manipulation", for multiuser collaborative manipulation of 3D objects. The 3-Hand Manipulation relies on the use of three manipulation points that can be used simultaneously by three different "hands" of two or three users. Interestingly, the three translation motions of the manipulation points can fully determine the resulting 6 degrees of freedom motion of the manipulated object. We describe the implementation of the 3-Hand Manipulation, its graphical representations and an illustration of its use by two or three users on an insertion task in immersive virtual environment. This technique could be used in various applications of collaborative VR such as for virtual prototyping, training simulations, assembly and maintenance simulations.

2013

Manipulation in immersive Virtual Environments (VEs) is often difficult and inaccurate because humans have difficulty in performing precise positioning tasks or in keeping the hand motionless in a particular position without any help of external devices or haptic feedback. To address this problem, we propose a set of four manipulation points attached to objects (called a 3-Point++ tool, including three handle points and their barycenter), by which users can control and adjust the position of objects precisely. By determining the relative position between the 3-Point++ tool and the objects, and by defining different states of each manipulation point (called locked/unlocked or inactive/active), these points can be freely configured to be adaptable and flexible to enable users to manipulate objects of varying sizes in many kinds of positioning scenarios.

In this paper a new method for fast visual grasp of unknown objects is presented. The method is composed of an object surface reconstruction algorithm and of a local grasp planner, evolving in a parallel way. The reconstruction algorithm makes use of images taken by a camera carried by the robot, mounted in an eye-in-hand configuration.An elastic reconstruction sphere, composed by masses interconnected each other by springs, is virtually placed around the object. The sphere is let to evolve dynamically under the action of external forces, which push the masses towards the object centroid. To smoothen the surface evolution, spatial dampers are attached to each mass. The reconstruction surface shrinks toward its center of mass until some pieces of its surface intercept the object visual hull, and thus local rejection forces are generated to push out the reconstruction points until they stay into the visual hull. This process shapes the sphere around the unknown object. Running in parallel to the reconstruction algorithm, the grasp planner moves the fingertips, floating on the current available reconstructed surface, according to suitable quality measures. The fingers keep moving towards local minima depending on the evolution of the reconstruction surface deformation. The process stops when the object has been completely reconstructed and the planner reaches a local minimum. Quality measures considering both hand and grasp proprieties are adopted. Simulations are presented, showing the effectiveness of the proposed algorithm.

9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005., 2005

Realistic simulation of grasping requires accurate modeling of forces and torques on the virtual object resulting from the fingers in contact. We present isometric 3By6 Finger Device for multi-fingered grasping in virtual environment (VE). The finger device was designed to measure forces of three fingers. Model of grasping adopted from the analysis of multifingered robot hands was used. The virtual object corresponded dynamically to the forces and torques applied by the three fingers. The multi-fingered grasping is demonstrated in four tasks aimed at the rehabilitation of the upper extremities of stroke patients. The tasks include opening of a safe, filling and pouring water from a glass, training of muscle strength with an elastic torus and force-tracking task.