Dynamic Movement Primitives

When describing robot motion with dynamic movement primitives (DMPs), goal (trajectory endpoint), shape and temporal scaling parameters are used. In reinforcement learning with DMPs, usually goals and temporal scaling parameters are predefined and only the weights for shaping a DMP are learned. Many tasks, however, exist where the best goal position is not a priori known, requiring to learn it. Thus, here we specifically address the question of how to simultaneously combine goal and shape parameter learning. This is a difficult problem because learning of both parameters could easily interfere in a destructive way. We apply value function approximation techniques for goal learning and direct policy search methods for shape learning. Specifically, we use ''policy improvement with path integrals'' and ''natural actor critic'' for the policy search. We solve a learning-to-pour-liquid task in simulations as well as using a Pa10 robot arm. Results for learning from scratch, learning initialized by human demonstration, as well as for modifying the tool for the learned DMPs are presented. We observe that the combination of goal and shape learning is stable and robust within large parameter regimes. Learning converges quickly even in the presence of disturbances, which makes this combined method suitable for robotic applications.

When describing robot motion with dynamic movement primitives (DMPs), goal (trajectory endpoint), shape and temporal scaling parameters are used. In reinforcement learning with DMPs, usually goals and temporal scaling parameters are predefined and only the weights for shaping a DMP are learned. Many tasks, however, exist where the best goal position is not a priori known, requiring to learn it. Thus, here we specifically address the question of how to simultaneously combine goal and shape parameter learning. This is a difficult problem because learning of both parameters could easily interfere in a destructive way. We apply value function approximation techniques for goal learning and direct policy search methods for shape learning. Specifically, we use ''policy improvement with path integrals'' and ''natural actor critic'' for the policy search. We solve a learning-to-pour-liquid task in simulations as well as using a Pa10 robot arm. Results for learning from scratch, learning initialized by human demonstration, as well as for modifying the tool for the learned DMPs are presented. We observe that the combination of goal and shape learning is stable and robust within large parameter regimes. Learning converges quickly even in the presence of disturbances, which makes this combined method suitable for robotic applications.

Forestry cranes are an important tool for safe and efficient timber harvesting with forestry machines. However, their complex manual control often led to inefficiencies and excessive energy usage, due to the many joysticks and buttons that must be used in a precise sequence to perform efficient movements. To address this, the industry is increasingly turning to partial automation, making manual control more intuitive for the operator and, consequently, achieving improvements in energy efficiency. This article introduces a novel approach to energy-optimal motion planning that can be used along with a feedback control system to automate crane motions, taking over portions of the operator's work. Our method combines dynamic movement primitives (DMPs) and an energy-optimization algorithm. DMPs is a machine learning technique for motion planning based on human demonstrations, while the optimization algorithm exploits the crane's redundancy to find energy-optimal trajectories. Simulation results show that DMPs can replicate human-like controlled motions with a 25% reduction in energy consumption. However, our energy optimization algorithm shows improvements of over 40%, providing substantial energy savings and a promising pathway towards environmentally friendly partially automated machines.

Teleoperating a robot for complex and intricate tasks demands a high mental workload from a human operator. Deploying multiple operators can mitigate this problem, but it can be also a costly solution. Learning from Demonstrations can reduce the human operator's burden by learning repetitive teleoperation tasks. Yet, the demonstrations via teleoperation tend to be inconsistent compared to other modalities of human demonstrations. In order to handle less consistent and asynchronous demonstrations effectively, this paper proposes a learning scheme based on Dynamic Movement Primitives. In particular, a new Expectation Maximization algorithm which synchronizes and encodes demonstrations with high temporal and spatial variances is proposed. Furthermore, we discuss two shared teleoperation architectures, where, instead of multiple human operators, a learned artificial agent and a human operator share authority over a task while teleoperating cooperatively. The agent controls the more mundane and repetitive motion in the task whereas human takes charge of the more critical and uncertain motion. The proposed algorithm together with the two shared teleoperation architectures (human-synchronized and agent-synchronized shared teleoperation) has been tested and validated through simulation and experiments on 3 Degrees-of-Freedom Phantom-to-Phantom teleoperation. Conclusively, the both proposed shared teleoperation architectures have shown superior performance when compared with the human-only teleoperation for a peg-in-hole task.

While teleoperation provides a possibility for a robot to operate at extreme conditions instead of a human, teleoperating a robot still demands a heavy mental workload from a human operator. Learning from demonstrations can reduce the human operator's burden by learning repetitive teleoperation tasks. However, one of challenging issues is that demonstrations via teleoperation are less consistent compared to other modalities of human demonstrations. In order to solve this problem, we propose a learning scheme based on Dynamic Movement Primitives (DMPs) which can handle less consistent, asynchronized and incomplete demonstrations. In particular we proposed a new Expectation Maximization (EM) algorithm which can synchronize and encode demonstrations with temporal and spatial variances, different initial and final conditions and partial executions. The proposed algorithm is tested and validated with three different experiments of a pegin-hole task conducted on 3-Degree of freedom (DOF) masterslave teleoperation system.

To achieve a certain task, a skilligent robot should be able to learn the skills embedded in that task. Furthermore, the robot should be able to infer such skills to handle uncertainties and perturbations, since most robot tasks are usually dailylife tasks that include many unexpected situations. Therefore, we propose a unified skill learning and inference framework. The framework includes six processing modules: 1) a human demonstration process, 2) an autonomous segmentation process, 3) a dynamic movement primitive learning process, 4) a Bayesian network learning process, 5) a motivation graph construction process, and 6) a skill-inferring process. Based on the framework, the robot learns and infers situation-adequate and goal-oriented skills to handle uncertainties and human perturbations. To show the validity of our framework, some experimental results are illustrated using a robot arm that performs a 'tea service' task.

In manipulation tasks, skills are usually modeled using the continuous motion trajectories acquired in the task space. The motion trajectories obtained from a human's multiple demonstrations can be broadly divided into four portions, according to the spatial variations between the demonstrations and the time spent in the demonstrations: the portions in which a long/short time is spent, and those in which the spatial variations are large/small. In these four portions, the portions in which a long time is spent and the spatial variation is small (e.g., passing a thread through the eye of a needle) are usually modeled using a small number of parameters, even if such portions represent the movement that is essential for achieving the task. The reason for this is that these portions are slightly changed in the task space as compared with the other portions. In fact, such portions should be densely modeled using more parameters (i.e., overfitting) to improve the performance of the skill because the movements of those portions must be accurately executed to achieve the task. In this paper, we propose a method for adaptively fitting these skills based on the temporal and the spatial entropies calculated by a Gaussian mixture model. We found that it is possible to retrieve accurate motion trajectories as compared with those of well-fitted models, whereas the estimation performance is generally higher than that of an overfitted model. To validate our proposed method, we present the experimental results and evaluations when using a robot arm that performed two tasks.

Ankle injuries are among the most common injuries in sport and daily life. However, for their recovery, it is important for patients to perform rehabilitation exercises. These exercises are usually done with a therapist’s guidance to help strengthen the patient’s ankle joint and restore its range of motion. However, in order to share the load with therapists so that they can offer assistance to more patients, and to provide an efficient and safe way for patients to perform ankle rehabilitation exercises, we propose a framework that integrates learning techniques with a 3-PRS parallel robot, acting together as an ankle rehabilitation device. In this paper, we propose to use passive rehabilitation exercises for dorsiflexion/plantar flexion and inversion/eversion ankle movements. The therapist is needed in the first stage to design the exercise with the patient by teaching the robot intuitively through learning from demonstration. We then propose a learning control scheme based on dyna...

Movement Primitives (MP) are a well-established approach for representing modular and re-usable robot movement generators. Many state-of-the-art robot learning successes are based MPs, due to their compact representation of the inherently continuous and high dimensional robot movements. A major goal in robot learning is to combine multiple MPs as building blocks in a modular control architecture to solve complex tasks. To this effect, a MP representation has to allow for blending between motions, adapting to altered task variables, and co-activating multiple MPs in parallel. We present a probabilistic formulation of the MP concept that maintains a distribution over trajectories. Our probabilistic approach allows for the derivation of new operations which are essential for implementing all aforementioned properties in one framework. In order to use such a trajectory distribution for robot movement control, we analytically derive a stochastic feedback controller which reproduces the given trajectory distribution. We evaluate and compare our approach to existing methods on several simulated as well as real robot scenarios.

While teleoperation provides a possibility for a robot to operate at extreme conditions instead of a human, teleoperating a robot still demands a heavy mental workload from a human operator. Learning from demonstrations can reduce the human operator's burden by learning repetitive teleoperation tasks. However, one of challenging issues is that demonstrations via teleoperation are less consistent compared to other modalities of human demonstrations. In order to solve this problem, we propose a learning scheme based on Dynamic Movement Primitives (DMPs) which can handle less consistent, asynchronized and incomplete demonstrations. In particular we proposed a new Expectation Maximization (EM) algorithm which can synchronize and encode demonstrations with temporal and spatial variances, different initial and final conditions and partial executions. The proposed algorithm is tested and validated with three different experiments of a pegin-hole task conducted on 3-Degree of freedom (DOF) masterslave teleoperation system.

Collaborative robots (cobots) built to work alongside humans must be able to quickly learn new skills and adapt to new task configurations. Learning from demonstration (LfD) enables cobots to learn and adapt motions to different use conditions. However, state-of-the-art LfD methods require manually tuning intrinsic parameters and have rarely been used in industrial contexts without experts. In this paper, the development and implementation of a LfD framework for industrial applications with naive users is presented. We propose a parameter-free method based on probabilistic movement primitives, where all the parameters are predetermined using Jensen-Shannon divergence and bayesian optimization; thus, users do not have to perform manual parameter tuning. This method learns motions from a small dataset of user demonstrations, and generalizes the motion to various scenarios and conditions. We evaluate the method extensively in two field tests: one where the cobot works on elevator door maintenance, and one where three Schindler workers teach the cobot tasks useful for their workflow. Errors between the cobot end-effector and target positions range from 0 to 1.48±0.35mm. For all tests, no task failures were reported. Questionnaires completed by the Schindler workers highlighted the method's ease of use, feeling of safety, and the accuracy of the reproduced motion. Our code and recorded trajectories are made available online for reproduction.

While teleoperation provides a possibility for a robot to operate at extreme conditions instead of a human, teleoperating a robot still demands a heavy mental workload from a human operator. Learning from demonstrations can reduce the human operator's burden by learning repetitive teleoperation tasks. However, one of challenging issues is that demonstrations via teleoperation are less consistent compared to other modalities of human demonstrations. In order to solve this problem, we propose a learning scheme based on Dynamic Movement Primitives (DMPs) which can handle less consistent, asynchronized and incomplete demonstrations. In particular we proposed a new Expectation Maximization (EM) algorithm which can synchronize and encode demonstrations with temporal and spatial variances, different initial and final conditions and partial executions. The proposed algorithm is tested and validated with three different experiments of a pegin-hole task conducted on 3-Degree of freedom (DOF) masterslave teleoperation system.

While teleoperation provides a possibility for a robot to operate at extreme conditions instead of a human, teleoperating a robot still demands a heavy mental workload from a human operator. Learning from demonstrations can reduce the human operator's burden by learning repetitive teleoperation tasks. However, one of challenging issues is that demonstrations via teleoperation are less consistent compared to other modalities of human demonstrations. In order to solve this problem, we propose a learning scheme based on Dynamic Movement Primitives (DMPs) which can handle less consistent, asynchronized and incomplete demonstrations. In particular we proposed a new Expectation Maximization (EM) algorithm which can synchronize and encode demonstrations with temporal and spatial variances, different initial and final conditions and partial executions. The proposed algorithm is tested and validated with three different experiments of a pegin-hole task conducted on 3-Degree of freedom (DOF) masterslave teleoperation system.

Probabilistic Movement Primitives (ProMPs) are a widely used representation of movements for human-robot interaction. They also facilitate the factorization of temporal and spatial structure of movements. In this work we investigate a method to temporally align observations so that when learning ProMPs, information in the spatial structure of the observed motion is maximized while maintaining a smooth phase velocity. We apply the method on recordings of hand trajectories in a two-dimensional reaching task. A system for simultaneous recognition of movement and phase is proposed and performance of movement recognition and movement reproduction is discussed.

In this paper, we propose and implement an advanced manipulation framework that enables parametric learning of complex action trajectories along with their haptic feedback profiles. Our framework extends Dynamic Movement Primitives (DMPs) method with a new parametric non-linear shaping function and a novel force-feedback coupling term. The non-linear trajectories of the action control variables and the haptic feedback trajectories measured during execution are encoded with parametric temporal probabilistic models, namely Parametric Hidden Markov Models (PHMMs). PHMMs enable autonomous segmentation of a taught skill based on the statistical information extracted from multiple demonstrations, and learning the relations between the model parameters and the properties extracted from the environment. Hidden states with highvariances in observation probabilities are interpreted as parts of the skill that could not be reliably learned and autonomously executed due to possibly uncertain or missing information about the environment. In those parts, our proposed force-feedback coupling term, which computes the deviation of the actual force feedback from the one predicted by the force-feedback PHMM, acts as a compliance term, enabling a human to scaffold the ongoing movement trajectory to accomplish the task. Our method is verified in a number of tasks including a real pick and place task that involves obstacles of different heights. Our robot, Baxter, successfully learned to generate the trajectory taking into the heights of the obstacles, move its end effector stiffly (and accurately) along the generated trajectory while passing through apertures, and allow human-robot collaboration in the autonomously detected segments of the motion, for example when the gripper picks up the object whose position is not provided to the robot.

Task-parameterized skill learning aims at adaptive motion encoding to new situations. While existing approaches for task parameterized skill learning have demonstrated good adaptation within the demonstrated region, the extrapolation problem of task parameterized skills has not been investigated enough. In this work, with the aim of good adaptation not only within the demonstrated region but also outside of the region, we propose to combine a generative model with a Dynamic Movement Primitive (DMP) by formulating learning as a density estimation problem. Moreover, for efficient learning from relatively few demonstrations, we propose to augment training data with additional incomplete data. The proposed method is tested and compared with existing works in simulations and real robot experiments. Experimental results verified its generalization in the extrapolation region.

— Since 1980, several programming by demonstration tools have been developed in order to automatically synthesize robot behaviors starting from a limited amount of examples. Most of these algorithms, even if compliant and adaptive to small changes in the environment, fail to create a human like behavior able to cope with different scenarios that address the same procedure. The present work reviews methodologies that learn, model and simulate human motion gestures from a given set of examples. A specific attention will be given to the ability of these algorithms to generate proper behaviors in the controlling an external exogenous process (the application task). The similarities and differences with existing tools at state of the art will be examined, then few new methodologies will be proposed and discussed. An application with balls juggling is used as a case study to analyze the capabilities of these tools to coordinate and learn appropriate models.

Dynamic Movement Primitives (DMPs) are widely used for encoding motion data. Task parameterized DMP (TP-DMP) can adapt a learned skill to different situations. Mostly a customized vision system is used to extract task specific variables. This limits the use of such systems to real world scenarios. This paper proposes a method for combining the DMP with a Convolutional Neural Network (CNN). Our approach preserves the generalization properties associated with a DMP, while the CNN learns the task specific features from the camera images. This eliminates the need to extract the task parameters, by directly utilizing the camera image during the motion reproduction. The performance of the developed approach is demonstrated through a trash cleaning task, executed with a real robot. We also show that by using the data augmentation, the learned sweeping skill can be generalized for arbitrary objects. The experiments show the robustness of our approach for several different settings.

Task-parameterized skill learning aims at adaptive motion encoding to new situations. While existing approaches for task parameterized skill learning have demonstrated good adaptation within the demonstrated region, the extrapolation problem of task parameterized skills has not been investigated enough. In this work, with the aim of good adaptation not only within the demonstrated region but also outside of the region, we propose to combine a generative model with a Dynamic Movement Primitive (DMP) by formulating learning as a density estimation problem. Moreover, for efficient learning from relatively few demonstrations, we propose to augment training data with additional incomplete data. The proposed method is tested and compared with existing works in simulations and real robot experiments. Experimental results verified its generalization in the extrapolation region.

Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is precoded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset.

A novel approach ensuing Gaussian Mixture Model is proposed in this research to segregate the input image into assorted strokes. Once the strokes are extracted, they are combined to reproduce the same character using Gaussian Mixture Regression. The resulted learned strokes are then implemented on KUKA Light Weight Robot and the learned skill is further improved using Reinforcement Learning.

Abstract—We present a probabilistic approach to learn robust models of human motion through imitation. The association of Hidden Markov Model (HMM), Gaussian Mixture Regression (GMR) and dynamical systems allows us to extract redundancies across multiple demonstrations and build time-independent models to reproduce the dynamics of the demonstrated movements.