Academia.eduAcademia.edu

Figure 12 – uploaded by Affan Pervez

See full PDFdownloadDownload figure
Fig. 7: Learned regression surfaces for y-axis (a) with GMR!° without DAEM, (b) with LWR?S, (c) with GPR?* and (d with GPR?S.

Figure 7 Learned regression surfaces for y-axis (a) with GMR!° without DAEM, (b) with LWR?S, (c) with GPR?* and (d with GPR?S.

Related Figures (25)

Fig. 1: This figure shows the illustration of a sweeping task. The position of the trash (colored circles) can be considered as the task parameter, governing variations in the demonstra- tions. For a new trash position (blue circle), which is away from the demonstrated region, the robot should be able to generate trajectory for moving trash to the collection point.
Fig. 1: This figure shows the illustration of a sweeping task. The position of the trash (colored circles) can be considered as the task parameter, governing variations in the demonstra- tions. For a new trash position (blue circle), which is away from the demonstrated region, the robot should be able to generate trajectory for moving trash to the collection point.
Fig. 2: TP-DMP learning using mixture of GMMs. (a) The mixture of GMMs, (b) the underlying regression surface and (c) the intuitive reasoning for such a response.
Fig. 2: TP-DMP learning using mixture of GMMs. (a) The mixture of GMMs, (b) the underlying regression surface and (c) the intuitive reasoning for such a response.
ere the subscript o and m denote the indexes of demon- trations and components respectively while the terms and F in superscript denote the dimensions corre- ponding to s and F respectively. Since the task param- ters remain constant during a demonstration, we can simply concatenate their values in the learned means of the GMM components. The diagonal values corre- sponding to the task parameters in the covariances are not learned and are set to small value ¢ OM w® ND
ere the subscript o and m denote the indexes of demon- trations and components respectively while the terms and F in superscript denote the dimensions corre- ponding to s and F respectively. Since the task param- ters remain constant during a demonstration, we can simply concatenate their values in the learned means of the GMM components. The diagonal values corre- sponding to the task parameters in the covariances are not learned and are set to small value ¢ OM w® ND
where b;,, represents the responsibility that the o'’ GMM takes for explaining the i*” data point while di,oym Ye@p- resents the responsibility that the m‘” component o the o” GMM takes for explaining the i*” data point. It is acommon practice to apply regularization on learned covariances for avoiding singularities during EM. As a regularization measure, if any of the eigenvalues of the learned covariances becomes lower than a predefined threshold ¢, then we reset it to «. We apply the de- scribed approach to the mixture of GMMs learned for encoding the forcing terms of a DMP, with variations along a scalar task parameter. The learned mixture of GMMs, which is also the estimated density function, is shown in Figure 2a. The GMMs remain unchanged. The regression for the mixture of GMMs is calculated as:
where b;,, represents the responsibility that the o'’ GMM takes for explaining the i*” data point while di,oym Ye@p- resents the responsibility that the m‘” component o the o” GMM takes for explaining the i*” data point. It is acommon practice to apply regularization on learned covariances for avoiding singularities during EM. As a regularization measure, if any of the eigenvalues of the learned covariances becomes lower than a predefined threshold ¢, then we reset it to «. We apply the de- scribed approach to the mixture of GMMs learned for encoding the forcing terms of a DMP, with variations along a scalar task parameter. The learned mixture of GMMs, which is also the estimated density function, is shown in Figure 2a. The GMMs remain unchanged. The regression for the mixture of GMMs is calculated as:
For evaluation, we generate linearly spaced samples of JT and s within the demonstrated ranges and use GMR to predict the value of F for each sample, i.e. {s x T} 4 F. The surface plot of this data can be visualized in Figure 2b. We can see that it has a step along the task parameter. This shows that as we have sparse data in the task space (only two trajectories), each GMM in the mixture kept concentrated at regions of demonstrations. Due to data sparsity, the density es- timate is overfitted in its current form. The reason for the step in surface plot can be explained by Figure 2c. If we have well separated Gaussians (Figure 2c-top) then their activation functions, calculated as the responsi- bility term in EM, switches in a very narrow region (Figure 2c-bottom). The same phenomenon occurred in the regression surface where regions close to a GMM are mostly influenced by it. As we move away, the ac- tivation transits sharply to a nearby GMM. This also means that this model is overfitted and can only be use- ful for exact reproduction of task parameter values in training dataset and it fails to generalize for novel task
For evaluation, we generate linearly spaced samples of JT and s within the demonstrated ranges and use GMR to predict the value of F for each sample, i.e. {s x T} 4 F. The surface plot of this data can be visualized in Figure 2b. We can see that it has a step along the task parameter. This shows that as we have sparse data in the task space (only two trajectories), each GMM in the mixture kept concentrated at regions of demonstrations. Due to data sparsity, the density es- timate is overfitted in its current form. The reason for the step in surface plot can be explained by Figure 2c. If we have well separated Gaussians (Figure 2c-top) then their activation functions, calculated as the responsi- bility term in EM, switches in a very narrow region (Figure 2c-bottom). The same phenomenon occurred in the regression surface where regions close to a GMM are mostly influenced by it. As we move away, the ac- tivation transits sharply to a nearby GMM. This also means that this model is overfitted and can only be use- ful for exact reproduction of task parameter values in training dataset and it fails to generalize for novel task
Fig. 3: Result of manually setting the variance along task variable. (a) The mixture of GMMs, (b) the mixture of GMMs after manually setting the variance along task variable and (c) the underlying regression surface.
Fig. 3: Result of manually setting the variance along task variable. (a) The mixture of GMMs, (b) the mixture of GMMs after manually setting the variance along task variable and (c) the underlying regression surface.
As the output dimension is not considered for now, we term the input data distribution as Incomplete Data GMM (IDGMM). Fig. 5: If we have data points whose certain dimensions are missing then they can still be used when applying EM for fitting a GMM. For instance, the output value of the data plotted on the input axis is missing. For this incomplete data, the expected values of the missing output values are also es- timated within EM.
As the output dimension is not considered for now, we term the input data distribution as Incomplete Data GMM (IDGMM). Fig. 5: If we have data points whose certain dimensions are missing then they can still be used when applying EM for fitting a GMM. For instance, the output value of the data plotted on the input axis is missing. For this incomplete data, the expected values of the missing output values are also es- timated within EM.
In the M-step, the value of d‘t! calculated only 1,0,m on the observable dimensions can be directly used for updating got.
In the M-step, the value of d‘t! calculated only 1,0,m on the observable dimensions can be directly used for updating got.
One may wonder about the result of fitting a single GMM instead of using the mixture of GMMs. As the data is concentrated at discrete regions of input space (task parameters), the components of a single GMM will also get attracted to same regions as by the mixture of GMMs. An appropriate regularization term must be used for a single GMM, to avoid singularity issues of co- variance matrices, as the data concentrated at discrete regions in higher dimensional space. The reason for us- ing the mixture of GMMs instead of a single GMM is that now we optimize the weight @ of each GMM sep- arately, without disturbing the weights a within each GMM, as they are important for preserving the local behavior of each GMM. This also results in a smaller number of parameters update during the second EM cycle, in contrast to updating the mixing weights 7 of a single GMM. 5 Related Work Learning TP-DMP using mixture of GMMs
One may wonder about the result of fitting a single GMM instead of using the mixture of GMMs. As the data is concentrated at discrete regions of input space (task parameters), the components of a single GMM will also get attracted to same regions as by the mixture of GMMs. An appropriate regularization term must be used for a single GMM, to avoid singularity issues of co- variance matrices, as the data concentrated at discrete regions in higher dimensional space. The reason for us- ing the mixture of GMMs instead of a single GMM is that now we optimize the weight @ of each GMM sep- arately, without disturbing the weights a within each GMM, as they are important for preserving the local behavior of each GMM. This also results in a smaller number of parameters update during the second EM cycle, in contrast to updating the mixing weights 7 of a single GMM. 5 Related Work Learning TP-DMP using mixture of GMMs
Table 2: Task errors (simulation) with different approaches. Fig. 6: A step by step illustration of TP-DMP learning with additional incomplete data and DAEM. (a) Demonstrations (b) Initial GMMs encoding forcing terms of DMPs for y-axis, (c) transformed GMMs using incomplete data and DAEM, (d learned regression surfaces for y-axis, (e) multiple generated movements.
Table 2: Task errors (simulation) with different approaches. Fig. 6: A step by step illustration of TP-DMP learning with additional incomplete data and DAEM. (a) Demonstrations (b) Initial GMMs encoding forcing terms of DMPs for y-axis, (c) transformed GMMs using incomplete data and DAEM, (d learned regression surfaces for y-axis, (e) multiple generated movements.
Fig. 8: Motion reproductions (a) with GMR1° without DAEM, (b) with LWR?5, (c) with GPR!?%, (d) with GPR?° an (ce) with TP-GMM.
Fig. 8: Motion reproductions (a) with GMR1° without DAEM, (b) with LWR?5, (c) with GPR!?%, (d) with GPR?° an (ce) with TP-GMM.
Fig. 9: Motion reproductions for changing the height during execution (a) with GMR/15, (b) with LWR?5, (c) with GPR!5 (d) with GPR2S and (e) with TP-GMM. Table 3: Task errors (simulation) of different approaches when the height is changed during the trajectories
Fig. 9: Motion reproductions for changing the height during execution (a) with GMR/15, (b) with LWR?5, (c) with GPR!5 (d) with GPR2S and (e) with TP-GMM. Table 3: Task errors (simulation) of different approaches when the height is changed during the trajectories
Fig. 10: Schematic of our vision system.
Fig. 10: Schematic of our vision system.
Fig. 11: Demonstrations are collected by setting the robot in gravity compensation mode while a Cartesian impedance controller is used for motion generation. (a) A human provides the demonstration for moving the trash to the dustpan. (b) The robot generates motion for a new trash position.
Fig. 11: Demonstrations are collected by setting the robot in gravity compensation mode while a Cartesian impedance controller is used for motion generation. (a) A human provides the demonstration for moving the trash to the dustpan. (b) The robot generates motion for a new trash position.
Fig. 12: Comparison of GMR!5, LWR?°, GPR!5, GPR?° and TP-GMM for the sweeping task. (a) Demonstrations for the sweeping task. Circles represent the trash positions while the rectangle represents the bounding box enclosing these trash positions. (b) Blue dots represent trash positions for interpolation evaluation while the red ones are for extrapolation evaluations. (c,d,e,f) Generated movements for new trash positions.
Fig. 12: Comparison of GMR!5, LWR?°, GPR!5, GPR?° and TP-GMM for the sweeping task. (a) Demonstrations for the sweeping task. Circles represent the trash positions while the rectangle represents the bounding box enclosing these trash positions. (b) Blue dots represent trash positions for interpolation evaluation while the red ones are for extrapolation evaluations. (c,d,e,f) Generated movements for new trash positions.
Table 4: Task errors (KUKA) with different approaches. strations are provided for the trash positions in the upper half of the sweeping area, which makes it an interesting region for comparing different approaches. GMR"** also successfully generates the motion for the trash position in this region, as shown in Figure 12f. strations are provided for the trash positions in the tional task parameter, cannot be known in advance. As there is no distinction inbetween input and output vari- ables when fitting a GMM and during GMR, any set of variables can be selected as input, to retrieve the ex- pected value of remaining variables. Thus with GMR, the observable task variable can be used for predict- ing the expected value of missing task variables and for motion generation. So now, with GMR, we not only predict the goal terms g of each DMP but also generate the forcing terms. For GMR!* the components in each GMM (ce = 107+), as well as in the IDGMM, are set to 60. The number of basis functions in LWR?* and GPR?” is set to 100. The goal positions for LWR?° and GPR?* are predicted in the first step along with the DMP parameters by using LWR and GPR respec- tively. The goal position in GPR’ is predicted at each time step along with the forcing terms by using GPR. Two frames of reference are defined for TP-GMM: one at the start of the trajectory and one at the end of the trajectory. As mentioned before, the final joint configu- rations are not known and hence the TP-GMM cannot be used directly in this experiment. To use TP-GMM, we first predict the final joint configuration with GP and then the second frame of reference is placed at that final joint configuration for motion reproduction i.e. the offset vector for second frame of reference looks like bg = [ty Jf Jf2 Tf3 Jf4 Jf5 Jf6 dp7l where ty is the final time value and jfn is the predicted final joint angle for the n‘” joint. The transformation matrices for the two frames of references are set to identity matri- ces. The number of components in the TP-GMM is empirically set to 4. The remaining settings are same as in the previous experiments. Comparison: A binary evaluation criterion is de- Fe ee er ee hy ee, ee oe ee, 7 Cn) oc Se) ee 2; eee: renee
Table 4: Task errors (KUKA) with different approaches. strations are provided for the trash positions in the upper half of the sweeping area, which makes it an interesting region for comparing different approaches. GMR"** also successfully generates the motion for the trash position in this region, as shown in Figure 12f. strations are provided for the trash positions in the tional task parameter, cannot be known in advance. As there is no distinction inbetween input and output vari- ables when fitting a GMM and during GMR, any set of variables can be selected as input, to retrieve the ex- pected value of remaining variables. Thus with GMR, the observable task variable can be used for predict- ing the expected value of missing task variables and for motion generation. So now, with GMR, we not only predict the goal terms g of each DMP but also generate the forcing terms. For GMR!* the components in each GMM (ce = 107+), as well as in the IDGMM, are set to 60. The number of basis functions in LWR?* and GPR?” is set to 100. The goal positions for LWR?° and GPR?* are predicted in the first step along with the DMP parameters by using LWR and GPR respec- tively. The goal position in GPR’ is predicted at each time step along with the forcing terms by using GPR. Two frames of reference are defined for TP-GMM: one at the start of the trajectory and one at the end of the trajectory. As mentioned before, the final joint configu- rations are not known and hence the TP-GMM cannot be used directly in this experiment. To use TP-GMM, we first predict the final joint configuration with GP and then the second frame of reference is placed at that final joint configuration for motion reproduction i.e. the offset vector for second frame of reference looks like bg = [ty Jf Jf2 Tf3 Jf4 Jf5 Jf6 dp7l where ty is the final time value and jfn is the predicted final joint angle for the n‘” joint. The transformation matrices for the two frames of references are set to identity matri- ces. The number of components in the TP-GMM is empirically set to 4. The remaining settings are same as in the previous experiments. Comparison: A binary evaluation criterion is de- Fe ee er ee hy ee, ee oe ee, 7 Cn) oc Se) ee 2; eee: renee
reproduce the task as they are similar to the demonstra- tions. The LWR?» also encountered the same problem during the first experiment where successful trajectories were generated for interpolation intervals but it failed to reproduce during the extrapolation intervals. Figure 15 contains the reproduction results with GPR'. It can easily be observed that the initial end-effector trajec- tory required to approach the ball (and critical for the execution of the task) is very similar for the three re- productions. The trajectories lose the important shape information required for the successful execution of the task. Also the green and brown trajectories of one of the joints are almost overlapping. Since the reproduc- tion is in joint space, an incorrect motion of even a sin- gle joint can lead to the failure of the task. Figure 16 extrapolation performance is evaluated for the target x-coordinates of —0.4146m and —0.7933m. When gen- eralizing for novel goal positions, the DMPs can pro- duce high accelerations at the beginning of the move- ment [15]. This is due to the initial interaction inbe- tween the linear dynamics and forcing terms. This un- desirable behavior was slightly observed in this exper- iment. A simple solution to solve this problem is to gradually activate the forcing terms. Thus we multiply the predicted forcing terms by (1—s1°). As s decays ex- ponentially, the effect of this term vanishes very quickly.
reproduce the task as they are similar to the demonstra- tions. The LWR?» also encountered the same problem during the first experiment where successful trajectories were generated for interpolation intervals but it failed to reproduce during the extrapolation intervals. Figure 15 contains the reproduction results with GPR'. It can easily be observed that the initial end-effector trajec- tory required to approach the ball (and critical for the execution of the task) is very similar for the three re- productions. The trajectories lose the important shape information required for the successful execution of the task. Also the green and brown trajectories of one of the joints are almost overlapping. Since the reproduc- tion is in joint space, an incorrect motion of even a sin- gle joint can lead to the failure of the task. Figure 16 extrapolation performance is evaluated for the target x-coordinates of —0.4146m and —0.7933m. When gen- eralizing for novel goal positions, the DMPs can pro- duce high accelerations at the beginning of the move- ment [15]. This is due to the initial interaction inbe- tween the linear dynamics and forcing terms. This un- desirable behavior was slightly observed in this exper- iment. A simple solution to solve this problem is to gradually activate the forcing terms. Thus we multiply the predicted forcing terms by (1—s1°). As s decays ex- ponentially, the effect of this term vanishes very quickly.
Fig. 15: Reproductions with GPR!*
Fig. 15: Reproductions with GPR!*
TP-GMM suffers from a severe curse of dimensional- ity. contains the reproduction results with GPR*”. Some of the reproduced trajectories are very different from the demonstrated ones. As with GPR'*, the trajectories generated with GPR?” lose the initial shape informa- tion which is required for the successful execution of the task. The failure of both of the GP based approaches can be attributed the small amount of training data, i.e. only two trajectories. Figure 17 contains the repro- duction results with TP-GMM. The reproduced tra- jectories do not capture the demonstrated motions and it fails to learn anything useful for this task. The joint distribution of all the eight variables (one phase and seven joint angles) is encoded in the TP-GMM. As we only have two trajectories, the product of GMMs in
TP-GMM suffers from a severe curse of dimensional- ity. contains the reproduction results with GPR*”. Some of the reproduced trajectories are very different from the demonstrated ones. As with GPR'*, the trajectories generated with GPR?” lose the initial shape informa- tion which is required for the successful execution of the task. The failure of both of the GP based approaches can be attributed the small amount of training data, i.e. only two trajectories. Figure 17 contains the repro- duction results with TP-GMM. The reproduced tra- jectories do not capture the demonstrated motions and it fails to learn anything useful for this task. The joint distribution of all the eight variables (one phase and seven joint angles) is encoded in the TP-GMM. As we only have two trajectories, the product of GMMs in
(b) End-effector trajectory in Cartesian space. the different final joint configurations of the reproduced motions are also observed. We retain the local behavior of each GMM by keep- ing the means and mixing weights within the GMMs fixed. The task specific generalization is achieved by just adapting covariances and mixing coefficient of the already learned GMMs. The TP-DMP framework can perform learning in task space as well as in joint space and can even handle the learning of meta parameters of a DMP. As shown in the experiments, our approach requires very few demonstrations for learning and it outperforms the existing approaches specially when ex- trapolating beyond the demonstrated ranges of the task variables. As future work, we plan to extend our pro- posed work with sample reuse approach [8] and to in- vestigate the scalability issue to complex tasks.
(b) End-effector trajectory in Cartesian space. the different final joint configurations of the reproduced motions are also observed. We retain the local behavior of each GMM by keep- ing the means and mixing weights within the GMMs fixed. The task specific generalization is achieved by just adapting covariances and mixing coefficient of the already learned GMMs. The TP-DMP framework can perform learning in task space as well as in joint space and can even handle the learning of meta parameters of a DMP. As shown in the experiments, our approach requires very few demonstrations for learning and it outperforms the existing approaches specially when ex- trapolating beyond the demonstrated ranges of the task variables. As future work, we plan to extend our pro- posed work with sample reuse approach [8] and to in- vestigate the scalability issue to complex tasks.
Fig. 19: Executed motions for the two extrapolation evaluations. (a,c) Robot initial configuration and different target positions. (b) Executed striking motion for hitting the target in (a). (d) Executed striking motion for hitting the target in (c).
Fig. 19: Executed motions for the two extrapolation evaluations. (a,c) Robot initial configuration and different target positions. (b) Executed striking motion for hitting the target in (a). (d) Executed striking motion for hitting the target in (c).
Related topics:
RoboticsMachine LearningProgramming by DemonstrationDynamic Movement Primitives
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved