Academia.eduAcademia.edu

Outline

A comparison of SLAM algorithms based on a graph of relations

2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems

https://doi.org/10.1109/IROS.2009.5354691

Abstract

In this paper, we address the problem of creating an objective benchmark for comparing SLAM approaches. We propose a framework for analyzing the results of SLAM approaches based on a metric for measuring the error of the corrected trajectory. The metric uses only relative relations between poses and does not rely on a global reference frame. The idea is related to graph-based SLAM approaches in the sense that it considers the energy needed to deform the trajectory estimated by a SLAM approach to the ground truth trajectory. Our method enables us to compare SLAM approaches that use different estimation techniques or different sensor modalities since all computations are made based on the corrected trajectory of the robot. We provide sets of relative relations needed to compute our metric for an extensive set of datasets frequently used in the SLAM community. The relations have been obtained by manually matching laser-range observations. We believe that our benchmarking framework allows the user an easy analysis and objective comparisons between different SLAM approaches.

References (31)

  1. F. Amigoni, S. Gasparini, and M. Gini. Good experimental method- ologies for robotic mapping: A proposal. In Proc. of the IEEE Int. Conf. on Robotics & Automation (ICRA), 2007.
  2. B. Balaguer, S. Carpin, and S. Balakirsky. Towards quantitative com- parisons of robot algorithms: Experiences with SLAM in simulation and real world systems. In IROS 2007 Workshop, 2007.
  3. A. Bonarini, W. Burgard, G. Fontana, M. Matteucci, D. G. Sorrenti, and J. D. Tardos. Rawseeds a project on SLAM benchmarking. In Proc. of the IROS WS on Benchmarks in Robotics Research, 2006.
  4. A. Censi. The achievable accuracy for range finder localization. IEEE Transactions on Robotics. Under review.
  5. A. Censi. Scan matching in a probabilistic framework. In Proc. of the IEEE Int. Conf. on Robotics & Automation (ICRA), pages 2291-2296, 2006.
  6. EPFL and IROS. Cleaning Robot Contest, 2002. http://robotika.cz/competitions/cleaning2002/en.
  7. ESA. Lunar robotics challenge, 2008. http://www.esa.int/esaCP/SEM4GKRTKMF index 0.html.
  8. R. Eustice, H. Singh, and J.J. Leonard. Exactly sparse delayed-state filters. In Proc. of the IEEE Int. Conf. on Robotics & Automation (ICRA), pages 2428-2435, 2005.
  9. U. Frese. Dlr spatial cognition data set. http://www.informatik.uni- bremen.de/agebv/en/DlrSpatialCognitionDataSet, 2008.
  10. G. Grisetti, C. Stachniss, and W. Burgard. Improved techniques for grid mapping with rao-blackwellized particle filters. IEEE Transac- tions on Robotics, 23:34-46, 2007.
  11. G. Grisetti, C. Stachniss, S. Grzonka, and W. Burgard. A tree parameterization for efficiently computing maximum likelihood maps using gradient descent. In Proc. of Robotics: Science and Systems (RSS), 2007.
  12. J.-S. Gutmann and K. Konolige. Incremental mapping of large cyclic environments. In Proc. of the IEEE Int. Symposium on Computational Intelligence in Robotics and Automation (CIRA), 1999.
  13. J. Hermosillo, C. Pradalier, S. Sekhavat, C. Laugier, and G. Baille. Towards motion autonomy of a bi-steerable car: Experimental issues from map-building to trajectory execution. In Proc. of the IEEE Int. Conf. on Robotics & Automation (ICRA), 2003.
  14. A. Howard and N. Roy. Radish: The robotics data set repository, standard data sets for the robotics community, 2003. http://radish.sourceforge.net/.
  15. S. Julier, J. Uhlmann, and H. Durrant-Whyte. A new approach for filtering nonlinear systems. In Proc. of the American Control Conference, pages 1628-1632, 1995.
  16. R. Kümmerle, B. Steder, C. Dornhege, M. Ruhnke, G. Grisetti, C. Stachniss, and A. Kleiner. On measuring the accuracy of SLAM algorithms. Autonomous Robots, 2009. Condtionally accepted for publication.
  17. R. Kümmerle, B. Steder, C. Dornhege, M. Ruhnke, G. Grisetti, C. Stachniss, and A. Kleiner. Slam benchmarking webpage. http://ais.informatik.uni-freiburg.de/slamevaluation, 2009.
  18. J.J. Leonard and H.F. Durrant-Whyte. Mobile robot localization by tracking geometric beacons. IEEE Transactions on Robotics and Automation, 7(4):376-382, 1991.
  19. F. Lu and E. Milios. Globally consistent range scan alignment for environment mapping. Autonomous Robots, 4:333-349, 1997.
  20. M. Montemerlo, S. Thrun, D. Koller, and B. Wegbreit. FastSLAM 2.0: An improved particle filtering algorithm for simultaneous localization and mapping that provably converges. In Proc. of the Int. Conf. on Artificial Intelligence (IJCAI), pages 1151-1156, 2003.
  21. E. Olson. Robust and Efficient Robotic Mapping. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 2008.
  22. E. Olson, J. Leonard, and S. Teller. Fast iterative optimization of pose graphs with poor initial estimates. In Proc. of the IEEE Int. Conf. on Robotics & Automation (ICRA), pages 2262-2269, 2006.
  23. R. Smith, M. Self, and P. Cheeseman. Estimating uncertain spatial re- altionships in robotics. In I. Cox and G. Wilfong, editors, Autonomous Robot Vehicles, pages 167-193. Springer Verlag, 1990.
  24. C. Stachniss, U. Frese, and G. Grisetti. OpenSLAM.org -give your algorithm to the community. http://www.openslam.org, 2007.
  25. C. Stachniss, G. Grisetti, N. Roy, and W. Burgard. Evaluation of gaussian proposal distributions for mapping with rao-blackwellized particle filters. In Proc. of the Int. Conf. on Intelligent Robots and Systems (IROS), 2007.
  26. S. Thrun. An online mapping algorithm for teams of mobile robots. Int. Journal of Robotics Research, 20(5):335-363, 2001.
  27. S. Thrun and colleagues. Winning the darpa grand challenge. Journal on Field Robotics, 2006.
  28. S. Thrun, Y. Liu, D. Koller, A.Y. Ng, Z. Ghahramani, and H. Durrant- Whyte. Simultaneous localization and mapping with sparse extended information filters. Int. Journal of Robotics Research, 23(7/8):693- 716, 2004.
  29. O. Wulf, A. Nüchter, J. Hertzberg, and B. Wagner. Benchmarking urban six-degree-of-freedom simultaneous localization and mapping. Journal of Field Robotics, 25(3):148-163, 2008.
  30. M. Yguel, C.T.M. Keat, C. Braillon, C. Laugier, and O. Aycard. Dense mapping for range sensors: Efficient algorithms and sparse representations. In Proc. of Robotics: Science and Systems (RSS), 2007.
  31. Z. Zivkovic, O. Booij, B. Krose, E.A. Topp, and H.I. Christensen. From sensors to human spatial concepts: An annotated data set. IEEE Transactions on Robotics, 24(2):501-505, 2008.