Academia.eduAcademia.edu

Outline

The limitations of genetic algorithms in software testing

2010, ACS/IEEE International Conference on Computer Systems and Applications - AICCSA 2010

https://doi.org/10.1109/AICCSA.2010.5586984

Abstract

Software test-data generation is the process of identifying a set of data, which satisfies a given testing criterion. For solving this difficult problem there were a lot of research works, which have been done in the past. The most commonly encountered are random test-data generation, symbolic test-data generation, dynamic test-data generation, and recently, test-data generation based on genetic algorithms. This paper gives a survey of the majority of software test-data generation techniques based on genetic algorithms. It compares and classifies the surveyed techniques according to the genetic algorithms features and parameters. Also, this paper shows and classifies the limitations of these techniques.

References (41)

  1. Frankl P G, Weiss S N. An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Transactions on Software Engineering, 19(8), 774-787, 1993.
  2. Mills H D, Dyer M D, Linger R C. Cleanroom Software Engineering. IEEE Software 4(5), 19-25, September 1987.
  3. Voas J M, Morell L, Miller K W. Predicting where faults can hide from testing. IEEE, 8(2), 41-48, March 1991.
  4. Thévenod-Fosse P, Waeselynck H. STATEMATE: Applied to Statistical Software Testing. ACM SIGSOFT. Proceedings of the 1993 International Symposium on Software Testing and Analysis, Software Engineering Notes 23(2), pp. 78-81, June 1993.
  5. Boyer R S, Elspas B, Levitt K N. SELECT -a formal system for testing and debugging programs by symbolic execution. Proceedings of the International Conference on Reliable software, 234-245 (1975).
  6. Clarke L A. A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineering, 2(3), 215-222, 1976.
  7. King J C. Symbolic execution and program testing. Communications of the ACM, 19 (7), 385-394, 1976.
  8. Howden W E. Symbolic testing and the DISSECT Symbolic evaluation system. IEEE Transactions on Software Engineering, 3(4), 266-278, 1977.
  9. Hedley D, Hennell M A. The causes and effects of infeasible paths in computer programs. Proceedings of Eighth International Conference on Software Engineering, IEEE Computer Society, 259-266, 1985.
  10. Lindquist T E, Jenkins J R. Test-case generation with IOGen. IEEE Software, 5 (1), 72-79 (1988).
  11. Girgis M R. An experimental evaluation of a symbolic execution system. Software. Engineering Journal, 7(4), 285-290, 1992.
  12. Girgis M R. Using symbolic execution and data flow criteria to aid test data selection. The Journal of Software Testing, Verification and Reliability, 3(2), 101-112, 1993.
  13. Korel B. Automated Software Test Data Generation. IEEE Transactions on Software Engineering, 16(8): 870-879, August 1990.
  14. Ferguson R, Korel B. The Chaining Approach for Software Test Data Generation. ACM TOSEM, vol. 5, no. 1, pages 63-86, January 1996.
  15. McMinn P. Search-based Software Test Data Generation: A Survey. Journal of Software Testing Verification and Reliability, vol.14, no.2, pp.105-156, June 2004.
  16. Mantere T, Alander J T. Evolutionary Software Engineering, A Review. Journal of Applied Soft Computing, vol.5, pp.315-331, 2005.
  17. Xanthakis S, Ellis C, Skourlas C, Le Gall A, Kastiskas S, Karapoulios K. Application of genetic algorithms to software testing (Application des algorithms génétiques au test des logiciels). In 5th International Conference on Software Engineering and its Applications, pages 625- 636, Toulouse, France, 1992.
  18. Pei M,.
  19. Goodman E D, Gao Z, Zhong K. Automated Software Test Data Generation Using A Genetic Algorithm. Technical Report GARAGe of Michigan State University June 1994.
  20. Watkins A. A tool for the automatic generation of test data using genetic algorithms. In Proceedings of the fourth Software Quality Conference, Dundee, Great Britain, pp. 300-309, 1995.
  21. Roper M, Maclean I, Brooks A, Miller J, Wood M. Genetic Algorithms and the Automatic Generation of Test Data. Technical report RR/95/195[EFoCS-19-95], Department of Computer Science, University of Strathclyde, 1995.
  22. Jones B F, Sthamer H H, Eyres D E. Automatic Structural Testing Using Genetic Algorithms. Software Engineering Research Journal, pp. 299- 306, September 1996.
  23. Sthamer H. H., the automatic generation of software test data using genetic algorithms, Ph.D. Thesis, University of Glamorgan, Pontyprid, Wales, Great Britain, 1996.
  24. Weichselbaum R., software test automation by means of genetic algorithms, Proceedings of the Sixth International conference on Software testing, Analysis and review (EuroSTAR 98), Munich Germany (1998).
  25. Pargas R P, Harrold M J, Peck R R. Test Data Generation Using Genetic Algorithms. Journal of Software Testing, Verifications, and Reliability, vol. 9, pp. 263-282, September 1999.
  26. Tracey N., A Search-Based Automated Test Data Generation Framework for Safety Critical Software. Ph. D. thesis, University of York, 2000.
  27. Bueno P M S, Jino M. Identification of Potentially Infeasible Program Paths by Monitoring the Search for Test Data. Proceedings of the Fifteenth IEEE International Conference on Automated Software Engineering (ASE'00).
  28. Bueno P M S, Jino M. Automatic Test Data Generation for Program Paths Using Genetic Algorithms. International Journal of Software Engineering and Knowledge Engineering, vol. 12, no. 6, pp. 691-709, 2002.
  29. Wegener J., Baresel A., Sthamer H.. Evolutionary test environment for automatic structural testing. Journal of Information and Software Technology, vol. 43, pp. 841-854, 2001.
  30. J. Wegener, K. Buhr, and Pohlheim H.. Automatic test data generation for structural testing of embedded software systems by evolutionary testing. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2002), pp. 1233-1240, New York, UAS, 2002.
  31. Lin J, Yeh P. Automatic Test Data Generation for Path Testing Using GAs. Journal of Information Science vol. 131, pp. 47-64, 2001.
  32. Michael C C, McGraw G E, Schatz M A. Generating Software Test Data by Evolution. IEEE Transactions on Software Engineering, vol.27, no.12, pp. 1085-1110, December 2001.
  33. Berndt D, Fisher J, Johnson L , Pinglikar J, Watkins A. Breeding Software Test Cases with Genetic Algorithms, Proceedings of 36th Hawaii International Conference On System Sciences 2003.
  34. Baresel A, Sthamer H, Schmidt M. Fitness Function Design to Improve Evolutionary Structural Testing. In proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2002), pages 1329- 1336, New York, USA, 2002.
  35. Bottaci L. Instrumenting programs with flag variables for test data search by genetic algorithm. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2002), pages 1337- 1342, New York, USA, 2002. Morgan Kaufmann.
  36. Harman M, Hu L, Hierons R, Baresel A, Sthamer H. Improving evolutionary testing by flag removal. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2002), pages 1359- 1366, New York, USA, 2002. Morgan Kaufmann.
  37. Baresel A, Sthamer H. Evolutionary testing of flag conditions. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2003), Lecture Notes in Computer Science vol. 2724, pages 2442-2454, Chicago, USA, 2003. Springer-Verlag.
  38. Harman M, Hu L, Zhang X, and Munro M. Side-effect removal transformation. In Proceedings of the 9th IEEE International Workshop on Program Comprehension (IWPC2001), pages 310-319, Toronto, Canada, 2001. IEEE Computer Society Press.
  39. Harman M, Hu L, Zhang X, Munro M, Dolado J J, Otero M C, Wegener J. A post-placement side-effect removal algorithm. In Proceedings of the 18th IEEE International Conference on Software Maintenance (ICSM 2002), pp 2-11, Montreal, Canada, 2002.
  40. Baresel A, Pohlheim H, and Sadeghipour S. Structural and functional sequence test of dynamic and state-based software with evolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2003), Lecture Notes in Computer Science vol. 2724, pages 2428 -2441, Chicago, USA, 2003. Springer- Verlag.
  41. McMinn P, Holcombe M. The state problem for evolutionary testing. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2003), Lecture Notes in Computer Science vol. 2724, pages 2488-2497, Chicago, USA, 2003. Springer-Verlag.