Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Subject Programs and Test Cases
Experimental Procedure
Results and Analysis
Conclusions and Future Work
References

An experimental comparison of four test suite reduction techniques

Profile image of Hao ZhongHao Zhong

2006

https://doi.org/10.1145/1134380
visibility

…

description

4 pages

link

1 file

Abstract

As a test suite usually contains redundancy, a subset of the test suite (representative set) may still satisfy all the test objectives. As the redundancy increases the cost of executing the test suite, many test suite reduction techniques have been brought out in spite of the NP-completeness of the general problem of finding the optimal representative set of the test suite. In the literature, some experimental studies of test suite reduction techniques have already been reported, but there are still shortcomings of the studies of these techniques. This paper presents an experimental comparison of the four typical test suite reduction techniques: heuristic H, heuristic GRE, genetic algorithm-based approach and ILP-based approach. The aim of the study is to provide a guideline for choosing the appropriate test suite reduction techniques.

Related papers

AN ANALYSIS OF TEST SUITE MINIMIZATION TECHNIQUES
IJESRT Journal

In software development, testing is widely used by developers to reveal faults that cause failures and improve the quality of the software by omitting or removing the detected faults. Test suites for testing and validating software programs can contain large numbers of test cases to execute various parts of the software program code to check for defects and code compliance. Test suite sizes may grow significantly with subsequent modifications to the software over time that causes redundancy in test suites. Due to time and resource constraints retesting the modified software every time, it is advantageous to develop techniques that manage test suite sizes by periodically removing duplicate test cases, in turn, can improve the maintenance efforts required for regression testing. The primary objective of test suite reduction is to effectively improve the testing process while maintaining Fault Detection Effectiveness and fulfilling all the testing requirements as well. Researchers have proposed some TSR heuristics based on different variations of a greedy algorithm. This paper presents a comparative study of test suite reduction techniques based on some parameters.

View PDFchevron_right
A Sound Assessment of Test Suite Minimization Techniques
Fayaz Ahmad Khan

Asian Journal of Computer Science and Technology, 2018

During software development, testing and re-testing occurs frequently to ensure that the software is working correctly before and after modifications. To carry out an effective testing process a test suite is created and executed to detect the faults in the existing code as well as in the modified code. The manual approach of test suite creation and execution is time consuming and labour intensive task as compared to automatically generated test data or test suite. The automatic test data generation is supposed to be an effective way, but a lot of redundant test cases are generated that increase the time, effort and cost of testing. Therefore, test suite minimization techniques are used to further minimize or reduce the number of test cases by selecting a subset from an initially random and large test suite to test the code before as well as after modification. In this study, a comprehensive analysis of the different test suite minimization techniques is presented in order to extend the existing studies and to propose new ideas in this direction.

View PDFchevron_right
Test Suite Reduction Using HGS Based Heuristic Approach
Khanna Nehemiah

Comput. Informatics, 2015

Regression testing is performed throughout the software lifecycle to uncover the faults as early as possible and to ensure that changes do not have any adverse effect in the software that is operational. Test suites once developed are reused and updated frequently. As the software evolves, test cases in the test suite may become redundant. The reason behind this is that the requirements covered by newly added test cases may also be covered by the existing test cases. This redundant nature of test suite increases the cost of executing the same. Further, resource and time constraints impose the necessity to develop techniques to minimize test suites by removing redundant test cases. Few heuristic approaches have been used to solve the test suite minimization problem. Even though solutions exist, still the redundancy of test case remains. In order to solve this problem, this paper proposes two Harrold-Gupta-Soffa (HGS) based heuristic algorithms namely, Non Redundant HGS and Enhanced H...

View PDFchevron_right
On the optimization approach towards test suite minimization
saeed parsa

Regression testing is a critical activity which occurs during the maintenance stage of the software lifecycle. However, it requires large amounts of test cases to assure the attainment of a certain degree of quality. As a result, test suite sizes may grow significantly. To address this issue, Test Suite Reduction techniques have been proposed. However, suite size reduction may lead to significant loss of fault detection efficacy. To deal with this problem, a greedy algorithm is presented in this paper. This algorithm attempts to select a test case which satisfies the maximum number of testing requirements while having minimum overlap in requirements coverage with other test cases. In order to evaluate the proposed algorithm, experiments have been conducted on the Siemens suite and the Space program. The results demonstrate the effectiveness of the proposed algorithm by retaining the fault detection capability of the suites while achieving significant suite size reduction.

View PDFchevron_right
IJISET-Software Test Case Reduction using Genetic Algorithm A Modified Approach.pdf
Gaurav Gupta

Software test cases play an important role in Software Testing. A lot of test cases are developed while designing test cases that are of no use or redundant in nature. These types of test cases increase the effort and cost involved in testing and results in decrease of software testing efficiency. In this paper, a Genetic Algorithm based methodology has been proposed which can significantly reduce the test suite and leads to optimization of the efficiency of software testing. The testing efficiency can be increased by identifying the most critical paths. The proposed Genetic Algorithm based approach select the software path clusters that are weighted in accordance with the criticality of the path. Proposed algorithm has been evaluated by applying simple Genetic Algorithm and Fitness Scaling on a case study. GA with Fitness scaling helps in improving the efficiency of software testing by effectively reducing software test cases.

View PDFchevron_right
A Survey on Test Case Reduction Techniques
Yogita Chauhan

2014

Test case reduction is traditional in nature. It is very old field on which engineers work from the beginning of the third generation. Reduction is not an easy task, several techniques in this field is developed. There are several challenges faced by the engineers, for instance, cost and time. The reduction is the nightmare of the developer, as, the level of testing is increases, techniques of reduction also becomes more complex. An old technique teaches us the basic of the reduction of test cases without any complexity and through simple algorithms. So, this paper gives the overview of the existing techniques and gives the literature review of test case reduction techniques in software testing field.

View PDFchevron_right
Importance of an Effective Test Suite Minimization Technique in Software Testing
Fayaz Ahmad Khan

International Journal of Recent Technology and Engineering (IJRTE), 2019

Software testing is an important activity for developing a quality end product. But it is expensive as it requires both time and cost to perform it during development phase and after its delivery. Therefore, it is required to develop some techniques in this direction in order to make software testing a desirable as well as an affordable activity. Hence, in this study we have proposed an effective method that minimizes the size and redundancy from an automated random generated test suite using data clustering technique. The proposed technique compared to other existing techniques achieves a substantial amount of reduction in the size of the test suite without affecting the fault detection capability of the test suite.

View PDFchevron_right
Cost-Aware Test Suite Minimization Approach Using TBAT Optimization Algorithm for Software Testing
Eswara Reddy

Traditional way of optimizing regression testing cost is to reduce subsets of test cases from a test suite without compromising the test requirement. In order to reduce the test suite, researchers have presented various test-suite reduction techniques using coverage metrics and greedy search algorithms. Besides greedy algorithms, optimization-based algorithms have played a major role in test suite reduction. Accordingly, we developed a new optimization algorithm called, TBAT algorithm to handle the diversity problem in generating new solutions while finding the optimal test cases. Here, a fitness function is developed to select the test cases optimally through the TBAT algorithm using two constraints, satisfying the entire test requirement and minimizing the cost measure. The proposed TBAT algorithm is experimented with five programs from SIR using four different evaluation metrics. The empirical study on the performance of the TBAT algorithm is analyzed with various parameters and ...

View PDFchevron_right
Test Suite Minimization
Javairia Khan

Programming Testing is a basic piece of programming improvement. Programming testing and retesting happens ceaselessly amid the product advancement life cycle. As programming develops and advances, new experiments are produced furthermore, added to a test suite to practice the most recent changes to the product. More than a few forms of the advancement of the product, some experiments in the test suite may wind up noticeably excess as for the testing prerequisites for which they were produced since these prerequisites are currently likewise fulfilled by other experiments in the suite that were added to cover changes in the later forms of programming. Because of time and asset imperatives for retesting the product each time it is adjusted, it is critical to create systems that keep test suite sizes reasonable by occasionally expelling repetitive experiments. This procedure is called test suite minimization. Test suite diminishment (TSR) is to discover a subset of the test suite containing an insignificant number of experiments that can fulfill all test prerequisites. Test suite decrease systems endeavor to evacuate excess experiments Programming testing and retesting happens ceaselessly amid the delicate product improvement lifecycle to recognize blunders as ahead of schedule as could reasonably be expected and to guarantee that progressions to existing programming don't break the delicate product. Test suites once created are reused and refreshed every now and again as the product develops. Therefore, some experiments in the test suite may end up noticeably excess as the product is adjusted after some time since the necessities secured by them are additionally secured by other experiments. Because of the asset and time limitations for re-executing substantial test suites, it is imperative to create methods to limit accessible test suites by evacuating repetitive experiments. By and large, the test suite minimization issue is NP finished. The fundamental goal of test suite minimization, purported test suite improvement or test suite diminished, is to reduce the quantity of experiments inside a test suite while keeping up

View PDFchevron_right
Reducing the size of test suite using a variant of Non-dominated Sorting Genetic Algorithm II
International Journal of Advance Research in Computer Science and Management Studies [IJARCSMS] ijarcsms.com

and resources spent on producing software. Therefore, it would be useful to have ways to reduce the cost of software testing.

View PDFchevron_right

References (13)

  1. REFERENCES
  2. J. Black, E Melachrinoudis, and D. Kaeli. "Bi-Criteria Models for All-Uses Test Suite Reduction," International Conference on Software Engineering, 2004, pp. 106-115.
  3. T.Y. Chen and M.F. Lau, "A New Heuristic for Test Suite Reduction," Information and Software Technology, Vol. 40, No. 5, 1998, pp. 347-354.
  4. T.Y. Chen and M.F. Lau, "A Simulation Study on Some Heuristics for Test Suite Reduction," Information and Software Technology, Vol. 40, No. 13, 1998, pp. 777- 787.
  5. M.J. Harrold, R. Gupta, and M.L. Soffa. "A Methodology for Controlling the Size of a Test Suite," ACM Transactions on Software Engineering and Methodology, Vol. 2, No.3, 1993, pp. 270-285.
  6. M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, "Experiments on the Effectiveness of Dataflow-and Control Flow-Based Test Adequacy Criteria," International Conference on Software Engineering, 1994, pp. 191-200.
  7. N. Mansour and K. El-Fakin. "Simulated Annealing and Genetic Algorithms for Optimal Regression Testing," Journal of Software Maintenance: Research and Practice, Vol. 11, No. 1, 1999, pp. 19-34.
  8. T. Ralphs and M. Guzelsoy, "The SYMPHONY Callable Library for Mixed Integer Programming," The Ninth INFORMS Computing Society Conference, 2005, pp. 61-73.
  9. G. Rothermel, M.J. Harrold, J. von Ronne, and C. Hong. "Empirical Studies of Test-Suite Reduction," Software Testing Verification and Reliability, Vol. 12, No. 4, 2002, pp. 219-249.
  10. P. Thevenod-Fosse, H. Waeselynck, and Y. Crouzet, "An Experimental Study on Software Structural Testing: Deterministic verses Random Input Generation," IEEE International Symposium on Fault Tolerant Computing, 1991, pp. 410-417.
  11. M. B. Wall, A Genetic Algorithm for Resource- Constrained Scheduling. MIT, PhD thesis, 1996.
  12. W.E. Wong, J.R. Horgan, S. London, and A.P. Mathur, "Effect of Test Set Minimization on Fault Detection Effectiveness," Proc. of the 17th International Conference on Software Engineering, 1995, pp. 41-50.
  13. W.E. Wong, J.R. Horgan, A.P. Mathur, and A. Pasquini, "Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in a Space Application," Annual International Computer Software and Applications Conference (COMPSAC), 1997, pp. 522-528.

Related papers

An Efficient Heuristic Based Test Suite Minimization Approach
DIBYA BORA

Indian Journal of Science and Technology, 2017

Objectives: Development of an efficient test suite minimization approach in order to reduce the size of a previously acquired test suite and produce a new representative suite which will guarantee the same requirement coverage that was achieved before minimization for an effective and efficient regression testing. Method: Test suite minimizations techniques try to reduce the size and redundancy of test suite by removing certain test cases since requirement covered by them are already covered by other test cases. But, it has been found that the acquired test cases after minimization severely lacks ability to achieve the desirable code coverage because the minimization was done based on a single test adequacy criteria. In this paper, we propose an efficient heuristic based test suite minimization algorithm which will reduce the size of the test suites with respective to multiple test adequacy criterions in order to preserve the fault detection effectiveness and code coverage characteristics of the final test suite. Findings: Our experimental results indicate that a significant percentage of reduction in the test suite size is achieved when the minimization is performed with respect to multiple test adequacy criterions. Our approach is unique compared to the existing approaches in the sense that, we carried out minimization based on multiple test adequacy criterions while most of the existing approaches usually take one or two criterions into consideration. The proposed approach is evaluated based on two well known software testing metrics; one indicate the percentage of reduction in test suite size and the second one indicate the percentage of code coverage achieved by the minimized test suite. Our experimental results indicate that a significant percentage of reduction in the size as well as significant code coverage characteristics is achieved when the minimization is done according to the proposed approach. Improvements: The important contribution of this study is that, it presents a novel and efficient test suite minimization technique that optimizes the test suite size based on multiple adequacy criterions.

View PDFchevron_right
Pre and Post Test Suite Reduction Techniques: A Comparison Study
Science Park Research Organization & Counselling

Test suite reduction is a critical activity which occurs pre or post test cases generation process. As software keeps growing large amounts of new test cases will be generated and added to the test pool and others will be updated. Accordingly test suite size will increase. Test suite reduction techniques have been pr oposed to eliminate redundant or irrelevant test cases based on variant criteria, while seeking to maintain the total effectiveness of the reduced test suite. In this paper we address and compare four test case reduction techniques: CBR (case-based reasoning) deletion algorithms Technique, GE & GRE Heuristics and priority cost technique, Model-checker based technique and Base choice coverage criterion technique. The aim of the study is to provide a guideline for choosing the appropriate test suite reduction techniques.

View PDFchevron_right
TEST SUIT REDUCTION BY FINDING COST OPTIMAL REPRESENTATIVE SET
Bimal Prasad Kar

Software testing is one of the important stages of software development. In software development, developers always depend on testing to reveal bugs. In the maintenance stage test suite size grow because of integration of new technique. Addition of new technique force to create new test case which increase the size of test suite. In regression testing new test case may be added to the test suite during the whole testing process. These additions of test cases create possibility of presence of redundant test cases. Due to limitation of time and resource, reduction techniques should be used to identify and remove them. Research shows that a subset of the test case in a suit may still satisfy all the test objectives which is called as representative set. Redundant test case increase the execution cost of the test suite, in spite of NP-completeness of the problem there are few good reduction techniques have been available. In this paper the previous technique proposed [17] is improved to find out cost optimal representative set.

View PDFchevron_right
A survey on Test Suite Reduction frameworks and tools
Saif Ur Rehman Khan

International Journal of Information Management, 2016

Software testing is a widely accepted practice that ensures the quality of a System under Test (SUT). However, the gradual increase of the test suite size demands high portion of testing budget and time. Test Suite Reduction (TSR) is considered a potential approach to deal with the test suite size problem. Moreover, a complete automation support is highly recommended for software testing to adequately meet the challenges of a resource constrained testing environment. Several TSR frameworks and tools have been proposed to efficiently address the test-suite size problem. The main objective of the paper is to comprehensively review the state-of-the-art TSR frameworks to highlights their strengths and weaknesses. Furthermore, the paper focuses on devising a detailed thematic taxonomy to classify existing literature that helps in understanding the underlying issues and proof of concept. Moreover, the paper investigates critical aspects and related features of TSR frameworks and tools based on a set of defined parameters. We also rigorously elaborated various testing domains and approaches followed by the extant TSR frameworks. The results reveal that majority of TSR frameworks focused on randomized unit testing, and a considerable number of frameworks lacks in supporting multi-objective optimization problems. Moreover, there is no generalized framework, effective for testing applications developed in any programming domain. Conversely, Integer Linear Programming (ILP) based TSR frameworks provide an optimal solution for multi-objective optimization problems and improve execution time by running multiple ILP in parallel. The study concludes with new insights and provides an unbiased view of the state-of-the-art TSR frameworks. Finally, we present potential research issues for further investigation to anticipate efficient TSR frameworks.

View PDFchevron_right
A new algorithm to Test Suite Reduction based on cluster analysis
Alireza Khalilian, Yalda Fazlalizadeh

2009 2nd IEEE International Conference on Computer Science and Information Technology, 2009

Regression testing is a significant activity which occurs during the maintenance level of a software lifecycle. However, it requires a large amount of test cases to test new or modified parts of the software. To address the issue, Test Suite Reduction techniques have been presented. An appropriate technique should generate a minimized test suite which exercises different execution paths within a program while retaining the fault detection capability of the suite admissible. To achieve this, a heuristic algorithm is proposed in this paper. The new algorithm clusters test cases based on the similarity of their execution profiles and sample some representatives to form the reduced test suite. The results of applying the new algorithm to the Siemens suite and comparing to the H algorithm manifest interesting insights into the effectiveness of the proposed algorithm.

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved