Using GUI Run-Time State as Feedback to Generate Test Cases

Advances in Computers - AC, 2010

Despite the ubiquity of software applications that employ a graphical-user interface (GUI) front-end, functional system testing of these applications has remained, until recently, an understudied research area. During “GUI testing,” test cases, modeled as sequences of user input events, are created and executed on the software by exercising the GUI's widgets. Because each possible sequence of user events may potentially be a test case and today's GUIs offer enormous flexibility to end-users, in principle, GUI testing requires a prohibitively large number of test cases. Any practical test-case generation technique must sample the vast GUI input space. Existing techniques are largely manual, and hence extremely resource intensive. Several new automated model-based techniques have been developed in the past decade. All these techniques develop, either manually or automatically, a model of the GUI and employ it to generate test cases. This chapter presents the first detailed taxonomy of these techniques. A small GUI application is used as a running example to demonstrate each technique and illustrate its relative strengths and weaknesses.

ACM SIGSOFT Software Engineering Notes, 2003

Although graphical user interfaces (GUIs) constitute a large part of the software being developed today and are typically created using rapid prototyping, there are no effective regression testing techniques for GUIs. The needs of GUI regression testing differ from those of traditional software. When the structure of a GUI is modified, test cases from the original GUI are either reusable or unusable on the modified GUI. Since GUI test case generation is expensive, our goal is to make the unusable test cases usable. The idea of reusing these unusable ( a.k.a. obsolete ) test cases has not been explored before. In this paper, we show that for GUIs, the unusability of a large number of test cases is a serious problem. We present a novel GUI regression testing technique that first automatically determines the usable and unusable test cases from a test suite after a GUI modification. It then determines which of the unusable test cases can be repaired so they can execute on the modified G...

IEEE Access

Traditionally, software testing is aimed at showing the presence of faults. This paper proposes a novel approach to testing graphical user interfaces (GUI) for showing both the presence and absence of faults in the sense of ideal testing. The approach uses a positive testing concept to show that the GUI under consideration (GUC) does what the user expects; to the contrary, the negative testing concept shows that the GUC does not do anything that the user does not expect, building a holistic view. The first step of the approach models the GUC by a finite state machine (FSM) that enables the model-based generation of test cases. This is always possible as the GUIs are considered as strictly sequential processes. The next step converts the FSM to an equivalent regular expression (RE) that will be analyzed first to construct test selection criteria for excluding redundant test cases and construct test coverage criteria for terminating the positive test process. Both criteria enable us to assess the adequacy and efficiency of the positive tests performed. The negative tests will be realized by systematically mutating the FSM to model faults, the absence of which are to be shown. Those mutant FSMs will be handled and assessed in the same way as in positive testing. Two case studies illustrate and validate the approach; the experiments' results will be analyzed to discuss the pros and cons of the techniques introduced. INDEX TERMS GUI testing, holistic testing, ideal testing, model-based testing, mutation testing, test generation, regular expression.

IEEE Transactions on Software Engineering, 2005

Software is increasingly being developed/maintained by multiple, often geographically distributed developers working concurrently. Consequently, rapid-feedback-based quality assurance mechanisms such as daily builds and smoke regression tests, which help to detect and eliminate defects early during software development and maintenance, have become important. This paper addresses a major weakness of current smoke regression testing techniques, i.e., their inability to automatically (re)test graphical user interfaces (GUIs). Several contributions are made to the area of GUI smoke testing. First, the requirements for GUI smoke testing are identified and a GUI smoke test is formally defined as a specialized sequence of events. Second, a GUI smoke regression testing process called Daily Automated Regression Tester (DART) that automates GUI smoke testing is presented. Third, the interplay between several characteristics of GUI smoke test suites including their size, fault detection ability, and test oracles is empirically studied. The results show that: 1) the entire smoke testing process is feasible in terms of execution time, storage space, and manual effort, 2) smoke tests cannot cover certain parts of the application code, 3) having comprehensive test oracles may make up for not having long smoke test cases, and 4) using certain oracles can make up for not having large smoke test suites.

2011

Advanced methods and tools for GUI software development allow a rapid and iterative process of prototyping and usability testing. Unfortunately, even with the support of test automation tools, testing of GUI software requires a lot of manual work, especially when the application under test is changing rapidly. In this paper we present an improved method and tool support for automated test modeling of Java GUI applications for model-based testing (MBT) purposes. The implemented GUI Driver tool generates structural models combined with a GUI state model presenting the behavior of the GUI application that is executed and observed automatically. The GUI Driver tool is combined with an open source MBT tool to form a tool chain to support automated testing of Java GUI applications. The models generated by the GUI Driver are used to generate test sequences with MBT tool, and the test sequences are then executed with the GUI Driver to generate a test report.

Software Testing, Verification and Reliability, 2007

Graphical user interfaces (GUIs) are by far the most popular means used to interact with today's software. The functional correctness of a GUI is required to ensure the safety, robustness and usability of an entire software system. GUI testing techniques used in practice are resource intensive; model-based automated techniques are rarely employed. A key reason for the reluctance in the adoption of model-based solutions proposed by researchers is their limited applicability; moreover, the models are expensive to create. Over the past few years, the present author has been developing different models for various aspects of GUI testing. This paper consolidates all of the models into one scalable event-flow model and outlines algorithms to semi-automatically reverse-engineer the model from an implementation. Earlier work on model-based test-case generation, test-oracle creation, coverage evaluation, and regression testing is recast in terms of this model by defining event-space exploration strategies (ESESs) and creating an end-to-end GUI testing process. Three such ESESs are described: for checking the event-flow model, test-case generation, and testoracle creation. Two demonstrational scenarios show the application of the model and the three ESESs for experimentation and application in GUI testing.

2010

Model-based GUI testing (MGT) is emerging as a promising approach for testing applications with a graphical user interface (GUI). Currently, test models in MGT approaches are close to the GUI implementation with limited ability to represent abstract actions. This paper introduces the Action-Event Framework (AEF), a MGT framework. This framework helps testers abstract away from low-level details of the GUI

Proceedings of the 2006 international workshop on Automation of software test - AST '06, 2006

This paper describes an ongoing research on test case generation based on Unified Modeling Language (UML). The described approach builds on and combines existing techniques for data and graph coverage. It first uses the Category-Partition method to introduce data into the UML model. UML Use Cases and Activity diagrams are used to respectively describe which functionalities should be tested and how to test them. This combination has the potential to create a very large number of test cases. This approach offers two ways to manage the number of tests. First, custom annotations and guards use the Category-Partition data which allows the designer tight control over possible, or impossible, paths. Second, automation allows different configurations for both the data and the graph coverage. The process of modeling UML activity diagrams, annotating them with test data requirements, and generating test scripts from the models is described. The goal of this paper is to illustrate the benefits of our model-based approach for improving automation on software testing. The approach is demonstrated and evaluated based on use cases developed for testing a graphical user interface (GUI).