Modern test case generation techniques can automatically achieve high code coverage. If they oper... more Modern test case generation techniques can automatically achieve high code coverage. If they operate on the unit level, they run the risk of generating nonsensical inputs, which, when causing failures, are painful to identify and eliminate. Running a unit test generator on five open source Java programs, we found that all of the 181 reported failures were false failures-that is, indicating a problem in the generated test case rather than the program. By generating test cases at the GUI level, our EXSYST prototype can avoid such false failures by construction. In our evaluation, it achieves higher coverage than search-based test generators at the unit level; yet, every failure can be shown to be caused by a real sequence of input events. Whenever a system interface is available, we recommend considering search-based system testing as an alternative to avoid false failures.
The Art and Science of Analyzing Software Data, 2015
How do we know a program does what it claims to do? Our CHABADA prototype can cluster Android app... more How do we know a program does what it claims to do? Our CHABADA prototype can cluster Android apps by their description topics and identify outliers in each cluster with respect to their API usage. A "weather" app that sends messages thus becomes an anomaly; likewise, a "messaging" app would typically not be expected to access the current location. In this paper we present a new approach for anomaly detection that improves the classification results of our original CHABADA paper [1]. Applied on a set of 22,500+ Android applications, our CHABADA prototype can now predict 74% of novel malware as such, without requiring any known malware patterns, maintaining the false positive rate close to 10%.
2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing, 2015
Search-based testing has been successfully applied to generate complex sequences of events for gr... more Search-based testing has been successfully applied to generate complex sequences of events for graphical user interfaces (GUIs), but typically relies on simple heuristics or random values for data widgets like text boxes. This may greatly reduce the effectiveness of test generation for applications which expect specific input values to be entered in their GUI by users. Generating such specific input values is one of the virtues of dynamic symbolic execution (DSE), but DSE is less suitable to generate sequences of events. Therefore, this paper describes a hybrid approach that uses search-based testing to generate sequences of events, and DSE to build input data for text boxes. This is achieved by replacing standard widgets in a system under test with symbolic ones, allowing us to execute GUIs symbolically. In this paper, we demonstrate an extension of the search-based GUI testing tool EXSYST, which uses DSE to successfully increase the obtained code coverage on two case study applications.
Over the years, many different indexing techniques and search algorithms have been proposed, incl... more Over the years, many different indexing techniques and search algorithms have been proposed, including CSS-trees, CSB +-trees, k-ary binary search, and fast architecture sensitive tree search. There have also been papers on how best to set the many different parameters of these index structures, such as the node size of CSB +-trees. These indices have been proposed because CPU speeds have been increasing at a dramatically higher rate than memory speeds, giving rise to the Von Neumann CPU-Memory bottleneck. To hide the long latencies caused by memory access, it has become very important to well-utilize the features of modern CPUs. In order to drive down the average number of CPU clock cycles required to execute CPU instructions, and thus increase throughput, it has become important to achieve a good utilization of CPU resources. Some of these are the data and instruction caches, and the translation lookaside buffers. But it also has become important to avoid branch misprediction penalties, and utilize vectorization provided by CPUs in the form of SIMD instructions. While the layout of index structures has been heavily optimized for the data cache of modern CPUs, the instruction cache has been neglected so far. In this paper, we present NitroGen, a framework for utilizing code generation for speeding up index traversal in main memory database systems. By bringing together data and code, we make index structures use the dormant resource of the instruction cache. We show how to combine index compilation with previous approaches, such as binary tree search, cache-sensitive tree search, and the architecture-sensitive tree search presented by Kim et al. v vi Acknowledgments This work would not have been completed without the help and support of the many individual people who have helped me along the way. Particularly: Prof. Dr. Jens Dittrich for providing me with the opportunity of pursuing this research idea under his guidance, for advising me over the course of it, and for being the first to encourage me to get involved in research. Prof. Dr. Sebastian Hack for serving as my second reviewer, and for his valuable suggestions. To Dr. Martin Theobald for offering his advise. To Stefan Richter, and Christoph Pinkel for offering me excellent feedback on draft copies of this work. To Alekh, Jorge, Jörg, Stefan, Pascal, and Julian for helping out with bits and ends. To the many friends and my family for their constant support. Lastly, I am deeply indebted to Katrin for her understanding, patience, and encouragement throughout the long process of work that went into this thesis. vii viii
Modern test case generation techniques can automatically achieve high code coverage. If they oper... more Modern test case generation techniques can automatically achieve high code coverage. If they operate on the unit level, they run the risk of generating inputs infeasible in reality, which, when causing failures, are painful to identify and eliminate. Running a unit test generator on five open source Java programs, we found that all of the 181 reported failures were false failures-that is, indicating a problem in the generated test case rather than the program. By generating test cases at the GUI level, our EXSYST prototype can avoid such false alarms by construction. In our evaluation, it achieves higher coverage than search-based test generators at the unit level; yet, every failure can be shown to be caused by a real sequence of input events. Whenever a system interface is available, we recommend considering search-based system testing as an alternative to avoid false failures. public class RandoopTest0 extends TestCase { public void test8() throws Throwable { AddressBook var0 = new AddressBook(); EventHandler var1 = var0.getEventHandler(); Category var2 = var0.getRootCategory(); Contact var3 = new Contact(); . . . 75 more calls. . . AddressBook var79 = new AddressBook(); EventHandler var80 = var79.getEventHandler(); Category var81 = var79.getRootCategory(); String var82 = var81.getName(); var77.categorySelected(var81); Category var85 = var65.createCategory(var81, "hi!"); String var86 = var85.toString(); Category var88 = var0.createCategory(var85, . . .); // raises NameAlreadyInUseException } }
2012 34th International Conference on Software Engineering (ICSE), 2012
Test generation tools commonly aim to cover structural artefacts of software, such as the source ... more Test generation tools commonly aim to cover structural artefacts of software, such as the source code or the user interface. However, focusing only on source code can lead to unrealistic or irrelevant test cases, while exploring a user interface often misses much of the underlying program behavior. Our EXSYST prototype takes a new approach by exploring user interfaces while aiming to maximize code coverage, thus combining the best of both worlds. Experiments show that such an approach can achieve high code coverage matching and exceeding the code coverage of traditional unitbased test generators; yet, by construction every test case is realistic and relevant, and every detected failure can be shown to be caused by a real sequence of input events.
Test generation tools commonly aim to cover structural artefacts of software, such as the source ... more Test generation tools commonly aim to cover structural artefacts of software, such as the source code or the user interface. However, focusing only on source code can lead to unrealistic or irrelevant test cases, while exploring a user interface often misses much of the underlying program behavior. Our EXSYST prototype takes a new approach by exploring user interfaces while aiming to maximize code coverage, thus combining the best of both worlds. Experiments show that such an approach can achieve high code coverage matching and exceeding the code coverage of traditional unitbased test generators; yet, by construction every test case is realistic and relevant, and every detected failure can be shown to be caused by a real sequence of input events.
Modern test case generation techniques can automatically achieve high code coverage. If they oper... more Modern test case generation techniques can automatically achieve high code coverage. If they operate on the unit level, they run the risk of generating inputs infeasible in reality, which, when causing failures, are painful to identify and eliminate. Running a unit test generator on five open source Java programs, we found that all of the 181 reported failures were false failures-that is, indicating a problem in the generated test case rather than the program. By generating test cases at the GUI level, our EXSYST prototype can avoid such false alarms by construction. In our evaluation, it achieves higher coverage than search-based test generators at the unit level; yet, every failure can be shown to be caused by a real sequence of input events. Whenever a system interface is available, we recommend considering search-based system testing as an alternative to avoid false failures. public class RandoopTest0 extends TestCase { public void test8() throws Throwable { AddressBook var0 = new AddressBook(); EventHandler var1 = var0.getEventHandler(); Category var2 = var0.getRootCategory(); Contact var3 = new Contact(); . . . 75 more calls. . . AddressBook var79 = new AddressBook(); EventHandler var80 = var79.getEventHandler(); Category var81 = var79.getRootCategory(); String var82 = var81.getName(); var77.categorySelected(var81); Category var85 = var65.createCategory(var81, "hi!"); String var86 = var85.toString(); Category var88 = var0.createCategory(var85, . . .); // raises NameAlreadyInUseException } }
Uploads
Papers by Florian Gross