Aditya Mathur

Fault coverage of Constrained Random Test Selection for access control: A formal analysis

by Ammar Masood and Aditya Mathur

A probabilistic model of fault coverage is presented. This model is used to analyze the variation... more

Using Sensitivity Analysis to Validate a State Variable Model of the Software Test Process

IEEE Transactions on Software Engineering, 2003

AbstractWe report on the sensitivity analysis of a state variable model ( Model S) proposed earli... more

Vulnerability Testing of Software System Using Fault Injection

We describe an approach for testing a software system for possible security flaws. Traditionally,... more We describe an approach for testing a software system for possible security flaws. Traditionally, security testing is done using penetration analysis and formal methods. Based on the observation that most security flaws are triggered due to a flawed interaction with the environment, we view the security testing problem as the problem of testing for the fault-tolerance properties of a software system. We consider each environment perturbation as a fault and the resulting security compromise a failure in the toleration of such faults. Our approach is based on the well known technique of fault-injection. Environment faults are injected into the system under test and system behavior observed. The failure to tolerate faults is an indicator of a potential security flaw in the system. An Environment-Application Interaction (EAI) fault model is proposed. EAI allows us to decide what faults to inject. Based on EAI, we present a security-flaw classification scheme. This scheme was used to classify 142 security flaws in a vulnerability database. This classification revealed that 91 of the security flaws in the database are covered by the EAI model.

Download

Mutation Versus All-uses: An Empirical Evaluation of Cost, Strength and Effectiveness

Although mutation has been empirically found to be effective in detecting faults, it often impose... more Although mutation has been empirically found to be effective in detecting faults, it often imposes unacceptable demands on computing and human resources because of the large number of mutants that need to be compiled and executed on one or more test cases. We designed a case study to explore two alterna.tives of mutation to reduce its cost without significantly deteriorating its strength and effectiveness. The alternatives are (1) constrained abs/ror mutation which examines only the abs and ror mutants and ignores the others, and (2) randomly selected 10% mutation which examines only 10% of the randomly selected mutants of each mutation operator in Mothra. Data collected during experimentation have shown tha.t both alternatives provide a significant cost reduction in terms of the number of test cases required to satisfy an adequacy criterion and the number of mutants to be examined. Such gain is accompanied by a small strength loss in the a.bility to distinguish non-equivalent mutants and cover feasible all-uses. In terms of decreasing order of fault detection effectiveness the ranking is mutation, constrained abs/ror mutation, all-uses, and randomly selected 10% mutation. These data suggest that examining only a small percentage of the mutants may be a useful heuristic for evaluating and constructing test sets in practice.

Download

A Formal Model of the Software Test Process

IEEE Transactions on Software Engineering, 2002

AbstractÐA novel approach to model the system test phase of the software life cycle is presented.... more AbstractÐA novel approach to model the system test phase of the software life cycle is presented. This approach is based on concepts and techniques from control theory and is useful in computing the effort required to reduce the number of errors and the schedule slippage under a changing process environment. Results from these computations are used, and possibly revised, at specific checkpoints in a feedback-control structure to meet the schedule and quality objectives. Two case studies were conducted to study the behavior of the proposed model. One study reported here uses data from a commercial project. The outcome from these two studies suggests that the proposed model might well be the first significant milestone along the road to a formal and practical theory of software process control.

Download

Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in a Space Application

An important question in software testing is whether it is reasonable to apply coverage based cri... more An important question in software testing is whether it is reasonable to apply coverage based criteria as a filter to reduce the size of a test set. An empirical study was conducted using a test set minimization technique to explore the effect of reducing the size of a test set, while keeping block coverage constant, on the fault detection strength of the resulting minimized test set. Two types of test sets were examined. For those with respect to a fixed size, no test case screening was conducted during the generation, whereas for those with respect to a fixed coverage, each subsequent test case had to improve the overall coverage in order to be included. The study reveals that no matter how a test set is generated (with or without any test case screening) block minimized test sets have a size/effectiveness advantage, in terms of a significant reduction in test set size but with almost the same fault detection effectiveness, over the original non-minimized test sets

Download

Performance, effectiveness, and reliability issues in software testing

The author has identified two problems that need to be overcome in order that some of the powerfu... more

Reducing the cost of mutation testing: An empirical study

Journal of Systems and Software, 1995

Amongst the various testing strategies, mutation testing has been empirically found to be the mos... more Amongst the various testing strategies, mutation testing has been empirically found to be the most e ective in detecting faults. However, mutation often imposes unacceptable demands on computing and human resources due to the large number of mutants that need to be compiled and executed on one or more test cases. In addition, the tester needs to examine many mutants and analyze these for possible equivalence with the program under test. For these reasons, mutation is generally regarded by the practicing test engineer as too expensive to use. As one signi cant component of the cost of mutation is the execution of mutants against test cases, we believe that the cost can be reduced dramatically by reducing the number of mutants that need to be examined. We report the results from a case study designed to investigate two alternatives to reduce the cost of mutation. The alternatives considered are: (1) the randomly selected x% mutation, and (2) the constrained mutation. We provide experimental data indicating that both alternatives lead to test sets that distinguish a signi cant number of all mutants and provide high all-uses coverage.

Download

Effect of test set size and block coverage on the fault detection effectiveness

Effect of Test Set Size and Block Coverage on the Fault Detection Effectiveness * W. Eric Wong, J... more

Assessing Testing Tools in Research and Education

IEEE Software, 1992

An evaluation of three software engineering tools based on their use in research and educational ... more An evaluation of three software engineering tools based on their use in research and educational environments is presented. The three testing tools are Mothra, a mutation-testing tool, Asset, a dataflow testing tool, and ATAC, a dataflow testing tool. Asset, ATAC, and Mothra were used in research projects that examined relative and general fault-detection effectiveness of testing methods, how good a test set is after functional testing based on program specification, how reliability estimates from existing models vary with the testing method used, and how improved coverage affects reliability. Students used ATAC and Mothra by treating the tools as artifacts and studying them from the point of view of documentation, coding style, and possible enhancements, solving simple problems given during testing lectures, and conducting experiments that supported ongoing research in software testing and reliability. The strengths, weaknesses, and performances of Asset, Mothra, and ATAC are discussed.>

Download

Design Of Mutant Operators For The C Programming Language

Download

Uploads

Papers by Aditya Mathur

Log In