Falsification Testing for Usability Inspection Method Assessment

1999

Inspection is a fundamental means of achieving software usability. Past research showed that the current usability inspection techniques were rather ineffective. We developed perspective-based usability inspection, which divides the large variety of usability issues along different perspectives and focuses each inspection session on one perspective. We conducted a controlled experiment to study its effectiveness, using a post-test only control group experimental design, with 24 professionals as subjects.

2012

Industry case studies are required to assess the suitability of a software technology to the industrial environment. The use of experimental studies in the development of software technologies provides a reliable knowledge basis and helps determine the best situations to use the technology, reducing the uncertainties in its adoption by the software industry. Therefore, this paper presents two case studies carried out in partnership with two companies of an Industrial Park, in order to evaluate and improve the WDP-RT (Web Design Perspectives-Based Inspection -Reading Technique), a specific usability inspection technique for Web applications.

Several approaches have been proposed to ensure the quality of interactive systems. However, interactive systems continue to reach users with malfunctions, such as usability, communicability and interaction errors. Researches show that the lack of usability knowledge in software development organizations is an obstacle for usability evaluation. Our research goal is to popularize usability inspections so that even novice inspectors are able to perform it. Aiming to provide an approach to be used during the development process of web application, we have proposed the WE-QT technique. We are using an experimental methodology to evolve our technique and transfer it from the academy to industry. This paper presents a new comparative study; the results show that WE-QT technique is more efficient than and as effective as the compared technique.

CLEI Electronic Journal

Usability is one of the most important software quality attributes regarding its acceptability by end users. It is even more important in the context of Web applications. One way of evaluating application usability is through inspections. The WDP (Web Design Perspectives- based Usability Inspection Technique) presents evidence of industrial use feasibility; however some computerized support had been suggested by practitioners. Therefore, the WDP Tool was built, aiming to provide automated support the WDP technique application, reducing the effort involved in usability inspections with WDP. This paper presents two observational studies regarding the use of the WDP Tool use, one in vivo and one in vitro, which aimed to analyze the cost-effectiveness of its application and its appropriateness to the industrial environment through the Technology Acceptance Model (TAM). The results provide indications about the feasibility of using the WDP Tool to support usability inspections in real so...

Within the literature of human-computer interaction there is a vigorous debate on the relative merits of two classes of evaluation methods; those that carry out an empirical study of users' task performance and those that employ experts to inspect a design. The central themes in this debate are effectiveness and cost-efficiency. While these concerns are important in commercial usability work, an analysis of project goals and constraints may be more useful in selecting and justifying methods. This paper proposes a model for selecting methods based on the above criteria. It also suggests that, as inspection methods are more grounded in the concerns of practice than those of science, their selection should also be influenced by factors arising from practice. Empirical methods Empirical methods rely on observing and measuring representative users attempting typical tasks. For example, in a user test, a researcher might: a) recruit a sample of participants from a demographic profile ...

2003

Usability inspections are a cost-effective alternative to conventional usability testing for evaluating and improving the usability of software and Web-based systems. The collaborative usability inspection technique described here is a highly structured, scenario-driven inspection process optimized for rapidly identifying, classifying, and prioritizing usability defects. The process has been refined through extensive application into a highly efficient, easily learned technique. The basic structure is introduced and guidelines for conducting successful inspections are described.

2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS), 2015

A Use Case Model is composed of use cases that describe software functionalities through Use Case Specifications. The evaluation of the specifications that compose such a model can allow for an early identification of usability defects. We previously proposed MIT 1-Model Inspection Technique for Usability Evaluation that aims to support the identification of usability defects through the evaluation of use cases specifications. In this paper, we present the evaluation of this technique through a controlled experiment that measured its efficiency, effectiveness, perceived ease of use, and perceived usefulness when compared to the Use Case Evaluation (UCE) method. Our quantitative findings indicate that MIT 1 allows users to find more usability defects in less time than UCE. However, UCE was considered easiest to use and more useful than MIT 1, highlighting improvement needs for MIT 1.

International Journal of Human-Computer Interaction, 2001

Computer professionals have a need for robust, easy-to-use usability evaluation methods (UEMs) to help them systematically improve the usability of computer artifacts. However, cognitive walkthrough (CW), heuristic evaluation (HE), and thinking-aloud study (TA)-3 of the most widely used UEMs-suffer from a substantial evaluator effect in that multiple evaluators evaluating the same interface with the same UEM detect markedly different sets of problems. A review of 11 studies of these 3 UEMs reveals that the evaluator effect exists for both novice and experienced evaluators, for both cosmetic and severe problems, for both problem detection and severity assessment, and for evaluations of both simple and complex systems. The average agreement between any 2 evaluators who have evaluated the same system using the same UEM ranges from 5% to 65%, and no 1 of the 3 UEMs is consistently better than the others. Although evaluator effects of this magnitude may not be surprising for a UEM as informal as HE, it is certainly notable that a substantial evaluator effect persists for evaluators who apply the strict procedure of CW or observe users thinking out loud. Hence, it is highly questionable to use a TA with 1 evaluator as an authoritative statement about what problems an interface contains. Generally, the application of the UEMs is characterized by (a) vague goal analyses leading to variability in the task scenarios, (b) vague evaluation procedures leading to anchoring, or (c) vague problem criteria leading to anything being accepted as a usability problem, or all of these. The simplest way of coping with the evaluator effect, which cannot be completely eliminated, is to involve multiple evaluators in usability evaluations.

1998

Abstract Inspection is a fundamental means of achieving software usability. Past research showed that during usability inspection the success rate (percentage of problems detected) of each individual inspector was rather low. We developed perspective-based usability inspection, which divides the large variety of usability issues along different perspectives and focuses each inspection session on one perspective.

Journal of Visual Languages & Computing, 2015

Today, the success of a software application strongly depends on the usability of its interface, so the evaluation of interfaces has become a crucial aspect of software engineering. It is recognized that automatic tools for graphical user interface evaluation may greatly reduce the costs of traditional activities performed during expert evaluation or user testing in order to estimate the success probability of an application. However, automatic methods need to be empirically validated in order to prove their effectiveness with respect to the attributes they are supposed to evaluate. In this work, we empirically validate a usability evaluation method conceived to assess consistency aspects of a GUI with no need to analyze the back-end. We demonstrate the validity of the approach by means of a comparative experimental study, where four web sites and a stand-alone interactive application are analyzed and the results compared to those of a human-based usability evaluation. The analysis of the results and the statistical correlation between the tool's rating and humans' average ratings show that the proposed methodology can indeed be a useful complement to standard techniques of usability evaluation.