Usability in practice

Drawing upon various studies and expert opinion, I will examine possible appropriate quality metrics for evaluating research using formative user testing.Quality research should be structured to be appropriate to the project, utilizing best methodological practices where feasible. Data gathered should be meaningful in that results are indicative of major usability problems and are actionable. Finally, quality research using formative testing should generate insight into understanding users and should provide information to address user needs. Mindful compromises against internal validity and reliability can be made to achieve these goals.

The paper presents the results of a study on usability methods for evaluating Web sites. lt summarizes the "Heuristics for Web Communications, " and reports the practical experiences with these heuristics, contrasting them with the "Keevil Index " and combining them with user testing with thinking aloud. It concludes that working with the "Heuristics for Web Communications " takes more time and effort than working with the "Kevil Index," but produces more consistent results. The heuristics proved to be applicable both in heuristic evaluation and in combination with user testing. Zusammenfassung Der Beitrag präsentiert eine Studie über Evaluationsmethoden zur Web-Usability. Er beschreibt die "Heuristics for Web Communications " und berichtet von den praktischen Erfahrungen mit den Heuristiken, die mit dem "Keevil Index" verglichen und mit Benutzertests mit lautem Denken kombiniert werden. Das Ergebnis zeigt, dass eine Evaluation mit den beschriebenen Heuristiken gegenüber dem "Keevil Index" mehr Zeit und Aufwand erfordert, aber konsistentere Ergebnisse bringt. Die Heuristiken haben sich sowohl in der experten-zentrierten Evaluation als auch in Kombination mit dem Benutzertest insgesamt als geeignete Evaluationsmethode erwiesen.

Proceedings of the International Workshop on Interplay …

Knowledge of the factors that affect developers’ priority of usability evaluation results is important in order to improve the interplay between usability evaluation and software development. In the presented study, the effect of usability inspection severity ratings on the developers’ priority of evaluation results was investigated. The usability inspection results with higher severity ratings were associated with higher developer priority. This result contradicts Sawyer et al. [7], but is in line with Law’s [5, 6] finding related to the impact of user test results. The findings serve as a reminder for HCI professionals to focus on high severity issues.

Proceedings of the International Workshop on Interplay …

2007

The paper investigates the performance of novice evaluators in usability evaluations by presenting the results of a comparative usability evaluation that was conducted by nine novice evaluator teams. The evaluation teams performed considerably well in terms of the validity of their results, which counts for their participation in usability evaluation projects. The thoroughness of the results obtained was found to be in a relatively stable ratio of about 20-25% of the total number of problems found for eight of the nine teams, which gives a clear indication of the degree to which novice evaluators can identify usability problems. The consistency of the results was not satisfactory, although similar to results of other studies that involve professional evaluators. The paper suggests that when novice evaluators have to be employed for usability evaluations and it is important to find most usability problems then parallel usability evaluations can provide overall valid and thorough results.

2007

ABSTRACT In this position paper, I argue that a focus on problem counts in user-focused evaluation is a limiting view, particularly in relation to downstream utility. While fixing problems is an important consideration, a broader perspective that also seeks to develop systems that better fit their users–their ways of thinking and motivations, etc.–will result in new design opportunities that go beyond small evolutions of existing designs. I illustrate this view with a sketchy analysis of the use of a travel website.

For much of the last century, relational ontologies guided progressive innovative thinking. This paper contrasts relational with essentialist approaches to evaluation and its role within an embracing development process. Downstream utility is argued to be largely dependent on upstream fecundity, which requires clarity of design purpose and constant coordination of all design, implementation and evaluation. Without both clarity and constant coordination, upstream futility can result.