Remote possibilities?: international usability testing at a distance

Developments in Human Factors in …, 2006

Whilst testing the usability of a product in a usability laboratory, usability researchers generally try to extract usability information from a representation of ‗context in use'. However, there might be cases where it is difficult to simulate the real-world usage due to lack of contextual information (e.g. scarcity of information on low-socio-economic status users' living conditions, etc.), or physical limitations (e.g. difficulty in representing the actual physical environment, etc.). This set of facts requires testing in natural settings. Discussion is focused on the relative advantages and disadvantages of each setting, along with the technical infrastructure.

2014

Several emerging countries experience increasing software development activities. With the purpose of provide useful feedback on possible courses of action for increasing application of usability evaluation in such countries, this paper explores the status of usability evaluation in a digitally emerging country. Our aim is to identifying common characteristics or behavioral patterns that could be compared with digitally advanced countries. We used an online survey answered by 26 software development organizations, which gave a snapshot of the application of usability evaluation in these organizations. We found many similarities with advanced countries, several completely new obstacles more connected with software development matters and a relatively positive improvement in the lack of “usability culture”. These findings suggest good conditions to improve conduction of usability evaluations in digitally emerging countries.

2005

Abstract In the past, remote user testing has offered promising results when compared to its local counterpart. Nonetheless, the comparison studies conducted insofar have relied on small independent samples, measuring key metrics and have not focused on the qualitative objectives of user testing. The current study addresses these issues in order to reinforce the efficiency of the remote method in the key metrics but to also examine its ability to reveal qualitative data.

interactions, 2005

WITH THE INTERNET TUCKING INTO EVERY CORNER of the globe, products and services that were once local have become global. This ubiquity begets a need to sharpen our tools and learn how to conduct user experience research apart from our cultural comfort zone; it requires new tactics and refinement of old methods. Our perspective is one of a consulting firm that conducts usability tests abroad for US firms and in the US for non-US firms. Organizations that routinely conduct testing across borders will resonate with our thoughts below and can likely think of anecdotes to accompany them. When thinking of multinational testing, several things come to mind. Obvious issues of language and culture spring to the fore, but many of the challenges are more subtle. We will expose both types of issues as well as share lessons learned. Our starting point is the organization that needs to test globally but is looking for where and how to begin.

2004

Usability testing is becoming a more important part of the software design process. New methods allow remote usability testing to occur. Remote testing can be less costly and allow more data to be collected in less time in many cases, provided the user can still provide meaningful data. However, little is known about differences in the user experience between the two testing methods. In an effort to find differences in user experience between remote and traditional website usability testing, this study randomly assigned participants into two groups, one completing a usability test in a traditional lab setting, while the other group utilizing a remote testing location. Both groups completed two tasks, one simple, one complex, using Amazon.com as a test interface. Task time and number of critical incidents reported were the dependent measures. Significant differences were found for task times both in the between and within-subjects conditions for task times. Task times differed signif...

Proceedings of HCI …, 1999

International Journal of Advanced Computer Science and Applications

Online Nurse Test for Indonesian Nurse Competency (ONT UKNI) is a mobile application that was developed to help increase the success rate of nurse competency test participants. By using this application, users can learn more about the materials tested and conduct try out as a competency test simulation. However, ONT UKNI has not yet passed adequate testing stages, especially in terms of User Interface/User Experience (UI/UX). The Covid-19 pandemic situation presents challenges in the UI/UX testing process. Testing process which is ideally carried out face-to-face with respondents to get further insight, have to be carried out using another approach following the new normal protocol. This study aims to test the usability of UI/UX with an unmoderated remote testing approach on ONT UKNI application using a USE questionnaire. The test was performed using 26 respondents and all were nursing profession students of Universitas Muhammadiyah Yogyakarta. Respondents performed 8 tasks on ONT UKNI and answered set of questionnaire that will be tabulated and analyzed. The results indicate that usefulness, ease of learning, and satisfaction variables get the Very Good category while the ease of use variable gets the Good category. Overall, usability testing using an unmoderated remote testing approach can be carried out and able to provide information about areas where users are satisfied with ONT UKNI application. However, some areas still have room for improvement such as better UI design and implementation of gamification.

2004

Local user testing accrues high costs when solely relied on during the iterative cycle of usercentered design. A number of automatic evaluation methods have emerged with the aim of reducing costs incurred by repeated local tests, international user testing and post-deployment assessments. When considering costs, it is important not to overlook the benefits provided in human-moderated user testing. This paper focuses on the benefits of human moderation, provides test administration protocols for human-moderated remote testing and examines those protocols in real life settings.

PsycEXTRA Dataset, 2000

In this paper we discuss a pilot usability study using wireless Internet-enabled personal digital assistants (PDAs). We compared usability data gathered in traditional lab studies with a proxy-based clickstream logging and analysis tool. We found that this remote testing technique can more easily gather many of the contentrelated usability issues, but device-related issues are more difficult to capture.