Mechanical and Industrial Engineering

Social robots, conversational agents, voice assistants, and other embodied AI are increasingly a feature of everyday life. What connects these various types of intelligent agents is their ability to interact with people through voice. Voice is becoming an essential modality of embodiment, communication, and interaction between computer-based agents and end-users. This survey presents a meta-synthesis on agent voice in the design and experience of agents from a human-centered perspective: voice-based human-agent interaction (vHAI). Findings emphasize the social role of voice in HAI as well as circumscribe a relationship between agent voice and body, corresponding to human models of social psychology and cognition. Additionally, changes in perceptions of and reactions to agent voice over time reveals a generational shift coinciding with the commercial proliferation of mobile voice assistants. The main contributions of this work are a vHAI classification framework for voice across various agent forms, contexts, and user groups, a critical analysis grounded in key theories, and an identification of future directions for the oncoming wave of vocal machines.

Artificial intelligences (AI) are increasingly being embodied and embedded in the world to carry out tasks and support decision-making with and for people. Robots, recommender systems, voice assistants, virtual humans—do these disparate types of embodied AI have something in common? Here we show how they can manifest as “socially embodied AI.” We define this as the state that embodied AI “circumstantially” take on within interactive contexts when perceived as both social and agentic by people. We offer a working ontology that describes how embodied AI can dynamically transition into socially embodied AI. We propose an ontological heuristic for describing the threshold: the Tepper line. We reinforce our theoretical work with expert insights from a card sort workshop. We end with two case studies to illustrate the dynamic and contextual nature of this heuristic.

Introduction The global COVID-19 pandemic continues to have wide-ranging implications for health, including psychological well-being. A growing corpus of research reviews has emerged on the topic of psychological resilience in the context of the pandemic. However, this body of work has not been systematically reviewed for its quality, nor with respect to findings on the effectiveness of tools and strategies for psychological resilience. To this end, a meta-review protocol is proposed with the following objectives: (1) identify review work on the topic of psychological resilience during COVID-19; (2) assess the quality of this review work using A MeaSurement Tool to Assess systematic Reviews; (3) assess the risk of bias in this work; (4) generate a narrative summary of the key points, strengths and weaknesses; (5) identify the psychological resilience strategies that have been reviewed; (6) identify how these strategies have been evaluated for their effectiveness; (7) identify what outcomes were measured and (8) summarise the findings on strategies for psychological resilience so far, providing recommendations, if possible. Methods and analysis A systematic meta-review will be conducted in accordance with the Preferred Reporting Items for Systematic Reviews for Protocols and Joanna Briggs Institute umbrella review guidelines. Electronic searches of general databases, especially Web of Science, Scopus and PubMed, will be conducted. Only results from January 2020 onwards will be considered, coinciding with the COVID-19 pandemic. Only results in English will be included. Descriptive statistics, thematic analysis and narrative summaries describing the nature of the reviewed work and evaluation of psychological resilience strategies will be carried out. Ethics and dissemination Ethical approval is not needed for systematic review protocols. The results of the metareview will be published in an international peer-reviewed journal. The raw and summarised data will be shared in the journal or other open venues. PROSPERO registration number CRD42021235288.

Computer voice is experiencing a renaissance through the growing popularity of voice-based interfaces, agents, and environments. Yet, how to measure the user experience (UX) of voice-based systems remains an open and urgent question, especially given that their form factors and interaction styles tend to be non-visual, intangible, and often considered disembodied or “body-less.” As a first step, we surveyed the ACM and IEEE literatures to determine which quantitative measures and measurements have been deemed important for voice UX. Our findings show that there is little consensus, even with similar situations and systems, as well as an overreliance on lab work and unvalidated scales. In response, we offer two high-level descriptive frameworks for guiding future research, developing standardized instruments, and informing ongoing review work. Our work highlights the current strengths and weaknesses of voice UX research and charts a path towards measuring voice UX in a more comprehensive way.

Japan is a super-aging society, one marked by a steadily silvering population and shrinking birthrates. Social effects range from a decline in intergenerational homes to new tensions between the generations as elders delay retirement or return to work. Towards improving relations between the generations, two versions of an inclusive, intergenerational game employing "shared action," or time-movement synchrony, were developed: a finger-based version inspired by medical exercise technology from gerontology and a smart shoe-based version as a familiar form factor. A series of three exploratory studies were conducted with young Japanese adults and elders. Findings suggest that the foot version was most suitable overall and reveal initial, complex effects of age and gender on empathy measures that can inform affective computing initiatives within and beyond the cultural milieu of modern Japan.

Addressing societal problems is complex; little is known about which paths or approaches are successful. We discuss what is involved in knowing when and how and for whom change needs to occur, as well as the impact of doing so at scale-especially when novelty and academic contributions may be compromised. To this end, we present a 'scaling up' framework based on a societal project where we worked with multiple stakeholders to improve food waste recycling rates in a housing estate. We propose three main factors involved in scaling up: (i) 'the people,' through reimagining roles and relationships, (ii) 'the method,' requiring flexibility in design and research, and (iii) 'the impact,' informing new measures by handing over the evaluation. We reflect on the challenges, dilemmas, and successes encountered, as well as discuss the benefit of 'handing over' the evaluation process to gather scalable metrics based on economic modelling.

An extended design and evaluation framework of eudaimonia (personal growth, expressiveness) and hedonia (pleasure, comfort) was applied to a cooperative game for older adults who rely on power mobility. The purpose was to address two psychosocial well-being needs (perceptions of performance mastery and empathy enhancement) through a game with an interaction format that augments the experience of powered chair use: mixed reality with power mobility-based interaction and movement. Two versions of the game—one in a mixed reality format and one without—were created to evaluate the efficacy of using this framework and explore relevant user experience factors. Qualitative findings suggest that the mixed reality version of the game elicited eudaimonic experiences, where the older adult participants who were typically reliant on power mobility perceived a positive change in performance mastery while their partners, who were not reliant on power mobility developed a greater appreciation for everyday powered chair use. Further, a role reversal, where the elders took on a ‘caregiving’ role in the context of powered chair expertise, was observed. Moreover, inferential findings showed evidence of a predictive relationship between hedonic and eudaimonic orientations, psychological well-being, and game engagement, regardless of age, disability status, and game version.

Ensuring that mixed reality spaces are inclusive to those who have motor impairments requires evaluating and, if necessary, modifying existing technology for the assistive mobility devices they use. Heads-up displays (HUDs) are a common facilitator of virtual augmentations on physical reality, especially for motor vehicles. However, little research to date has explored the efficacy of such displays for powered chairs, which is paramount to ensure safety and effectiveness of use. In this pilot study, we compared (a) transparency of the display (transparent, semi-transparent, and opaque) and (b) presentation mode of the information (text, icon, and text + icon) through dynamic powered chair navigation tasks in a controlled course setup. We found no difference for workload and error among transparency and presentation styles, but a significant difference between presentation styles on delay. These results will be evaluated in full through an mixed reality platform with powered chair users.

For older adults, adopting a powered mobility aid signals a life transition that affects several well-being factors: their self-image, motivation, and self-perception of ability, but also how other people - friends, family, coworkers, strangers - relate to and perceive these individuals. We present initial findings from a user study in which we (a) recorded a baseline of well-being using established psychological instruments, and (b) evaluated a web-based prototype of a cooperative mixed reality game for improving the well-being of older powered chair users and their friends and family. Our results show that most participants considered themselves to be happy and were driven by hedonic and eudaimonic motives during gameplay, but were mixed with respect to their attitudes towards disability.

Older powered chair users’ perceptions on and attitudes towards mixed reality and modern facilitating technologies, such as tablets and smartphones, was explored to inform the design of mixed reality games that involve power mobility. Eleven older powered chair users (aged 55 and over) were interviewed in focus groups about their knowledge of, adoption of, experiences with, attitudes about, preferences for, and interest in these topics. Questionnaire and interview data were analysed using descriptive statistics and thematic analysis. Findings suggest that community-dwelling older powered chair users are a technologically forward group that use or are interested in trying new technologies and interaction paradigms, with key barriers being identity mismatch, affordability, and social acceptability. Although they did not see their use of powered chairs, new technologies, and digital entertainment as an integrated system, participants were receptive to the idea of synchronizing assistive and non-assistive technologies for the purpose of social entertainment, advanced training, and/or promoting understanding and empathy in others. In particular, the concept of a game-based mixed reality platform designed around performance mastery for older powered chair users and empathy training for able-bodied friends and family members was well-received. Initial resistance due to identity mismatch or lack of knowledge was mitigated through explanation and examples.

Gamification, or the use of game elements in non-game contexts, has become a popular and increasingly accepted method of engaging learners in educational settings. However, there have been few comparisons of different kinds of courses and students, particularly in terms of discipline and content. Additionally, little work has reported on course instructor/designer perspectives. Finally, few studies on gamification have used a conceptual framework to assess the impact on student engagement. This paper reports on findings from evaluating two gamified multimedia and social media undergraduate courses over the course of one semester. Findings from applying a multidimensional framework suggest that the gamification approach taken was moderately effective for students overall, with some elements being more engaging than others in general and for each course over time." Post-term questionnaires posed to the instructors/course designers revealed congruence with the student perspective and several challenges pre- and post-implementation, despite the use of established rules for gamifying curricula.

The advent of affordable and powerful mobile technology has allowed for explorations in mixed reality that merge virtual and physical space. However, the social and entertainment value and efficacy of mixed reality platforms for adult powered chair users has not been widely explored. In this paper, we introduce the Mobility Games project, which aims to produce a series of inclusive entertainment technologies and services for people who use powered chairs. We describe our first offering: an accessible, social mixed reality game for co-located mobile play in everyday spaces. Findings from two exploratory field studies and a post hoc observer survey show that adult powered chair users found the game to be entertaining and used a variety of path strategies as they learned to play the game. An initial set of theoretically-and empirically-informed guidelines for making mobile mixed reality games accessible to adult powered chair users with diverse abilities is proposed.

The past decade has seen the emergence of well-being/quality of life as a strand of inquiry in human factors research that has expanded the field's reach to matters beyond fit, functionality and usability. This effort has been spearheaded by "hedonomics," a human factors conceptualization of well-being that reflects the philosophical notion of hedonia, traditionally defined as pleasure. However, recent work in the psychology of well-being has shown that hedonia constitutes only one facet of well-being. In light of this, the concept of "eudaimonics" as a complement to hedonomics is introduced. First, these concepts are positioned relative to their counterparts in philosophy: where hedonomics is characterized by pleasure and avoidance of pain (hedonia), eudaimonics is characterized by flourishing and personal excellence (eudaimonia). Following this, a working conceptual framework for eudaimonics that is informed by the psychological literature, particularly the concepts of psychological well-being (PWB) and subjective well-being (SWB), is presented. An expansion of the hedonomics model of design priority hierarchy is offered. Applications to the domains of ageing well and technologies for older populations are proposed. Directions for future work, including the adoption and modification of psychology instruments for human factors research, is discussed.

Gamification has drawn the attention of academics, practitioners and business professionals in domains as diverse as education, information studies, human–computer interaction, and health. As yet, the term remains mired in diverse meanings and contradictory uses, while the concept faces division on its academic worth, underdeveloped theoretical foundations, and a dearth of standardized guidelines for application. Despite widespread commentary on its merits and shortcomings, little empirical work has sought to validate gamification as a meaningful concept and provide evidence of its effectiveness as a tool for motivating and engaging users in non-entertainment contexts. Moreover, no work to date has surveyed gamification as a field of study from a human–computer studies perspective. In this paper, we present a systematic survey on the use of gamification in published theoretical reviews and research papers involving interactive systems and human participants. We outline current theoretical understandings of gamification and draw comparisons to related approaches, including alternate reality games (ARGs), games with a purpose (GWAPs), and gameful design. We present a multidisciplinary review of gamification in action, focusing on empirical findings related to purpose and context, design of systems, approaches and techniques, and user impact. Findings from the survey show that a standard conceptualization of gamification is emerging against a growing backdrop of empirical participants-based research. However, definitional subjectivity, diverse or unstated theoretical foundations, incongruities among empirical findings, and inadequate experimental design remain matters of concern. We discuss how gamification may to be more usefully presented as a subset of a larger effort to improve the user experience of interactive systems through gameful design. We end by suggesting points of departure for continued empirical investigations of gamified practice and its effects.

Games are an important part of popular culture and have successfully engaged audiences from different age groups [8]. This has caused many educators to start thinking of different ways to infuse game-based techniques (playing or construction) to enhance education. Over the past five years, there has been several research efforts on curriculum development and the evaluation of game construction and playing methods to promote STEM (Science Technology, Engineering and Mathematics) concepts to middle and high school students. In this paper, we report on a new study following the lead of previous efforts. In particular, we were granted a block within Frank Hurt High School by the Surrey school district for which to develop a game construction-based curriculum as a basis for computer science education within the school, replacing a traditional computer science class. From February till June 2011 students were taught elementary programming using GameMaker, a digital game development tool. In addition to collecting students' overall impressions, we also evaluated students' technical competency (using tests) and self-efficacy (using surveys) at the start and end of the semester. Our findings show that the curriculum had a positive, statistically significant effect on concept comprehension, which suggests that the curriculum was effective for understanding computer science and game design concepts. In this paper we present the course structure, methods, and evaluation as a contribution.

Our mediated world is increasingly embracing tactile forms of interaction and information presentation in tandem with visual and auditory modes. Multimodal objects, from touch-screen devices, such as the iPad, to vibrotactile and kinaesthetic game controllers, such as the Nintendo Wiimote, are melding sensory modes and enhancing their communicative potential. While such objects have received much scholarly attention, particularly with respect to their haptic (invoking the sense of touch), gestural and kinaesthetic capacities and effects, little work has explored the visual presentation of texture in a haptically augmented medium. In this paper, I explore how texture is visually applied in the graphical user interface (GUI) of the iPhone, a touch-based mobile device. First, I will characterize the application of texture using Djonov and Van Leeuwen's (2011) six parameters for describing the visual-tactile quality of texture dimensions. I will then highlight and analyze how touch-based interaction with visual texture in the GUI establish and reinforce patterns of meaning, including texture understanding and conceptual mapping of tactile domains. I seek to contribute to the developing area of software semiotics through an analysis of a touch-based mobile device context.

In this paper I discuss how light and heat information can be designed to affect feedforward in a tangible puzzle game. In particular I focus on temperature, which has not been widely considered as an information modality in feedback or feedforward in tangibles. I describe a prototypal research instrument that implements both information modes in a tangible game interface. Finally, I outline a user study in which these modes are explored as feed-forward coaching devices. The expected outcomes of the proposed user study are an understanding of the role of temperature as a feedforward modality in tangible game interfaces and a set of design guidelines for designers of tangible games working with these physical characteristics.

Competency models have been widely employed within education, training and development contexts. In particular, medical education programs have become a platform of exploration around innovations in competency-based assessment. Approaches that employ dynamic, flexible models in a social context are now being sought. In this paper, we present the design of a deliberately gamifiable competency model system that encourages pre-professional development through collaborative peer appraisals. We propose a framework for incorporating social game mechanics into a competency modelling system and draw attention to how a collaborative approach to evidencing progress is a potentially fruitful solution to the threat of demotivation associated with the competitive slant of traditional leaderboards.

Psychological resilience has emerged as a key factor in mental health during the global COVID-19 pandemic. However, no work to date has synthesised findings across review work or assessed the reliability of findings based on review work quality, so as to inform public health policy. We thus conducted a meta-review on all types of review work from the start of the pandemic (January 2020) until the last search date (June 2021). Of an initial 281 papers, 30 were included for review characteristic reporting and 15 were of sufficient review quality for further inclusion in strategy analyses. High-level strategies were identified at the individual, community, organisational, and governmental levels. Several specific training and/or intervention programmes were also identified. However, the quality of findings was insufficient for drawing conclusions. A major gap between measuring the psychological resilience of populations and evaluating the effectiveness of strategies for those populations was revealed. More empirical work, especially randomised controlled trials with diverse populations and rigorous analyses, is strongly recommended for future research.

Gender/ing guides how we view ourselves, the world around us, and each other—including non-humans. Critical voices have raised the alarm about stereotyped gendering in the design of socially embodied artificial agents like voice assistants, conversational agents, and robots. Yet, little is known about how this plays out in research and to what extent. As a first step, we critically reviewed the case of Pepper, a gender-ambiguous humanoid robot. We conducted a systematic review (n=75) involving meta-synthesis and content analysis, examining how participants and researchers gendered Pepper through stated and unstated signifiers and pronoun usage. We found that ascriptions of Pepper's gender were inconsistent, limited, and at times discordant, with little evidence of conscious gendering and some indication of researcher influence on participant gendering. We offer six challenges driving the state of affairs and a practical framework coupled with a critical checklist for centering gender in research on artificial agents.