Fraser Allison

University of Melbourne, Computing and Information Systems, Post-Doc

Followers

Following

Public Views

NOTE: I do not keep this profile up to date. See my staff page at the University of Melbourne for a more complete profile: https://findanexpert.unimelb.edu.au/profile/764723-fraser-allison

less

Uploads

Papers by Fraser Allison

Understanding the Utility of Digital Flight Assistants: A Preliminary Analysis

CUI 2021 - 3rd Conference on Conversational User Interfaces

Interactions with intelligent systems have become common in domestic and professional life. Howev... more Interactions with intelligent systems have become common in domestic and professional life. However, little is known about how pilots, who already work in a highly automated environment, envisage using intelligent systems in their work environment. This preliminary analysis investigates pilots' needs and wants for digital flight assistants (DFAs) through an interview study. We show that the adoption of DFAs may be hindered by pre-existing concerns, such as inadequate automatic speech recognition, linked to past experiences with digital assistants. Furthermore, we identify important contextual and environmental factors that will need to be accounted for in the design of DFAs such as "cross cockpit" relationships, noisy environments, or pilot's cognitive workload. CCS CONCEPTS • Hardware → Emerging interfaces; • Human-centered computing → Natural language interfaces.

Download

A History of Voice Interaction in Games

INTRODUCTION For more than forty years, voice interaction in gaming has been a mirage: a vision f... more INTRODUCTION For more than forty years, voice interaction in gaming has been a mirage: a vision for how games of the future will play that has never quite arrived. Voice has been heralded as a “natural” mode of interaction (Aylett et al. 2014, 749), unbounded by the limitations of screen space and controller ergonomics. Yet in practice, it is more often described as “unnatural” and “uncomfortable” (Carter et al. 2015). Although the majority of contemporary game platforms have the technological capability to handle speech recognition, voice interaction remains a marginal and under-studied modality in games.

Download

Enabling Creative Crowd Work through Smart Speakers

Digital voice assistants or smart speakers have rapidly changed the landscape of voice user inter... more Digital voice assistants or smart speakers have rapidly changed the landscape of voice user interfaces over the past few years. In this paper we discuss how we could utilise the affordances of these devices to create a novel crowdsourcing platform that deliver crowd tasks through voice with particular focus on creative tasks. We describe the potential benefits and challenges of using this technology for these purposes, and outline our future work in this research area

Download

Whose mind is the signal? Focalization in video game narratives

In this paper, I explore instances in which video games convey an experience of subjectivity, uti... more In this paper, I explore instances in which video games convey an experience of subjectivity, utilizing an appropriation of Gerard Genette’s (1980) concept of focalization. Through an analysis of The Sims 3 (The Sims Studio 2009), Top Spin 4 (2K Czech 2011), Mirror’s Edge (EA Digital Illusions CE 2008) and Grand Theft Auto V (Rockstar North 2014), I demonstrate that internal focalization can be made apparent in a video game through its audiovisual presentation, its selection and restriction of private knowledge, and its ludic affordances. I provide a framework for analyzing games that seek to present a diversity of perspectives, and to allow players to access modes of thinking that accord with a perspective other than their own. This framework can assist researchers, critics, and designers to identify ways in which video games express elements of internal focalization that communicate the mental patterns of a perspective character.

Download

Nonverbal Communication in Human-AI Interaction: Opportunities and Challenges

In recent years, we have explored the use of gaze—an important nonverbal communication signal and... more In recent years, we have explored the use of gaze—an important nonverbal communication signal and cue in everyday human-human interaction—for use with AI systems. Specifically, our work investigated whether an artificial agent, given the ability to observe human gaze, can make inferences on intentions, and how aspects of these inferences can be communicated to a human collaborator. We leveraged a range of humancomputer interaction techniques to inform the design of a gaze-enabled artificial agent that can predict and communicate predictions. In this paper, we include a snapshot of how AI and HCI can be brought together to inform the design of an explainable interface for an artificial agent. To conclude, we outline the challenges we faced when designing AI systems that incorporate nonverbal communication stemming from our work.

Download

Voice interaction game design and gameplay

This thesis is concerned with the phenomenon of voice-operated interaction with characters and en... more This thesis is concerned with the phenomenon of voice-operated interaction with characters and environments in videogames. Voice interaction with virtual characters has become common in recent years, due to the proliferation of conversational user interfaces that respond to speech or text input through the persona of an intelligent personal assistant. Previous studies have shown that users experience a strong sense of social presence when speaking aloud to a virtual character, and that voice interaction can facilitate playful, social and imaginative experiences of the type that are often experienced when playing a videogame. Despite this, the user experience of voice interaction is frequently marred by frustration, embarrassment and unmet expectations. The aim of this thesis is to understand how voice interaction can be used in videogames to support more enjoyable and meaningful player experiences. Voice-operated videogames have existed for more than three decades, yet little resear...

Fear, loss and meaningful play: Permadeath in DayZ

Designing Interactions with Intention-Aware Gaze-Enabled Artificial Agents

Human-Computer Interaction – INTERACT 2019

As it becomes more common for humans to work alongside artificial agents on everyday tasks, it is... more As it becomes more common for humans to work alongside artificial agents on everyday tasks, it is increasingly important to design artificial agents that can understand and interact with their human counterparts naturally. We posit that an effective way to do this is to harness nonverbal cues used in human-human interaction. We, therefore, leverage knowledge from existing work on gaze-based intention recognition, where the awareness of gaze can provide insights into the future actions of an observed human subject. In this paper, we design and evaluate the use of a proactive intention-aware gaze-enabled artificial agent that assists a human player engaged in an online strategy game. The agent assists by recognising and communicating the intentions of a human opponent in real-time, potentially improving situation awareness. Our first study identifies the language requirements for the artificial agent to communicate the opponent’s intentions to the assisted player, using an inverted Wizard of Oz method approach. Our second study compares the experience of playing an online strategy game with and without the assistance of the agent. Specifically, we conducted a within-subjects study with 30 participants to compare their experience of playing with (1) detailed AI predictions, (2) abstract AI predictions, and (3) no AI predictions but with a live visualisation of their opponent’s gaze. Our results show that the agent can facilitate awareness of another user’s intentions without adding visual distraction to the interface; however, the cognitive workload was similar across all three conditions, suggesting that the manner in which the agent communicates its predictions requires further exploration. Overall, our work contributes to the understanding of how to support human-agent teams in a dynamic collaboration scenario. We provide a positive account of humans interacting with an intention-aware artificial agent afforded by gaze input, which presents immediate opportunities for improving interactions between the counterparts.

Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft

Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

An increasing number of our technological interactions are mediated through virtually embodied ch... more An increasing number of our technological interactions are mediated through virtually embodied characters and software agents powered by machine learning. To understand how users relate to and evaluate these types of interfaces, we designed a Wizard of Oz prototype of an embodied agent in Minecraft that learns from users' actions, and conducted a user study with 18 school-aged Minecraft players. We categorised nine main ways users spontaneously attempted to interact with and teach the agent: four using game controls, and five using natural language text input. This study lays groundwork for a better understanding of human interaction with learning agents in virtual worlds.

Download

Frame Analysis of Voice Interaction Gameplay

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Voice control is an increasingly common feature of digital games, but the experience of playing w... more Voice control is an increasingly common feature of digital games, but the experience of playing with voice control is often hampered by feelings of embarrassment and dissonance. Past research has recognised these tensions, but has not offered a general model of how they arise and how players respond to them. In this study, we use Erving Goffman's frame analysis, as adapted to the study of games by Conway and Trevillian, to understand the social experience of playing games by voice. Based on 24 interviews with participants who played voice-controlled games in a social setting, we put forward a frame analytic model of gameplay as a social event, along with seven themes that describe how voice interaction enhances or disrupts the player experience. Our results demonstrate the utility of frame analysis for understanding social dissonance in voice interaction gameplay, and point to practical considerations for designers to improve engagement with voice-controlled games.

Design Patterns for Voice Interaction in Games

Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play

Voice interaction is increasingly common in digital games, but it remains a notoriously difficult... more Voice interaction is increasingly common in digital games, but it remains a notoriously difficult modality to design a satisfying experience for. This is partly due to limitations of speech recognition technology, and partly due to the inherent awkwardness we feel when performing some voice actions. We present a pattern language for voice interaction elements in games, to help game makers explore and describe common approaches to this design challenge. We define 25 design patterns, based on a survey of 449 videogames and 22 audiogames that use the player's voice as an input to affect the game state. The patterns express how games frame and structure voice input, and how voice input is used for selection, navigation, control and performance actions. Finally, we argue that academic research has been overly concentrated on a single one of these design patterns, due to an instrumental research focus and a lack of interest in the fictive dimension of videogames.

Cybernetic Funeral Systems

2021 IEEE Conference on Norbert Wiener in the 21st Century (21CW)

Using Postphenomenology (one of many methods informed by Wiener's cybernetics) as an analytic... more Using Postphenomenology (one of many methods informed by Wiener's cybernetics) as an analytical approach, this paper examines three examples of robot participation in, and mediation of, funerals. The analysis of robot mediation of funerals challenges the idea that death rituals are exclusively human performances and experiences, and instead repositions them as cybernetic systems of entanglement and impact. The paper begins with an introduction to the relevance of postphenomenological theory, then moves to the case of CARL, a robot that enables remote participation in funeral ceremonies. We argue that the [Human–Robot–Funeral] relation and its variants are both engaging and alienating, through revealing-concealing, magnification-reduction and a more generalised enabling-constraining. Technological mediation is also evident in the case of Pepper, a robot that has officiated at funerals as a Buddhist monk. We describe similarities and differences in the way CARL and Pepper manifest the [Human–Robot–Funeral] relation. The final example is AIBO, a companion robot that becomes the locus of a funeral ritual. This offers a radical case that directly challenges humans' self-proclaimed exceptional ontology.

Evaluating Real-Time Gaze Representations to Infer Intentions in Competitive Turn-Based Strategy Games

Proceedings of the Annual Symposium on Computer-Human Interaction in Play

In this paper, we investigate nine different visual representations of gaze in a competitive digi... more In this paper, we investigate nine different visual representations of gaze in a competitive digital game setting. We evaluate the ability of spectators to infer a player's intentions in the game for each visual representation. Our results show that spectators have a remarkable ability to infer intent accurately using all nine visualizations, but that visualizations with certain characteristics were more comprehensible and more readily revealed the player's intent. The real-time Heatmap visualization was the most highly preferred by participants and the most effective in revealing intent, due to its ability to balance real-time gaze information with a persistent summary of recent gaze behaviour. Our findings show that eye-tracking visualization can enable playful interactions in competitive games based on players' ability to interpret opponents' attention and intention through gaze information.

Download

Looks Can Be Deceiving

Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

In competitive co-located gameplay, players use their opponents' gaze to make predictions abo... more In competitive co-located gameplay, players use their opponents' gaze to make predictions about their plans while simultaneously managing their own gaze to avoid giving away their plans. This socially competitive dimension is lacking in most online games, where players are out of sight of each other. We conducted a lab study using a strategic online game; finding that (1) players are better at discerning their opponent's plans when shown a live visualisation of the opponent's gaze, and (2) players who are aware that their gaze is tracked will manipulate their gaze to keep their intentions hidden. We describe the strategies that players employed, to various degrees of success, to deceive their opponent through their gaze behaviour. This gaze-based deception adds an effortful and challenging aspect to the competition. Lastly, we discuss the various implications of our findings and its applicability for future game design.

Hi! I am the Crowd Tasker" Crowdsourcing through Digital Voice Assistants

Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Inspired by the increasing prevalence of digital voice assistants, we demonstrate the feasibility... more Inspired by the increasing prevalence of digital voice assistants, we demonstrate the feasibility of using voice interfaces to deploy and complete crowd tasks. We have developed Crowd Tasker, a novel system that delivers crowd tasks through a digital voice assistant. In a lab study, we validate our proof-ofconcept and show that crowd task performance through a voice assistant is comparable to that of a web interface for voicecompatible and voice-based crowd tasks for native English speakers. We also report on a field study where participants used our system in their homes. We find that crowdsourcing through voice can provide greater flexibility to crowd workers by allowing them to work in brief sessions, enabling multitasking, and reducing the time and effort required to initiate tasks. We conclude by proposing a set of design guidelines for the creation of crowd tasks for voice and the development of future voice-based crowdsourcing systems.

Download

Understanding the Utility of Digital Flight Assistants: A Preliminary Analysis

CUI 2021 - 3rd Conference on Conversational User Interfaces, 2021

Word Play

Games and Culture

The use of voice interaction in digital games has a long and varied history of experimentation bu... more The use of voice interaction in digital games has a long and varied history of experimentation but has never achieved sustained, widespread success. In this article, we review the history of voice interaction in digital games from a media archaeology perspective. Through detailed examination of publicly available information, we have identified and classified all games that feature some form of voice interaction and have received a public release. Our analysis shows that the use of voice interaction in digital games has followed a tidal pattern: rising and falling in seven distinct phases in response to new platforms and enabling technologies. We note characteristic differences in the way Japanese and Western game developers have used voice interaction to create different types of relationships between players and in-game characters. Finally, we discuss the implications for game design and scholarship in light of the increasing ubiquity of voice interaction systems.

How Players Speak to an Intelligent Game Character Using Natural Language Messages

Transactions of the Digital Games Research Association

AI-driven characters that learn directly from human input are rare in digital games, but recent a... more AI-driven characters that learn directly from human input are rare in digital games, but recent advances in several fields of machine learning suggests that they may soon be much more feasible to create. This study explores the design space for interacting with such a character through natural language text dialogue. We conducted an observational study with 18 high school students, who played Minecraft alongside a Wizard of Oz prototype of a companion AI character that learned from their actions and inputs. In this paper, we report on an analysis of the 186 natural language messages that players sent to the character, and review key variations in syntax, function and writing style. We find that players’ behaviour and language was differentiated by the extent to which they expressed an anthropomorphic view of the AI character and the level of interest that they showed in interacting with it.

Download

Are You Out of Your Mind? Focalization in Digital Games

Transactions of the Digital Games Research Association

This paper demonstrates how the concept of focalization, as defined by Gérard Genette (1980), can... more This paper demonstrates how the concept of focalization, as defined by Gérard Genette (1980), can be used to analyze experiences of subjectivityin digital games. Strategies to create internal focalization are identifiedin games’ audiovisual presentation, provision and restriction of private knowledge, and ludic affordances. This provides a framework for games that seek to present diverse or distinctive perspectives, to allow players to access modes of thinking that accord with a mind other than their own. This framework can assist researchers, critics, and designers to identify ways in which digital games express elements of internal focalization that communicate the mental patterns of a perspective character.

Download

Player Identity Dissonance and Voice Interaction in Games

Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '15, 2015

In the past half-decade, advances in voice recognition technology and the proliferation of consum... more In the past half-decade, advances in voice recognition technology and the proliferation of consumer devices like the Microsoft Kinect have seen a significant rise in the use of voice interaction in games. While the use of player-to-player voice is widespread and well-researched, the use of voice as an input is relatively unexplored. In this paper we make the argument that notions of player and avatar identity are inextricable from the successful implementation of voice interaction in games, and consequently identify opportunities for future research and design.

Fraser Allison

Uploads

Papers by Fraser Allison

Log In