Mapping the Synesthetic Interface

Materializing Digital Futures Touch, Movement, Sound and Vision, 2022

Contemporary videogames make use of a diverse array of physical and graphical user interfaces (GUIs) with diverse mechanics and logics (Jorgensenn, 2013). As these new interfaces beckon, one might well wonder if we are still phenomenally limited, as Thomas Nagel (1974) once argued, to the 'resources' of our own minds. This chapter proposes that scholarship consider seriously how videogame aesthetics can articulate novel kinds of sense-perception at the edges of ludic and narrative experiences, and in forms distinct from previous scholarship concerned with experiential implications of VR (Helsel and Roth, 1991) or the phenomenology of physical interfaces (Keogh, 2018). Through a post-phenomenological understanding of atmospheres as ‘quasi-objective’ aesthetic objects (Griffero 2017; B hme, 2017) I argue that videogames offer us cyborg access to alien forms of experience, as well as new defamiliarized perspectives on human sensation.

This paper discusses affordance with respect to computer games and game play activity. The game environment, with its complex and seemingly multi- ple affordances, presents a challenge, for players as well as researchers, since it consists of two worlds: the virtual and the real one. For games to be played successfully, affordances of both worlds need to be integrated. However, since Gibson's The Ecological Approach to Visual Perception (1986), numerous different opinions on what constitutes an affordance have appeared, most of them deviating from the original Gibsonian conceptualisation. This has lead to confusion and misunderstandings among researchers, and is now also spreading into computer games research. This paper aims to raise awareness of what the affordance concept can, and cannot, explain and of the fact that some of the possible actions perceived by a player in a game also are rooted in socio-cultural conventions and the player's experience of having a body.

HCII 2020, LNCS 12199, 2020

To play is often known as a voluntary process in a temporary sphere with a unique disposition. In contemporaneity, games are a promising field for application and study, since they are sociocultural products with great impacts on the economy and in technology. Throughout the history of the development of this medium, the sensory experience included the perceptual channels of vision, hearing , and cognition to solve problems. Therefore, this study presents the theme through a systematic literature review and describes several applications and approaches that accentuate and diversify the user's sensory experience. The main AQ1 contribution of the study is to elucidate and illustrate the multisensory experiences that involve the use of the kinesthetic system (voice and body) and the haptic system (smell and taste). Moreover, visual experiences have been enhanced through augmented reality, virtual reality, and 3-D visualization. The results and discussion note the particularities of this media and aspects for the feasibility of future applications. It is hoped that this research will pave the way for future work involving case studies of games that allow for aesthetic experiences beyond solely visual experiences to deepen the sensitivity analysis and the process of generating meaning as well as emotional, and cognitive responses.

Analog Game Studies, 2019

This paper introduces visceral interaction, which we believe complements the vision of Perceptual User Interfaces (PUI) in creating more natural and intuitive interfaces. We describe a study that highlights the importance of recruiting user's natural abilities in physical manipulation so long as devices possess key features of physicality, in this case the 'natural inverse' property. The device, a Cubicle, controls a media application, but the mapping between the device and its effects were deliberately manipulated. Despite a lack of understanding of the mappings, the participants were able to react appropriately to feedback by merely focusing on the visceral interaction.

The popularity of Nintendo’s Wii is indicative of a fundamental change in the relationship between humans and computers. After decades of subservience to the keyboard and mouse, there is now a reawakening of our kinesthetic and proprioceptive senses. In this paper I am suggesting that the body be understood not merely as the controller of multimedia systems, but as a component within these systems and as a unique medium unto itself. I will demonstrate an application of this idea by describing a mixed reality system that was constructed to assist with the exploration of proprioceptive and kinesthetic awareness. The paper also includes a review of the informal experiments that I conducted using this system to explore possible kinesthetic and proprioceptive illusions.

This paper presents the results of a qualitative study of players' awareness of the mediation of gameplay by a hardware input device (gamepad). As a point of departure we adopt a definition of gameplay which places interfaces at the centre of the experience, but does not compromise the perception of other mediators and their interference in the process. We argue that indirect-mapping interfaces have their own advantages and can be the source of greater enjoyment in gameplay. Presentation of results is organized in 7 categories: software interface, movement in the gameworld, interaction with the gameworld, camera movement, materiality of the gamepad, corporeality of players and spatial awareness and perception. Results indicate that players remain aware of the hardware interfaces and of their own corporeality throughout gameplay. Rather than dematerialization and immersion, players experienced embodied presence and expansion of the gameworld towards the physical space.

HCI International 2011–Posters' Extended Abstracts, 2011

The present study has explored the expansibility of using everyday objects for game interface to provide additional multisensory stimuli. The perceived expansibility of game interfaces was investigated by measuring user various responses such as the degree of presence, immersion, and enjoyment. Offering actual sensations assumed in the virtual environment of game could enhance user experience by effectively accessing their emotions. Participants were provided with one of four output modes: basic condition, room light stimulation, chair vibration stimulation, & both light and vibration. Consequently, tactile stimulation has resulted in higher presence and immersion than visual did. The result also suggests that a gamer could experience the expanded interfaces with additional multisensory output, bringing more enjoyment. These tendencies are especially recognized for high immersive tendency group. These findings can be applied to various fields of game interface design.

2018

This project is driven by omissions at the intersection of ecological game studies and media-ecology. Although authors have studied videogames from a variety of ecological approaches, few have attempted to develop a holistic methodology, embracing videogames' specific attributes while recognising their role within larger physical systems. This thesis is an attempt to address this, reading videogames as simultaneously about and functioning as ecologies. My methodology draws on the agential-realist philosophy of Karen Barad whose theory of 'intra-activity' is abundant with ecological ramifications. Adapting Barad's 'intra-active' framework for use with contemporary videogames, I read them as assemblages of hardware, software and their human players. I explore three significant aspects of game studies: interaction, aesthetics and affect. Focusing on interaction, I analyse the game Shelter. Emphasising the role of hardware and software, I read these processes in ...