Emotion studies in HCI-a new approach

CHI Conference on Human Factors in Computing Systems Extended Abstracts

Emotion has been studied in HCI for two decades, with specifc traditions interested in sensing, expressing, transmitting, modelling, experiencing, visualizing, understanding, constructing, regulating, manipulating or adapting to emotion in human-human and humancomputer interactions. This CHI 2022 workshop on the Future of Emotion in Human-Computer Interaction brings together interested researchers to take stock of research on emotion in HCI to-date and to explore possible futures. Through group discussion and collaborative speculation we will address questions such as: What are the relationships between digital technology and human emotion? What roles does emotion play in HCI research? How should HCI researchers conceptualize emotion? When should HCI researchers use interdisciplinary theories of emotion or create new theory? Can specifc emotions be designed for, and where is this knowledge likely to be applied? What are the implications of emotion research for design, ethics and wellbeing? What is the future of emotion in human-computer interaction? CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI).

International Journal of Occupational Safety and Ergonomics Jose, 2003

Emotions are an increasingly important factor in Human-Computer Interaction (HCI). Up to the present, emotion recognition in HCI implies the use of explicit or intrusive methods, e.g. video cameras or physiological measurements. We develop and evaluate a method for the measurement of affective states through motor-behavioral parameters from the standard input devices mouse and keyboard.

2012 IEEE International Conference on Fuzzy Systems, 2012

Fusion and rejection 6.4 Results 6.4.1 Participants reports and protocol validation 6.4.2 Results of single classifiers 6.4.3 Results of feature selection 6.4.4 Results of fusion 6.4.5 Results of rejection 6.5 Conclusions Chapter 7 Assessment of emotions for computer games 7.1 Introduction: the flow theory for games 7.2 Data acquisition 7.2.1 Acquisition protocol 7.2.2 Feature extraction 7.3 Analysis of questionnaires and of physiological features 7.3.1 Elicited emotions 7.3.2 Evolution of emotions in engaged trials 7.4 Classification of the gaming conditions using physiological signals 7.4.1 Classification methods 7.4.2 Peripheral signals 7.4.3 EEG signals 7.4.4 EEG and peripheral signals 7.4.5 Fusion 7.5 Analysis of game-over events 7.5.1 Method 7.5.2 Results 7.6 Conclusion x Chapter 8 Conclusions 8.1 Outcomes 8.2 Future prospects Appendix A Consent form Appendix B Neighborhood table for the Laplacian filter Appendix C List of IAPS images used Appendix D Questionnaire results for the game protocol Appendix E Publications References List of figures List of tables xii Acronyms ANOVA ANalysis Of VAriance ANS Autonomic Nervous System BCI Brain-Computer Interface Chapter 1 In order to obtain such an "emotional machine" it is necessary to include emotions in the humanmachine communication loop (see Figure 1.1). This is what is defined as "affective computing". According to Picard [2], affective computing "proposes to give computers the ability to recognize [and] express […] emotions". Synthetic expression of emotions can be achieved by enabling avatars or simpler agents to have facial expressions, different tones of voice, and empathic behaviors [3, 4]. Detection of human emotions can be realized by monitoring and interpreting the different cues that are given in both verbal and non-verbal communication. However, a system that is "emotionally intelligent" should not only detect and express emotions but it should also take the proper action in response to the detected emotion. Expressing the adequate emotion is thus one of the outputs of this decision making. The proper action to take is dependent on the application but examples of reactions could be to provide help in the case the user feels helpless or to lower task demand in the case he / she is highly stressed. Many applications are detailed in chapter 1.2.2. Accurately assessing emotions is thus a critical step toward affective computing since this will determine the reaction of the system. The present work will focus on this first step of affective computing by trying to reliably assess emotion from several emotional cues. Figure 1.1. Including emotions in the human-machine loop 1.2 Emotion assessment 1.2.1 Multimodal expression of emotion A modality is defined as a path used to carry information for the purpose of interaction. In HMI, there are two possible approaches to define a modality: from the machine point of view and from the user point of view. On the machine side, a modality refers to the processes that generate information to the physical world and interpret information from it. It is thus possible to distinguish between input and output modalities and to associate them to the corresponding communication device. A keyboard, a mouse and a pad with the associated information processes are typical input modalities, while a text, an image and music presented on screens and speakers Emotional cues Senses Emotion assessment Decision of proper reaction Sensors User Emotion synthesis Ouput devices Machine Command execution Emotional adaptation Decision of proper reaction Emotion synthesis Evaluation Perception Interpretation Outcome evaluation Emotional evaluation / elicitation Execution Intentions formulation Actions specification Execution of actions Goals Emotional cues Machine User Games are also interesting from a HMI point of view because they are an ideal ground for the design of new ways to communicate with the machine. One of the main goals of games is to provide emotional experiences such as fun and excitement which generally occurs when the player is strongly involved in the course of the game action. However, a loss of involvement can occur if the game does not correspond to the player's expectations and competences; he might measuring various physiological signals requires the use of several sensors that are sometimes quite obtrusive since they can monopolize the use of one hand and are not comfortable. The price of those sensors should also to be taken into account. For these reasons the issue of determining the most useful sensors is of importance. Finally, there is also variability in physiological signals, from person to person but also from day to day, that yield to difficulty in designing a system that Reference Basic emotions Criteria Ekman [42] Anger,

Multimedia Tools and Applications, 2016

Measuring users' emotional reaction to interactive multimedia and hypermedia is important. One particularly popular self-reported method for emotion assessment is the Valence-Arousal (VA) Scale: a 9x9 affective grid. This paper aims to identify specific stress region(s) in the VA space by combining self-reported ratings (pairs of VA) and physiological signals (skin conductance). To this end, 31 healthy volunteers participated in an experiment by performing five stressful interaction tasks while their skin conductance was monitored. The selected interaction tasks were most frequently listed as stressful by a separate group of 15 interviewees. After each task, participants expressed their perceived emotional experience using the VA rating space. Our findings show which regions in the VA rating space may reliably indicate self-reported stress that is in alignment with one's measured skin conductance while using interactive applications. One additional important contribution of this work is the proposed approach for the empirical identification of affect regions in the VA space based on physiological signals.

International Journal of Human-Computer Studies, 2003

The past decade has witnessed an unprecedented growth in user interface and humancomputer interaction (HCI) technologies and methods. The synergy of technological and methodological progress on the one hand, and changing user expectations on the other, are contributing to a redefinition of the requirements for effective and desirable human-computer interaction. A key component of these emerging requirements, and of effective HCI in general, is the ability of these emerging systems to address user affect. The objective of this special issue is to provide an introduction to the emerging research area of affective HCI, some of the available methods and techniques, and representative systems and applications.

2000

Despite the rapid advancement of computer technology, computers remain incapable of understanding human emotion. As a result, users have often been unaided for their aversive emotion that may take place during their computer tasks. This may be detrimental to positive and productive interactions between users and computers. This paper reviews some empirical studies regarding the effect of emotion on computer work and conceptualizes what constitutes an emotional computer. It is proposed that the emotional computer be designed to understand human emotion and adapt its interface accordingly. This paper raises a number of research questions in relation to such issues as measurement (e.g., automatic detection of human emotion, time delay), signal processing (e.g., accuracy) and user interfaces (e.g., ways to alleviate the intensity of negative emotion). Considering that there has been very little research on the design and aftermath of emotional computers, further studies are urgently needed.

Peter, C., Crane, E., Fabri, M., Agius, H., Axelrod, L. (eds.) (2010) Emotion in HCI - Designing for people: Proceedings of the 2008 International Workshop, Fraunhofer Verlag, Stuttgart, ISBN 978-3-8396-0089-4

As computing is changing and becoming increasingly social in nature, the role of emotions in computing has become ever more relevant and commercial. Emotion are central to culture, creativity, and interaction. The topic attracts more and more researchers from a range of multidisciplinary fields including design, gaming, sensor technologies, psychology and sociology. The need for discussion, exchange of ideas, and interdisciplinary collaboration is ever-increasing as the community grows. This workshop will meet requirements of individuals working in the field, giving them a podium to explore different aspects of emotion in HCI, raise questions and network with like-minded people on common subjects. The workshop will focus around working group sessions, and will use predominantly small group work, rather than being presentation-based.

2009

Emotion is an important mental and physiological state, influencing cognition, perception, learning, communication, decision making, etc. It is considered as a definitive important aspect of user experience (UX), although at least well developed and most of all lacking experimental evidence. This paper deals with an application for emotion detection in usability testing of software. It describes the approach to utilize the valence arousal space for emotion modeling in a formal experiment. Our study revealed correlations between low performance and negative emotional states. Reliable emotion detection in usability tests will help to prevent negative emotions and attitudes in the final products. This can be a great advantage to enhance Universal Access.

Lecture Notes in Computer Science, 2014

Human emotion is a psycho-physiological state in most cases not obvious to the subject. Different permutations of emotional constituents sometimes cause similar outward expressions; therefore facial expression methods cannot achieve reliable estimates. Sensing physiological manifestations of hormonal and neural stimulations instigated by emotion and affect is widely accepted as a credible method of detecting psycho-physiological states. A major impediment in interactive environments employing physiological sensing affecting the credibility of measurements is the physical and psychological impairment caused by electrodes and wiring used for the acquisition of signals. In the system described in this paper, the above obstacle has been overcome. Physiological signals acquired via an in-house developed computer mouse and coinciding physiological patterns were investigated as reactions to emotion raising events. A classification algorithm analyzed herein produced a real time allocation model of states of engagement. Experiments have revealed strong correlations between events and respective emotional states.

This paper has discussed the observations of various emotions which are invoked in the human computer interface and how these emotions affect the decisions of the users online. This paper has also discussed usability test methods in the human computer interface design and its relation with the usability of the interfaces. Also some principles are put forward which should be implemented for the development of better interfaces.