Papers by Patrick Gebhard
Understanding Shame Signals: Functions of Smile and Laughter in the Context of Shame
Computational emotion recognition focuses on observable expressions. In the case of highly unplea... more Computational emotion recognition focuses on observable expressions. In the case of highly unpleasant emotions that are rarely displayed openly and mostly unconsciously regulated - such as shame - this approach can be difficult. In previous studies, we found participants to smile and laugh while experiencing shame. Most current approaches interpret smiles and laughter as signals of enjoyment. They neglect the internal emotional experience and the complexity of social signals. We present a planned mixed-methods study that will investigate underlying functions of smiles and laughter in shameful situations and how those reflect in the morphology of expression. Participants' smiles and laughter during shame-eliciting situations will be analyzed using behavioral observations. Semi-structured interviews will investigate their functions. The gained knowledge can improve computational emotion recognition and avoid misinterpretations of smiles and laughter. In the scope of the open science initiative, we describe the planned study in detail with its research questions, hypotheses, design, methods, and analyses.
Companion Publication of the 2020 International Conference on Multimodal Interaction, Oct 25, 2020
Mental stress is the psychological and physiological response to a high frequency of or continuou... more Mental stress is the psychological and physiological response to a high frequency of or continuous stressors. If prolonged and not regulated successfully, it has a negative impact on health. Developing stress coping techniques, as an emotion regulation strategy, is a crucial part of most therapeutic interventions. Interactive biofeedback agents can be employed as a digital health tool for therapists to let patients train and develop stress-coping strategies. This paper presents an interactive stress management training system using biofeedback derived from the heart rate variability (HRV), with an Interactive Social Agent as an autonomous biofeedback trainer. First evaluations have shown promising results. • Human-centered computing → Human computer interaction (HCI).
Stress Management Training using Biofeedback guided by Social Agents
Coping with stress is critical to mental health. Prolonged mental stress is the psychological and... more Coping with stress is critical to mental health. Prolonged mental stress is the psychological and physiological response to a high frequency of or continuous stressors, which has a negative impact on health. This paper presents a virtual stress management training using biofeedback derived from the cardiovascular response of the heart rate variability (HRV) with an interactive social agent as biofeedback trainer. The evaluation includes both, a subject-matter expert interview and an experiment with 71 participants. In the experiment, we compared our novel stress management training to a stress management training using stress diaries. The results indicate that our social agent-based stress management training using biofeedback significantly decreased the self-assessed stress levels immediately after the training, as well as in a socially stressful task. Moreover, we found a significant correlation between stress level and the assessment of one’s performance in a socially stressful task. Participants that received our training assessed their performance higher than participants getting stress diaries. Taken this together, our novel virtual stress management training with an interactive social agent as a trainer can be evaluated as a valid method for learning techniques on how to cope with stressful situations.
Understanding Shame Signals: Functions of Smile and Laughter in the Context of Shame
2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2021
Computational emotion recognition focuses on observable expressions. In the case of highly unplea... more Computational emotion recognition focuses on observable expressions. In the case of highly unpleasant emotions that are rarely displayed openly and mostly unconsciously regulated - such as shame - this approach can be difficult. In previous studies, we found participants to smile and laugh while experiencing shame. Most current approaches interpret smiles and laughter as signals of enjoyment. They neglect the internal emotional experience and the complexity of social signals. We present a planned mixed-methods study that will investigate underlying functions of smiles and laughter in shameful situations and how those reflect in the morphology of expression. Participants' smiles and laughter during shame-eliciting situations will be analyzed using behavioral observations. Semi-structured interviews will investigate their functions. The gained knowledge can improve computational emotion recognition and avoid misinterpretations of smiles and laughter. In the scope of the open science initiative, we describe the planned study in detail with its research questions, hypotheses, design, methods, and analyses.
Understanding emotions of others requires a theory of mind approach providing knowledge of intern... more Understanding emotions of others requires a theory of mind approach providing knowledge of internal appraisal and regulation processes of emotions. Multi-modal social signal classification is insufficient for understanding emotional expressions. Mainly, because many communicative, emotional expressions are not directly related to internal emotional states. Moreover, the recognition of the expression's direction is neglected so far. Even if social signals reveal emotional aspects, the recognition with signal classifiers cannot explain internal appraisal or regulation processes. Using that information is one approach for building cognitive empathic agents with the ability to address observations and motives in an empathic dialogue. In this paper, we introduce a computational model of user emotions for empathic agents. It combines a simulation of appraisal and regulation processes with a social signal interpretation taking directions of expressions into account. Our evaluation show...
Empirical Research in Affective Computing: An Analysis of Research Practices and Recommendations
2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII), 2021
In the last decade, empirical sciences have faced a tremendous change in the way of conducting re... more In the last decade, empirical sciences have faced a tremendous change in the way of conducting research. As a broad interdisciplinary field, research in Affective Computing often employs empirical user studies. The current paper analyzes research practices in Affective Computing and deduces recommendations for improving the quality of methods and reporting. We extracted a total of k = 65 empirical studies from the two most recent International Conferences on Affective Computing & Intelligent Interaction (ACII) ’17 and ’19. Three raters summarized characteristics of studies (e.g., number of experimental studies) and how much methodological (e.g., participant characteristics) and statistical information (e.g., degrees of freedom) were missing. Also, we conducted a p-curve analysis to test the overall evidential value of findings. Results showed that 1. in at least half of the studies, one important information about statistical results was missing, and 2. those k = 31 studies that had reported all necessary information to be included into the p-curve showed evidential value. In general, all criteria were never met in one single study. We provide concrete recommendations on how to implement open research practices for empirical studies in Affective Computing.
Companion Publication of the 2020 International Conference on Multimodal Interaction, 2020
Mental stress is the psychological and physiological response to a high frequency of or continuou... more Mental stress is the psychological and physiological response to a high frequency of or continuous stressors. If prolonged and not regulated successfully, it has a negative impact on health. Developing stress coping techniques, as an emotion regulation strategy, is a crucial part of most therapeutic interventions. Interactive biofeedback agents can be employed as a digital health tool for therapists to let patients train and develop stress-coping strategies. This paper presents an interactive stress management training system using biofeedback derived from the heart rate variability (HRV), with an Interactive Social Agent as an autonomous biofeedback trainer. First evaluations have shown promising results. • Human-centered computing → Human computer interaction (HCI).
2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), 2019
Virtual agents play an important role when we interact with machines. They are in the role of ass... more Virtual agents play an important role when we interact with machines. They are in the role of assistants or companions with less or more human-like appearance. Such agents influence our behavior. With an increasing and broader distribution, their influence might become stronger, and at some point, they might even adopt roles with a degree of authority. This paper presents the results of a study that examines the obedience of human users towards a) an embodied virtual agent in the role of an instructor and b) a human in the role of an instructor. Under a cover-story of a creativity test, participants should fulfill stressful and shameful tasks. Our results indicate that the embodied virtual agent has the same authority as the human instructor. The agent is also able to elicit the same level of the negative feelings stress and shame.
A human-driven control architecture for promoting good mental health in collaborative robot scenarios
2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), 2021
This paper introduces the control architecture of a platform aimed at promoting good mental healt... more This paper introduces the control architecture of a platform aimed at promoting good mental health for workers interacting with collaborative robots (cobots). The platform aim is to render industrial production cells capable of automatically adapting their behavior in order to improve the operator’s quality of experience and level of engagement and to minimize his/her psychological strain. In order to achieve such a goal, an extremely rich and complex framework is required. Starting from the identification of the parameters that could influence the collaboration experience, the envisioned human- driven control structure is presented together with a detailed description of the components required to implement such an automated system. Future works will include proper tuning of control parameters with dedicated experimental sessions, together with the definition of organizational and technical guidelines for the design of a mental-health-friendly cobot-based manufacturing workplace.
International Journal of Virtual Reality, 2020
In this paper we report about the use of computer generated affect to control body and mind of co... more In this paper we report about the use of computer generated affect to control body and mind of cognitively modeled virtual characters. We use the computational model of affect ALMA that is able to simulate three different affect types in real-time. The computation of affect is based on a novel approach of an appraisal language. Both the use of elements of the appraisal language and the simulation of different affect types has been evaluated. Affect is used to control facial expressions, facial complexions, affective animations, posture, and idle behavior on the body layer and the selection of dialogue strategies on the mind layer. To enable a fine-grained control of these aspects a Player Markup Language (PML) has been developed. The PML is player-independent and allows a sophisticated control of character actions coordinated by high-level temporal constraints. An Action Encoder module maps the output of ALMA to PML actions using affect display rules. These actions drive the real-ti...
Affective Interactions, 2000
A growing number of research projects in academia and industry have recently started to develop l... more A growing number of research projects in academia and industry have recently started to develop lifelike agents as a new metaphor for highly personalised human-machine communication. A strong argument in favour of using such characters in the interface is the fact that they make humancomputer interaction more enjoyable and allow for communication styles common in human-human dialogue. Our earlier work in this area has concentrated on the development of animated presenters that show, explain, and verbally comment textual and graphical output on a window-based interface. Even though first empirical studies have been very encouraging and revealed a strong affective impact of our Personas , they also suggest that simply embodying an interface agent is insufficient. To come across as believable, an agent needs to incorporate a deeper model of personality and emotions, and in particular directly connect these two concepts.
Close Engagements with Artificial Companions, 2010
Recently, a number of research projects have been started to create virtual agents that do not ju... more Recently, a number of research projects have been started to create virtual agents that do not just serve as assistants to which tasks may be delegated, but that may even take on the role of a companion. Such agents require a great deal of social intelligence, such as the ability to detect the user's affective state and to respond to it in an empathic manner. The objective of our work is to create an empathetic listener that is capable to react on affective and attentive input cues of the user. In particular, we discuss various forms of empathy and how they may be realized based on these cues.
Exploiting models of personality and emotions to control the behavior of animated interactive agents
… on Autonomous Agents, 2000
The German Research Centre for Artificial Intelligence (DFKI) recently started three new projects... more The German Research Centre for Artificial Intelligence (DFKI) recently started three new projects1 to advance our understanding of the fundamental technology required to drive the social behaviour of interactive animated agents. This initiative has been timed to catch the ...
Proceedings of the 8th International …, 2008
This paper describes a system to enhance the interaction between humans and virtual characters wi... more This paper describes a system to enhance the interaction between humans and virtual characters with emotional mimicry and role-taking. Such system increases the believability of virtual agents. Mimicking necessitates a model of emotional intelligence to understand and display user's emotions. A more complex processing is however necessary for a reactive behavior, where the virtual agent reacts eg in an encouraging way which allows to actively change user's current state. A virtual character with highly expressive ...
Lecture Notes in Computer Science, 2006
This paper presents results of the plausibility evaluation of computergenerated emotions and mood... more This paper presents results of the plausibility evaluation of computergenerated emotions and moods. They are generated by ALMA (A Layered Model of Affect), a real-time computational model of affect, designed to serve as a modular extension for virtual humans. By a unique integration of psychological models of affect, it provides three major affect types: emotions, moods and personality that cover short, medium, and long term affect. The evaluation of computer-generated affect is based on textual dialog situations in which at least two characters are interacting with each other. In this setup, elicited emotions or the change of mood are defined as consequences of dialog contributions from the involved characters. The results indicate that ALMA provides authentic believable emotions and moods. They can be used for modules that control cognitive processes and physical behavior of virtual humans in order to improve their lifelikeness and their believable qualities.
Uploads
Papers by Patrick Gebhard