Perception of Emotions Using Constructive Learning through Speech

Learner performances depend on the external conditions or circumstances that exist when learning occurs. Recent researches have shown that emotions play a powerful role in cognitive processes. In traditional learning environments, the teacher attempts to ensure the best conditions for learning and especially the emotional conditions. For instance, he introduces a joke in the classroom, in order to change the current emotions of the learners, if they are bored or frustrated. Whereas the existing Intelligent Tutoring Systems are lacked of considering the emotional conditions of learning that should exist in order to motivate the learners and to facilitate the achievement of learning.

2009

Emotions are assumed to have a great impact on our behaviour and also on our learning behaviour. In a face-to-face learning environment emotions can be expressed by (non-)verbal behaviour as way of speaking, facial expressions or words with an emotional loadings. In this paper we research the possibility to assess the emotional state of learners by analysis of nonverbal behaviour as speech analysis and analysis of facial expressions. We developed tools to extract features from sound and video recordings and used classifiers as SVM to label emotional states. We used discrete emotional states but also the well-known 2D valence and arousal score as a continuous score of the emotional state. From our experiments it proves that students show overt emotions under special conditions with strong emotional triggers. Our system was able to assess strong emotions up to some level.

Springer LNCS, 2014

This paper introduces the voice emotion recognition part of our framework for improving learning through webcams and microphones (FILTWAM). This framework enables multimodal emotion recognition of learners during game-based learning. The main goal of this study is to validate the use of microphone data for a real-time and adequate interpretation of vocal expressions into emotional states were the software is calibrated with end users. FILTWAM already incorporates a valid face emotion recognition module and is extended with a voice emotion recognition module. This extension aims to provide relevant and timely feedback based upon learner's vocal intonations. The feedback is expected to enhance learner’s awareness of his or her own behavior. Six test persons received the same computer-based tasks in which they were requested to mimic specific vocal expressions. Each test person mimicked 82 emotions, which led to a dataset of 492 emotions. All sessions were recorded on video. An overall accuracy of our software based on the requested emotions and the recognized emotions is a pretty good 74.6% for the emotions happy and neutral emotions; but will be improved for the lower values of an extended set of emotions. In contrast with existing software our solution allows to continuously and unobtrusively monitor learners’ intonations and convert these intonations into emotional states. This paves the way for enhancing the quality and efficacy of game-based learning by including the learner's emotional states, and links these to pedagogical scaffolding.

IRJET, 2023

Emotions are significant because they are essential to the learning process. Emotions, behaviour, and thoughts are intimately connected in such a way that the sum of these three factors determines how we act and what choices we make. The selection of a database, identifying numerous speech-related variables, and making an appropriate classification model choice are the three key hurdles in emotion recognition. In order to understand how these emotional states relate to students' and teachers' comprehension, it is important to first define the facial physical behaviours that are associated with various emotional states. The usefulness of facial expression and voice recognition between a teacher and student in a classroom was first examined in this study.

Journal of Physics: Conference Series, 2013

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

2003

This work aims at handling the affectivity in a learning interaction between student and tutor in MACES educational system: a collaborative environment with a multi-agent architecture based on Vygotsky's Socio-cultural Pedagogical Approach. The student's emotions are handled by the Mediating Agent of the system which has the goal of motivating and engaging the student to learn as well as promoting a positive mood in the student which is more appropriate to learning. We believe that the student can experience a more positive feeling through the exposure to emotional behaviour and encouragement messages sent by the Mediating Agent. In order to accomplish its function, it should recognise student's emotions to respond appropriately. Thus, it catches the student's emotions by his observable behaviour, stores this information in an affective model; and applies affective tactics according to these emotions. As these affective tactics can be emotive behaviour, we chose to represent the Mediating Agent as a lifelike character who has a personality and which interacts with the student through messages and emotive animations.

2019

Two types of students' performance evaluation mechanisms are called formative and summative. To have good results at the summative evaluation, the students must have frequent constructive feedback throughout a course. It is not an easy task for a teacher to give frequent feedback. As a solution for that, online tests can be introduced with feedback. A constructive feedback should consist of three features, i.e. Appreciation, Advice and Evaluation. When incorporating the appreciation in a feedback in an online education system, the emotions can be analysed to appreciate the student' effort. The most feasible way to analyze the emotions is through the facial expressions in an online system. However, there should be a systematic way to track the emotions based on each question, while the students are answering. This research proposes a time interval based on the transient emotion peak considering the time taken to read a question as the stimulus activity to cause an emotional change, to capture the emotional changes. A series of photographs were captured by using a webcam for each question and analysed the emotions by using MSemotion-API. At the end, a single photograph was selected based on this time, tracking the most different emotion out of the series. A video recording was taken simultaneously, and it was analysed by a psychiatrist and kept as the benchmark dataset. At the end, a cross validation test was done based on the output of API and the benchmark dataset and was noticed that this has given a good result.

Cognition and …, 2008

In an attempt to discover the facial action units for affective states that occur during complex learning, this study adopted an emote-aloud procedure in which participants were recorded as they verbalised their affective states while interacting with an intelligent tutoring system (AutoTutor). Participants' facial expressions were coded by two expert raters using Ekman's Facial Action Coding System and analysed using association rule mining techniques. The two expert raters received an overall kappa that ranged between .76 and .84. The association rule mining analysis uncovered facial actions associated with confusion, frustration, and boredom. We discuss these rules and the prospects of enhancing AutoTutor with non-intrusive affect-sensitive capabilities.

This paper presents the voice emotion recognition part of the FILTWAM framework for real-time emotion recognition in affective e-learning settings. FILT WAM (Framework for Improving Learning Through Webcams And Microphones) intends to offer timely and appropriate online feedback based upon learner’s vocal intonations and facial expressions in order to foster their learning. Whereas the facial emotion recognition part has been successfully tested in a previous study, the here presented study describes the development and testing of FILTWAM’s vocal emotion recognition software artefact. The main goal of this study was to show the valid use of computer microphone data for real-time and adequate interpretation of vocal intona- tions into extracted emotional states. The software that was developed was tested in a study with 12 participants. All participants individually received the same computer- based tasks in which they were requested 80 times to mimic specific vocal expressions (960 occurrences in total). Each individual session was recorded on video. For the validation of the voice emotion recognition software artefact, two experts annotated and rated participants’ recorded behaviours. Expert findings were then compared with the software recognition results and showed an overall accuracy of Kappa of 0.743. The overall accuracy of the voice emotion recognition software artefact is 67 % based on the requested emotions and the recognized emotions. Our FILTWAM-software allows to continually and unobtrusively observing learners’ behaviours and transforms these behaviours into emotional states. This paves the way for unobtrusive and real-time capturing of learners’ emotional states for enhancing adaptive e-learning approaches.

Proceedings of E- …, 2004

In an attempt to discover links between learning and emotions, this study adopted an emote-aloud procedure where participants were recorded as they verbalized their affective states while interacting with an intelligent tutoring system, AutoTutor. Participants' facial expressions were coding using the Facial Action Coding System and analyzed using association rule mining techniques. The resulting rules are discussed along with implications to the larger project of improving the AutoTutor system into a nonintrusive affect sensitive intelligent tutoring system.