Audio Emotion Recognition

This paper proposes EPyNet, a deep learning architecture designed for energy reduced audio emotion recognition.In the domain of audio based emotion recognition, where discerning emotional cues from audio input is crucial, the integration of artificial intelligence techniques has sparked a transformative shift in accuracy and performance.Deep learning , renowned for its ability to decipher intricate patterns, spearheads this evolution. However, the energy efficiency of deep learning models, particularly in resource-constrained environments, remains a pressing concern. Convolutional operations serve as the cornerstone of deep learning systems. However, their extensive computational demands leading to energy-inefficient computations render them as not ideal for deployment in scenarios with limited resources. Addressing these challenges, researchers came up with one-dimensional convolutional neural network (1D CNN) array convolutions, offering an alternative to traditional two-dimensional CNNs, with reduced resource requirements. However , this array-based operation reduced the resource requirement, but the energy-consumption impact was not studied. To bridge this gap, we introduce EPyNet, a deep learning architecture crafted for energy efficiency with a particular emphasis on neuron reduction. Focusing on the task of audio emotion recognition, We evaluate EPyNet on five public audio corpora-RAVDESS, TESS, EMO DB, CREMA D, and SAVEE.We propose three versions of EPyNet, a lightweight neural network designed for efficient emotion recognition, each optimized for different trade-offs between accuracy and energy efficiency. Experimental results demonstrated that the 0.06M EPyNet reduced energy consumed by 76.5% while improving accuracy by 5% on RAVDESS, 25% on TESS, and 9.75% on SAVEE. The 0.2M and 0.9M models reduced energy consumed by 64.9% and 70.3%, respectively. Additionally, we compared our Proposed 0.06M system with the MobileNet models on the CIFAR-10 dataset and achieved significant improvements. The 1035 proposed system reduces energy by 86.2% and memory by 95.7% compared to MobileNet, with a slightly lower accuracy of 0.8%. Compared to MobileNetV2, it improves accuracy by 99.2% and reduces memory by 93.8%. When compared to MobileNetV3, it achieves 57.2% energy reduction, 85.1% memory reduction, and a 24.9% accuracy improvement. We further test the scalability and robustness of the proposed solution on different data dimensions and frameworks.

In this work, we tackle a problem of speech emotion classification. One of the issues in the area of affective computation is that the amount of annotated data is very limited. On the other hand, the number of ways that the same emotion can be expressed verbally is enormous due to variability between speakers. This is one of the factors that limits performance and generalization. We propose a simple method that extracts audio samples from movies using textual sentiment analysis. As a result, it is possible to automatically construct a larger dataset of audio samples with positive, negative emotional and neutral speech. We show that pretraining recurrent neural network on such a dataset yields better results on the challenging EmotiW corpus. This experiment shows a potential benefit of combining textual sentiment analysis with vocal information.

In this work, we tackle a problem of speech emotion classification. One of the issues in the area of affective computation is that the amount of annotated data is very limited. On the other hand, the number of ways that the same emotion can be expressed verbally is enormous due to variability between speakers. This is one of the factors that limits performance and generalization. We propose a simple method that extracts audio samples from movies using textual sentiment analysis. As a result, it is possible to automatically construct a larger dataset of audio samples with positive, negative emotional and neutral speech. We show that pretraining recurrent neural network on such a dataset yields better results on the challenging EmotiW corpus. This experiment shows a potential benefit of combining textual sentiment analysis with vocal information.

on the IADS (the International Affective Digitized Sound system, a set of acoustic emotional stimuli for experimental investigations of emotion and attention), we construct two predictive and generic models ANN 1 ("EEG recordings of standardized group vs. emotions") and ANN 2 ("music audio content vs. emotion"). Both models are trained by an artificial neural network. We then collect a subject's EEG recordings when listening the selected IADS samples, and apply the ANN 1 to determine the subject's emotion vector. With the generic model and the corresponding individual differences, we construct the personalized model H by the projective transformation. In the testing phase, given a music object, the processing steps are: (1) to extract features from the music audio content, (2) to apply ANN 2 to calculate the vector in the arousal-valence emotion space, and (3) to apply the transformation matrix H to determine the personalized emotion vector. Moreover, with respect to a moderate music object, we apply a sliding window on the music object to obtain a sequence of personalized emotion vectors, in which those predicted vectors will be fitted and organized as an emotion trail for revealing dynamics in the affective content of music object. Experimental results suggest the proposed approach is effective.

on the IADS (the International Affective Digitized Sound system, a set of acoustic emotional stimuli for experimental investigations of emotion and attention), we construct two predictive and generic models ANN 1 ("EEG recordings of standardized group vs. emotions") and ANN 2 ("music audio content vs. emotion"). Both models are trained by an artificial neural network. We then collect a subject's EEG recordings when listening the selected IADS samples, and apply the ANN 1 to determine the subject's emotion vector. With the generic model and the corresponding individual differences, we construct the personalized model H by the projective transformation. In the testing phase, given a music object, the processing steps are: (1) to extract features from the music audio content, (2) to apply ANN 2 to calculate the vector in the arousal-valence emotion space, and (3) to apply the transformation matrix H to determine the personalized emotion vector. Moreover, with respect to a moderate music object, we apply a sliding window on the music object to obtain a sequence of personalized emotion vectors, in which those predicted vectors will be fitted and organized as an emotion trail for revealing dynamics in the affective content of music object. Experimental results suggest the proposed approach is effective.

In the era of data explosion, speech emotion plays crucial commercial significance. Emotion recognition in speech encompasses a gamut of techniques starting from mechanical recording of audio signal to complex modeling of extracted patterns. Most challenging part of this research purview is to classify the emotion of the speech purely based on the physical characteristics of the audio signal independent of language of speech. This paper focuses on the predictive modeling of audio speech data based on most viable feature set extraction and deployment of these features to predict the emotion of unknown speech data. We have used two most widely used classifiers, a variant of CART and Naïve Bayes, to model the dynamics of interplay of crucial features like Root Mean Square (RMS), Zero Cross Rate (ZCR), Pitch and Brightness of audio signal to determine the emotion of speech. In order to carry out comparative analysis of the proposed classifiers, a set of experiments on real speech data is conducted. Results clearly indicate that decision tree based classifier works well on accuracy whereas Naïve Bayes works fairly well on generality.

Music is not only a set of sounds, it evokes emotions, subjectively perceived by listeners. The growing amount of audio data available on CDs and in the Internet wakes up a need for content-based searching through these files. The user may be interested in finding pieces in a specific mood. The goal of this paper is to elaborate tools for such a search. A method for the appropriate objective description (parameterization) of audio files is proposed, and experiments on a set of music pieces are described. The results are summarized in concluding chapter.

Emotion recognition from generalized sounds is an interdisciplinary and emerging field of research. A vital requirement for this kind of investigations is the availability of ground truth datasets. Currently, there are 2 freely available datasets of emotionally annotated sounds, which, however, do not include sound evenets (SEs) with manifestation of the spatial location of the source. The latter is an inherent natural component of SEs, since all sound sources in real-world conditions are physically located and perceived somewhere in the listener's surrounding space. In this work we present a novel emotionally annotated sounds dataset consisting of 32 SEs that are spatially rendered using appropriate binaural processing. All SEs in the dataset are available in 5 spatial positions corresponding to source/receiver angles equal to 0, 45, 90, 135 and 180 degrees. We have used the IADS dataset as the initial collection of SEs prior to binaural processing. The annotation measures obtained for the novel binaural dataset demonstrate a significant accordance with the existing IADS dataset, while small ratings declinations illustrate a perceptual adaptation imposed by the more realistic SEs spatial representation.

ARTE270 is an innovative experimental platform that combines panoramic optical projection (video and photography) with surround sound. The idea is to immerse the spectator/listener into audiovisual content both in realistic and artistic terms. It initially emerged in order to perform artistic experimentation on data derived from the flourishing natural environment of the Ionian Islands. Nevertheless, in the course of time, it comes up that a very important potential application lies with sustainable development. Our platform has been tried as part of a holistic approach for capturing, preserving and promoting further forms of cultural heritage as well as cultural networking through the new rendering capabilities of digital media.

A variety of recent researches in Audio Emotion Recognition (AER) outlines high performance and retrieval accuracy results. However, in most works music is considered as the original sound content that conveys the identified emotions. One of the music characteristics that is found to represent a fundamental means for conveying emotions are the rhythmrelated acoustic cues. Although music is an important aspect of everyday life, there are numerous non-linguistic and nonmusical sounds surrounding humans, generally defined as sound events (SEs). Despite this enormous impact of SEs to humans, a scarcity of investigations regarding AER from SEs is observed. There are only a few recent investigations concerned with SEs and AER, presenting a semantic connection between the former and the listener's triggered emotion. In this work we analytically investigate the connection of rhythm-related characteristics of a wide range of common SEs with the arousal of the listener using sound events with semantic content. To this aim, several feature evaluation and classification tasks are conducted using different ranking and classification algorithms. High accuracy results are obtained, demonstrating a significant relation of SEs rhythmic characteristics to the elicited arousal.

Emotion recognition from sound signals represents an emerging field of recent research. Although many existing works focus on emotion recognition from music, there seems to be a relative scarcity of research on emotion recognition from general sounds. One of the key characteristics of sound events is the sound source spatial position, i.e. the location of the source relatively to the acoustic receiver. Existing studies that aim to investigate the relation of the latter source placement and the elicited emotions are limited to distance, front and back spatial localization and/or specific emotional categories. In this paper we analytically investigate the effect of the source angular position on the listener's emotional state, modeled in the well-established valence/arousal affective space. Towards this aim, we have developed an annotated sound events dataset using binaural processed versions of the available International Affective Digitized Sound (IADS) sound events library. All subjective affective annotations were obtained using the Self Assessment Manikin (SAM) approach. Preliminary results obtained by processing these annotation scores are likely to indicate a systematic change in the listener affective state as the sound source angular position changes. This trend is more obvious when the sound source is located outside of the visible field of the listener.

Sound events can carry multiple information, related to the sound source and to ambient environment. However, it is well-known that sound evokes emotions, a fact that is verified by works in the disciplines of Music Emotion Recognition and Music Information Retrieval that focused on the impact of music to emotions. In this work we introduce the concept of affective acoustic ecology that extends the above relation to the general concept of sound events. Towards this aim, we define sound event as a novel audio structure with multiple components. We further investigate the application of existing emotion models employed for music affective analysis to sonic, non-musical, content. The obtained results indicate that although such application is feasible, no significant trends and classification outcomes are observed that would allow the definition of an analytic relation between the technical characteristics of a sound event waveform and raised emotions.

Legacy technologies for word recognition can benefit from emerging affective voice retrieval, potentially leading to intelligent applications for smart houses enhanced with new features. In this work we introduce the implementation of a system, capable to react to common spoken words, taking into account the estimated vocal stress level, thus allowing the realization of a prioritized, affective aural interaction path. Upon the succesful word recognition and the corresponding stress level estimation, the system triggers particular ...

Sound events are proven to have an impact on the emotions of the listener. Recent works on the field of emotion recognition from sound events show, on one hand, the possibility of automatic emotional information retrieval from sound events and, on the other hand, the need for deeper understanding of the significance of the sound events' semantic content on listener's affective state. In this work we present a first, to the best of authors' knowledge, investigation of the relation between the semantic similarity of the sound events and the elicited emotion. For that cause we use two emotionally annotated sound datasets and the Wu-Palmer semantic similarity measure according to WordNet. Results indicate that the semantic content seems to have a limited role in the conformation of the listener's affective states. On the contrary, when the semantic content is matched to specific areas in the Arousal-Valence space or also the source's spatial position is taken into account, it is exhibited that the importance of the semantic content effect is higher, especially for the cases with medium to low valence and medium to high arousal or when the sound source is at the lateral positions of the listener's head, respectively.

Auralisation is becoming more commonplace as a means of communicating and disseminating plans for changes in our environmental landscape due to major infrastructure projects such as road, rail and airport developments, where noise can be a significant issue. This paper presents an approach to this challenge in the context of a major public consultation for a UK road project as well as commenting on different means for facilitating presentation of, and engagement with, both audio and visual material. Although such methods are generally used for information only, every effort is made to ensure that the final result is both plausible and comparable to existing on-site conditions, and calibrated accordingly, with a view to supporting other more standardised methods and metrics. However, it is also possible to develop more quantitative measures as to how effective and successful such auralisations are in terms of conveying a representation of a given scene, and any planned for interventions. In this work the Self Assessment Manikin and soundscape classification descriptors are used in a series of listening tests to better understand the approach that has been adopted for these types of projects.

Emotion recognition from sound signals represents an emerging field of recent research. Although many existing works focus on emotion recognition from music, there seems to be a relative scarcity of research on emotion recognition from general sounds. One of the key characteristics of sound events is the sound source spatial position, i.e. the location of the source relatively to the acoustic receiver. Existing studies that aim to investigate the relation of the latter source placement and the elicited emotions are limited to distance, front and back spatial localization and/or specific emotional categories. In this paper we analytically investigate the effect of the source angular position on the listener's emotional state, modeled in the well-established valence/arousal affective space. Towards this aim, we have developed an annotated sound events dataset using binaural processed versions of the available International Affective Digitized Sound (IADS) sound events library. All subjective affective annotations were obtained using the Self Assessment Manikin (SAM) approach. Preliminary results obtained by processing these annotation scores are likely to indicate a systematic change in the listener affective state as the sound source angular position changes. This trend is more obvious when the sound source is located outside of the visible field of the listener.

Emotion recognition from generalized sounds is an interdisciplinary and emerging field of research. A vital requirement for this kind of investigations is the availability of ground truth datasets. Currently, there are 2 freely available datasets of emotionally annotated sounds, which, however, do not include sound evenets (SEs) with manifestation of the spatial location of the source. The latter is an inherent natural component of SEs, since all sound sources in real-world conditions are physically located and perceived somewhere in the listener's surrounding space. In this work we present a novel emotionally annotated sounds dataset consisting of 32 SEs that are spatially rendered using appropriate binaural processing. All SEs in the dataset are available in 5 spatial positions corresponding to source/receiver angles equal to 0, 45, 90, 135 and 180 degrees. We have used the IADS dataset as the initial collection of SEs prior to binaural processing. The annotation measures obtained for the novel binaural dataset demonstrate a significant accordance with the existing IADS dataset, while small ratings declinations illustrate a perceptual adaptation imposed by the more realistic SEs spatial representation.

Emotion labels are usually obtained via either manual annotation, which is tedious and time-consuming, or questionnaires, which neglect the time-varying nature of emotions and depend on human's unreliable introspection. To overcome these limitations, we developed a continuous, real-time, joystick-based emotion annotation framework. To assess the same, 30 subjects each watched 8 emotion-inducing videos. They were asked to indicate their instantaneous emotional state in a valence-arousal (V-A) space, using a joystick. Subsequently, five analyses were undertaken: (i) a System Usability Scale (SUS) questionnaire unveiled the framework's excellent usability; (ii) MANOVA analysis of the mean V-A ratings and (iii) trajectory similarity analyses of the annotations confirmed the successful elicitation of emotions; (iv) Change point analysis of the annotations, revealed a direct mapping between emotional events and annotations, thereby enabling automatic detection of emotionally salient points in the videos; and (v) Support Vector Machines (SVM) were trained on classification of 5 second chunks of annotations as well as their change-points. The classification results confirmed that ratings patterns were cohesive across the participants. These analyses confirm the value, validity, and usability of our annotation framework. They also showcase novel tools for gaining greater insights into the emotional experience of the participants.

Emotion recognition from sound signals represents an emerging field of recent research. Although many existing works focus on emotion recognition from music, there seems to be a relative scarcity of research on emotion recognition from general sounds. One of the key characteristics of sound events is the sound source spatial position, i.e. the location of the source relatively to the acoustic receiver. Existing studies that aim to investigate the relation of the latter source placement and the elicited emotions are limited to distance, front and back spatial localization and/or specific emotional categories. In this paper we analytically investigate the effect of the source angular position on the listener's emotional state, modeled in the well-established valence/arousal affective space. Towards this aim, we have developed an annotated sound events dataset using binaural processed versions of the available International Affective Digitized Sound (IADS) sound events library. All subjective affective annotations were obtained using the Self Assessment Manikin (SAM) approach. Preliminary results obtained by processing these annotation scores are likely to indicate a systematic change in the listener affective state as the sound source angular position changes. This trend is more obvious when the sound source is located outside of the visible field of the listener.

Emotion recognition from sound signals represents an emerging field of recent research. Although many existing works focus on emotion recognition from music, there seems to be a relative scarcity of research on emotion recognition from general sounds. One of the key characteristics of sound events is the sound source spatial position, i.e. the location of the source relatively to the acoustic receiver. Existing studies that aim to investigate the relation of the latter source placement and the elicited emotions are limited to distance, front and back spatial localization and/or specific emotional categories. In this paper we analytically investigate the effect of the source angular position on the listener’s emotional state, modeled in the well–established valence/arousal affective space. Towards this aim, we have developed an annotated sound events dataset using binaural processed versions of the available International Affective Digitized Sound (IADS) sound events library. All sub...

This paper describes an approach to the sonification of realtime twitter music trend data realized for the ICAD 2012 Sonification Competition: Listening to the World Listening. The paper will discuss the techniques used to create the sonification and the motivations behind them, including details of the data analysis, mapping strategies, visual display and sonification output. The system analyses the Twitter Music Trends data feed, which aggregates music listening data from Twitter by artist, as well as the Echo Nest REST API to determine the perceived emotional affect and prevailing descriptions of a selection of the latest trending artists. The resulting data is visualized and sonified in real-time to facilitate analysis and generate an appealing visual and auditory display of the resulting data. Experience with the system suggests that it is successful in allowing users to determine perceived emotional affect and quality for a number of artists simultaneously, and could allow fur...

Perceiving Emotion in Sounds: Does Timbre Play a Role? (December 2011) Casady Diane Bowman, B.S., Oklahoma State University Chair of Advisory Committee: Dr. Takashi Yamauchi Acoustic features of sound such as pitch, loudness, perceived duration and timbre have been shown to be related to emotion in regard to sound, demonstrating that research involving the important connection between the perceived emotions and their timbres is lacking. This study investigates the relationship between acoustic features of sound and emotion with regard to timbre. In two experiments, we investigated whether particular acoustic components of sound can predict timbre and particular categories of emotion, and how these attributes are related. Two behavioral experiments related perceived emotion ratings with synthetically created sounds and International Affective Digitized Sounds (IADS). Also, two timbre experiments found a connection between acoustic components of synthetically created sounds, and IADS....

In this paper we present a status report of the design and development stages of an online social network system based on Soundscapes. Additionally, a detailed description of the auditory scene analysis module of the system is made. The term Soundscape is used to describe the relation and interaction between the acoustic environment of a place and its inhabitants, namely on how these perceive and judge the sound. From this premise we developed a mobile application that senses and shares information regarding the Soundscapes of the places inhabited by each user during his/her daily life. Thus, each user acts as a terminal of a sensor network linked through the use of sound. The goal of this research is to assess the use of Soundscapes in social network interactions and consequently promote sound awareness among people.

In the absence of a well suited measure for quantifying binaural data variations, this study presents the use of a global perceptual distance metric which can describe both HRTF as well as listener similarities. The metric is derived based on subjective evaluations of binaural renderings of a sound moving along predefined trajectories in the horizontal and median planes. Its characteristics and advantages in describing data distributions based on perceptually relevant attributes are discussed. In addition, the use of 24 HRTFs from two different databases of origin allows for an evaluation of the perceptual impact of some database-dependent characteristics on spatialization. The effectiveness of the experimental design as well as the correlation between the HRTF evaluations of the two plane trajectories are also discussed.

Legacy technologies for word recognition can benefit from emerging affective voice retrieval, potentially leading to intelligent applications for smart houses enhanced with new features. In this work we introduce the implementation of a system, capable to react to common spoken words, taking into account the estimated vocal stress level, thus allowing the realization of a prioritized, affective aural interaction path. Upon the successful word recognition and the corresponding stress level estimation, the system triggers particular affective-prioritized actions, defined within the application scope of an intelligent home environment. Application results show that the established affective interaction path significantly improves the ambient intelligence provided by an affective vocal sensor that can be easily integrated with any sensor-based home monitoring system.

In this paper we present a status report of the design and development stages of an online social network system based on Soundscapes. Additionally, a detailed description of the auditory scene analysis module of the system is made. The term Soundscape is used to describe the relation and interaction between the acoustic environment of a place and its inhabitants, namely on how these perceive and judge the sound. From this premise we developed a mobile application that senses and shares information regarding the Soundscapes of the places inhabited by each user during his/her daily life. Thus, each user acts as a terminal of a sensor network linked through the use of sound. The goal of this research is to assess the use of Soundscapes in social network interactions and consequently promote sound awareness among people.

Citizen Science aims to engage people in research activities on important issues related to their well-being. Smart Cities aim to provide them with services that improve the quality of their life. Both concepts have seen significant growth in the last years and can be further enhanced by combining their purposes with Internet of Things technologies that allow for dynamic and large-scale communication and interaction. However, exciting and retaining the interest of participants is a key factor for such initiatives. In this paper we suggest that engagement in Citizen Science projects applied on Smart Cities infrastructure can be enhanced through contextual and structural game elements realized through augmented audio interactive mechanisms. Our inter-disciplinary framework is described through the paradigm of a collaborative bird call recognition game, in which users collect and submit audio data that are then classified and used for augmenting physical space. We discuss the Playful Learning, Internet of Audio Things, and Bird Monitoring principles that shaped the design of our paradigm and analyze the design issues of its potential technical implementation.

In the absence of a well suited measure for quantifying binaural data variations, this study presents the use of a global perceptual distance metric which can describe both HRTF as well as listener similarities. The metric is derived based on subjective evaluations of binaural renderings of a sound moving along predefined trajectories in the horizontal and median planes. Its characteristics and advantages in describing data distributions based on perceptually relevant attributes are discussed. In addition, the use of 24 HRTFs from two different databases of origin allows for an evaluation of the perceptual impact of some database-dependent characteristics on spatialization. The effectiveness of the experimental design as well as the correlation between the HRTF evaluations of the two plane trajectories are also discussed.

Sonification is the utilization of sounds to convey information about data or events. There are two types of emotions associated with sounds: (1) “perceived” emotions, in which listeners recognize the emotions expressed by the sound, and (2) “induced” emotions, in which listeners feel emotions induced by the sound. Although listeners may widely agree on the perceived emotion for a given sound, they often do not agree about the induced emotion of a given sound, so it is difficult to model induced emotions. This paper describes the development of several machine and deep learning models that predict the perceived and induced emotions associated with certain sounds, and it analyzes and compares the accuracy of those predictions. The results revealed that models built for predicting perceived emotions are more accurate than ones built for predicting induced emotions. However, the gap in predictive power between such models can be narrowed substantially through the optimization of the ma...

Η παρούσα εργασία αφορά στο σχεδιασµό, την ανάπτυξη και την αξιολόγηση µιας εφαρµογής λογισµικού ηχοµέτρησης για έξυπνα τηλέφωνα (iPhone). Πρόκειται για µια πρόταση σχεδιασµού και υλοποίησης ενός ευέλικτου και φιλικού στη χρήση περιβάλλοντος µέτρησης της ηχητικής στάθµης, που να µπορεί εύκολα να χρησιµοποιηθεί από µη ειδικούς. Γνώµονας ανάπτυξης αποτέλεσε η επιδίωξη το παραγόµενο λογισµικό να παρέχει παρόµοιες λειτουργίες και ισάξιας πιστότητας µετρήσεις µε µια επαγγελµατική συσκευή ηχοµέτρησης. Παράλληλα, δόθηκε βαρύτητα στην ενσωµάτωση επιπρόσθετων δυνατοτήτων που θα προσφέρουν πολύτιµες ευκολίες στον χρήστη και έχουν να κάνουν µε τη διαχείριση των συνεδριών µέτρησης.

Μία από τις κύριες διεργασίες στο πεδίο αναγνώρισης συναισθημάτων από ηχητικά δεδομένα είναι η συλλογή της συναισθηματικής επισήμανσης. Λόγω της φύσης των δεδομένων αλλά και του πλήθους αυτών, κάθε τέτοια διεργασία μπορεί να είναι πολύπλοκη οργανωτικά και χρονοβόρος διαδικασία. Στην παρούσα εργασία παρουσιάζεται μία διαδικτυακή πλατφόρμα για την συλλογή συναισθηματικών επισημάνσεων στην οποία γίνεται χρήση του ευρύτερα καθιερωμένου Self Assessment Manikin (SAM). Η πλατφόρμα επιτρέπει στον χρήση να αναπαράγει τους προς αξιολόγηση ήχους, καταγράφει τα αποτελέσματα και παρέχει επισημάνσεις σχετικά με κύρια σημεία της διεργασίας, όπως την επανάληψη του πειράματος από τον ίδιο χρήστη.

Η αναγνώριση των συναισθημάτων των μαθητών είναι ουσιαστικό στοιχείο για μια επιτυχημένη εκπαιδευτική διαδικασία και ιδιαίτερα σε συνθήκες εξ αποστάσεως εκπαίδευσης είναι πολύ δύσκολο να το επιτύχει ένας εκπαιδευτικός. Η τεχνολογία αναγνώρισης συναισθημάτων μέσα από εικόνα ή βίντεο, μπορεί να συμβάλλει προς την κατεύθυνση αυτή. Η εργασία αυτή αφορά την παρουσίαση και την μεθοδολογία εγκυροποίησης μιας πλατφόρμας, με την οποία παρέχεται η δυνατότητα δοκιμής διαφόρων αλγορίθμων αναγνώρισης συναισθημάτων και επιλογής των καλύτερων μοντέλων τεχνητής ευφυΐας. Η πλατφόρμα εγκυροποιήθηκε επιτυχώς, μετά από χρήση κατάλληλων σετ εικόνων από προηγούμενες έρευνες και σε επόμενο στάδιο θα χρησιμοποιηθεί σε πραγματικές συνθήκες εξ αποστάσεως εκπαίδευσης

Modern interactive means combined with new digital media processing and representation technologies can provide a robust framework for enhancing user experience in multimedia entertainment systems and audiovisual artistic installations with non-traditional interaction/feedback paths based on user affective state. In this work, the 'Elevator' interactive audiovisual platform prototype is presented, which aims to provide a framework for signalling and expressing human behaviour related to emotions (such as anger) and finally produce a visual outcome of this behaviour, defined here as the emotional 'thumbnail' of the user. Optimised, real-time audio signal processing techniques are employed for monitoring the achieved anger-like behaviour, while the emotional elevation is attempted using appropriately selected combined audio/visual content reproduced using state-of-the-art audiovisual playback technologies that allow the creation of a realistic immersive audiovisual environment. The demonstration of the proposed prototype has shown that affective interaction is possible, allowing the further development of relative artistic and technological applications.

Στην παρούσα εισήγηση επιχειρείται μια πρώτη προσέγγιση της σύγκλισης των πεδίων των ηχητικών παιχνιδιών και της επαυξημένης πραγματικότητας με στόχο τη δημιουργία ηχητικά επαυξημένων παιγνιακών συστημάτων που θα αξιοποιούν τα εγγενή χαρακτηριστικά των δύο πεδίων βελτιστοποιώντας τις συνθήκες διάδρασης και εμβύθισης της παιγνιακής εμπειρίας. Αρχικά περιγράφονται οι συνηθέστερες τεχνικές ηχητικής αναπαράστασης, με έμφαση σε αυτές που αφορούν στη χωρική τοποθέτηση και κίνηση των ηχητικών συμβάντων, καθώς και τα βασικά δομικά στοιχεία της ηχητικής παιγνιακής μηχανικής. Έπειτα, αποτυπώνονται οι σύγχρονες κατευθύνσεις εφαρμογής ηχητικά επαυξημένων συστηματων, και τέλος προτείνεται η ανάπτυξη ενός προηγμένου μείκτη ηχητικής επαυξημένης πραγματικότητας με βάση την αμφιωτική ακουστότητα.

-This research work aimed at using word-level features for mood classification. The goal of this project is to build a classifier model to detect the mood of the song by analyzing the lyrics of Bangla Song. Na¨ıveNa¨ıve Bayes Classifier is used to predict the probability of different classes based on various attributes. Constructing dataset of 500 Bangla songs consisting of Rabindra Sangeet, Nazrul Geeti, Folk and Modern. We have worked on two types of Bangla moods such as happy and sad, corresponding to the Bangla meaning. At the end of the experiment, our proposed method performed well to classify the mood of the song with high accuracy.

The aim of this research work is to detect the music mood of the listener, by analyzing the lyrics of Bangla language based songs. We used a naive Bayes Classifier as the main feature to built the music recommendation system, trained the attitude of songs based on song lyrics to predict. We constructed our own dataset of 500 Bangla songs consisting Rabindra Sangeet, Nazrul Geeti, Folk, Modern. We have worked on two types of bangla moods such as happy and sad, corresponding to the bangla meaning. At the end of the experiment, our proposed method performed really well to classify the mood of the song with high accuracy.

Text classification is an important and common task in supervised machine learning. The Naive Bayes Classifier is a popular algorithm that can be used for this purpose. The goal of our research was prediction of song performer using Naive Bayes classification algorithm based solely on lyrics. A dataset that has been created consists of lyrics performed by Nirvana and Metallica, 207 songs in total. Model evaluation measures showed very good results: precision of 0.93, recall of 0.95 and F 1-measure of 0.94, therefore lyrics classification using Naive Bayes can be considered as successful.

— Acoustically expressed emotions can make communication with a robot more efficient. Detecting emotions like anger could provide a clue for the robot indicating unsafe/undesired situations. Recently, several deep neural network-based models have been proposed which establish new state-of-the-art results in affective state evaluation. These models typically start processing at the end of each utterance, which not only requires a mechanism to detect the end of an utterance but also makes it difficult to use them in a real-time communication scenario, e.g. human-robot interaction. We propose the EmoRL model that triggers an emotion classification as soon as it gains enough confidence while listening to a person speaking. As a result, we minimize the need for segmenting the audio signal for classification and achieve lower latency as the audio signal is processed incrementally. The method is competitive with the accuracy of a strong baseline model, while allowing much earlier prediction.

Technologies capable of automatically sensing and recognizing emotion are becoming increasingly prevalent in performance and compositional practice. Though these technologies are complex and diverse, we present a typology that draws on similarities with computational systems for expressive music performance. This typology provides a framework to present results from the development of two audio environments for the Emotional Imaging Composer, a commercial product for realtime arousal/valence recognition that uses signals from the autonomic nervous system. In the first environment, a spectral delay processor for live vocal performance uses the performer's emotional state to interpolate between subspaces of the arousal/valence plane. For the second, a sonification mapping communicates continuous arousal and valence measurements using tempo, loudness, decay, mode, and roughness. Both were informed by empirical research on musical emotion, though differences in desired output schemas manifested in different mapping strategies.

The close relationship between music and emotions seems to be a commonplace. Although numerous attempts have been made to better understand this relationship, however, there are still many unresolved questions regarding the manifold connections between these two human phenomena. Our panel wants to approach the relationship between music and emotions by understanding both – music and emotion – as expressions, and therefore as being very similar in character. Thereby the voice acts as link between music and emotions, since it is used expressively in both the musical (singing, declaming etc) as well as in the emotional realm (scream, shout, laugh etc) and thus is building a bridge between both the aesthetical and the affective side.
The panel wants to examine the relationship between music, voice and emotions from as manifold perspectives as possible. Thematic areas could include performance studies, aesthetical concepts, analysis or history of compositions, aspects of embodiment. Thereby, we are particularly interested to contrast examples that stem from different historical as well as different cultural areas and also invite proposals concerning the border between vocal sounds in music and vocal sounds in everyday life. The aim of the panel is to combine as manifold aspects of musical and vocal expressions of emotions and to thereby come as close as possible to the relationship of music and emotions in past and present.

ARTE270 is an innovative experimental platform that combines panoramic optical projection (video and photography) with surround sound. The idea is to immerse the spectator/listener into audiovisual content both in realistic and artistic terms. It initially emerged in order to perform artistic experimentation on data derived from the flourishing natural environment of the Ionian Islands. Nevertheless, in the course of time, it comes up that a very important potential application lies with sustainable development. Our platform has been tried as part of a holistic approach for capturing, preserving and promoting further forms of cultural heritage as well as cultural networking through the new rendering capabilities of digital media.

In this thesis we focus on the automatic emotion classification of music samples. We extract a set of features from the music signal and examine their discriminatory capability using various classification techniques. Our goal is to determine the features and the classification methods that lead to the best classification of the emotion a music sample conveys. During the course of the thesis, we generated
our own dataset of annotated song samples and we examined two distinct methods of describing an emotion: using clusters consisting of various emotional states, and using a two-dimensional representation of the emotion in the Valence-Activation plane. The latter method was chosen as the most successful. We also tried other approaches of music emotion classification (MEC) as well, such as treating the song sample as an amplitude and frequency modulated (AM-FM) signal, on which we subsequently perform multiband demodulation analysis (MDA) testing various Gabor filter banks (Mel scale-based filter bank, Bark scale-based filter bank, and a number of fractional octave-based filter banks). Statistics of the Frequency Modulation Percentages (FMPs) of each band derived from the demodulation, proved to be quite successful features in the classification of emotion. Finally, we
explored other modalities besides the music sound signal itself, such as a number of features derived from the chords of the song samples, classification of the song samples' lyrics using various techniques and a brief investigation of Electroencephalogram (EEG) data generated by one of the annotators
while performing the annotation of the song samples.  Our final feature-pack included a combination of the most successful features among the ones we studied: (i) music-inspired features (features based on music theory and psychoacoustics, derived from either the sound signal or the chords of the sample), (ii) statistics of the FMPs and (iii) statistics of the Mel-frequency cepstral coefficients (MFCCs). This feature-pack proved to be more robust than its three individual components and in the end we achieved results that reached 85.7% correct classification rate in the dimension of Valence and 85.1% correct classification rate in the dimension of Activation. We finally demonstrate that by discarding training samples that are assigned a label too close to the neutral value, our results can improve even further, especially in the dimension of Activation.

Sound events are proven to have an impact on the emotions of the listener. Recent works on the field of emotion recognition from sound events show, on one hand, the possibility of automatic emotional information retrieval from sound events and, on the other hand, the need for deeper understanding of the significance of the sound events’ semantic content on listener’s affective state. In this work we present a first, to the best
of authors’ knowledge, investigation of the relation between the semantic similarity of the sound events and the elicited emotion. For that cause we use two emotionally annotated sound datasets and the Wu-Palmer semantic similarity measure according to WordNet. Results indicate that the semantic content seems to have a limited role in the conformation of the listener’s affective states. On the contrary, when the semantic content is matched to specific areas in the Arousal - Valence space or also the source’s spatial position is taken into account, it is exhibited that the importance of the semantic content effect is higher, especially for the cases with medium to low valence and medium to high arousal or when the sound source is at the lateral positions of the listener’s head, respectively.