Perception of affect in unfamiliar musical chords

Frontiers in Psychology, 2013

The aim of this study is to manipulate musical cues systematically to determine the aspects of music that contribute to emotional expression, and whether these cues operate in additive or interactive fashion, and whether the cue levels can be characterized as linear or non-linear. An optimized factorial design was used with six primary musical cues (mode, tempo, dynamics, articulation, timbre, and register) across four different music examples. Listeners rated 200 musical examples according to four perceived emotional characters (happy, sad, peaceful, and scary). The results exhibited robust effects for all cues and the ranked importance of these was established by multiple regression. The most important cue was mode followed by tempo, register, dynamics, articulation, and timbre, although the ranking varied across the emotions. The second main result suggested that most cue levels contributed to the emotions in a linear fashion, explaining 77-89% of variance in ratings. Quadratic encoding of cues did lead to minor but significant increases of the models (0-8%). Finally, the interactions between the cues were non-existent suggesting that the cues operate mostly in an additive fashion, corroborating recent findings on emotional expression in music (Juslin and Lindström, 2010).

Journal of The American Society for Information Science and Technology, 2006

Understanding and modeling human experience and emotional response when listening to music are important for better understanding of the stylistic choices in musical composition. In this work, we explore the relation of audio signal structure to human perceptual and emotional reactions. Memory, repetition, and anticipatory structure have been suggested as some of the major factors in music that might influence and possibly shape these responses. The audio analysis was conducted on two recordings of an extended contemporary musical composition by one of the authors. Signal properties were analyzed using statistical analyses of signal similarities over time and information theoretic measures of signal redundancy. They were then compared to Familiarity Rating and Emotional Force profiles, as recorded continually by listeners hearing the two versions of the piece in a live-concert setting. The analysis shows strong evidence that signal properties and human reactions are related, suggesting applications of these techniques to music understanding and music information-retrieval systems.

This study investigated the role of timbre for the perception of emotion in instrumental sounds. In 2 experiments, 180 stimuli were created by mixing sounds of 10 instruments (flute, clarinet, trumpet, tuba, piano, French horn, violin, guitar, saxophone, and bell). In Experiment 1a, participants received stimuli 1 at a time and rated the degree to which each stimulus sounded like each of the 10 instruments (i.e., timbre judgment). In Experiment 1b, participants received the same sound stimuli and rated whether these stimuli sounded happy, sad, angry, fearful, and disgusting (i.e., emotion judgment). The authors extracted acoustic features from these instrumental sounds and examined the extent to which these features could predict both emotion and timbre ratings made for the same sound stimuli. Our regression analysis showed that regularity, envelope centroid, sub band 2, and sub band 9 explained timbre and emotion ratings. The relationship between acoustic features and emotion judgments of basic emotions, however, was not uniform. Sub band 7, related to perceived activity in a sound, predicted anger, fear and disgust, but not sadness. Because sub band 7 could predict all emotions except for sadness, this indicates that some timbre-related features play a substantial role in the perception of emotion and that timbre could be a more useful indicator for specific emotions rather than emotion in general.

We sustain that the structure of affect elicited by music is largely dependent on dynamic temporal patterns in low-level music structural parameters. In support of this claim, we have previously provided evidence that spatiotemporal dynamics in psychoacoustic features resonate with two psychological dimensions of affect underlying judgments of subjective feelings: arousal and valence. In this article we extend our previous investigations in two aspects. First, we focus on the emotions experienced rather than perceived while listening to music. Second, we evaluate the extent to which peripheral feedback in music can account for the predicted emotional responses, that is, the role of physiological arousal in determining the intensity and valence of musical emotions. Akin to our previous findings, we will show that a significant part of the listeners' reported emotions can be predicted from a set of six psychoacoustic features-loudness, pitch level, pitch contour, tempo, texture, and sharpness. Furthermore, the accuracy of those predictions is improved with the inclusion of physiological cues-skin conductance and heart rate. The interdisciplinary work presented here provides a new methodology to the field of music and emotion research based on the combination of computational and experimental work, which aid the analysis of the emotional responses to music, while offering a platform for the abstract representation of those complex relationships. Future developments may aid specific areas, such as, psychology and music therapy, by providing coherent descriptions of the emotional effects of specific music stimuli.

Quarterly Journal of Experimental Psychology, 2009

Salient sensory experiences often have a strong emotional tone, but the neuropsychological relations between perceptual characteristics of sensory objects and the affective information they convey remain poorly defined. Here we addressed the relationship between sound identity and emotional information using music. In two experiments, we investigated whether perception of emotions is influenced by altering the musical instrument on which the music is played, independently of other musical features. In the first experiment, 40 novel melodies each representing one of four emotions (happiness, sadness, fear, or anger) were each recorded on four different instruments (an electronic synthesizer, a piano, a violin, and a trumpet), controlling for melody, tempo, and loudness between instruments. Healthy participants (23 young adults aged 18 -30 years, 24 older adults aged 58 -75 years) were asked to select which emotion they thought each musical stimulus represented in a four-alternative forcedchoice task. Using a generalized linear mixed model we found a significant interaction between instrument and emotion judgement with a similar pattern in young and older adults ( p , .0001 for each age group). The effect was not attributable to musical expertise. In the second experiment using the same melodies and experimental design, the interaction between timbre and perceived emotion was replicated (p , .05) in another group of young adults for novel synthetic timbres designed to incorporate timbral cues to particular emotions. Our findings show that timbre Correspondence should be addressed to

The focus of this dissertation is to investigate the perception of single chords, both in terms of perceived emotions and psychoacoustic qualities. Previous empirical research on harmony perception has mainly been concerned with horizontal aspects of harmony: studies focusing on vertical harmony perception have been rare. The current work aims to fill some of the still evident gaps between musicology, music psychology, and psychoacoustics with regard to single chord perception by combining both empirical and theoretical approaches. The work comprises three publications, based on two large-scale empirical experiments conducted with the aim of drawing the attention of both expert and inexpert listeners to have a substantial, heterogeneous, and international participant pool. The applied chord stimuli spanned common triads and tetrads with inversions, and also more rare chord sonorities in the form of pentads and hexachords. Altogether two distinct timbres were used, piano and strings. The stimuli were analyzed in terms of psychoacoustic properties, including e.g., roughness, harmonicity, and sharpness to further account for the obtained results. The results suggest significant differences in emotion perception across different chord types. The inversions and register contributed to the evaluations significantly, non-musicians distinguishing between triadic inversions similarly to musicians. Mildly dissonant chords were more preferred than consonant chords, regardless of musical sophistication or music preferences. Certain chords, especially the major seventh chord played on the strings, conveyed the emotion of nostalgia/longing effectively. New theoretical models are suggested based on the results and psychoacoustic data to explain the capacity of certain chords to convey complex emotions, and the perception of consonance/dissonance in single chords.

Psychoacoustical and Psychophysiological Correlates of the Emotional Impact and the Perception of Music Music plays an important role in everyday human life. One important reason for this is the capacity of music to influence the emotions of the listeners. Sometimes music is even accompanied by physical reactions, the so-called "strong experiences of music" (SEMs). Very pleasurable emotions are referred to as "chills", which denote shivers down the spine or goose pimples. The present study consists of two experiments investigating both the role of psychoacoustical features and social backgrounds of participants, and physiological correlates of chills. An adequate study of emotions in music should be based on the real-time measurement of self-report data using a continuous response method. Therefore, a recording system is proposed in this thesis which considers two important aspects: firstly, experimental and technical standards for continuous measurement should be taken into account for a better comparison of results, and secondly, the recording system should be open to the inclusion of multimodal stimuli. Based on these two primary considerations, four basic principles of the continuous measurement of emotions are addressed: (a) the dimensionality of the emotion space; (b) the synchronization of media and self-reported data; (c) the interface construction for emotional responses; and (d) an integrated software solution. Researcherdeveloped software (EMuJoy) is presented as a freeware solution for the continuous measurement of responses to different media. Participants (n = 38) in the first experiment listened to both pre-selected musical pieces and their own favorite emotion-inducing pieces and reported their chill experiences. They were asked to report their emotions in real-time using a two-dimensional emotion space in a computer software. Additionally, psychophysiological measurements, such as heart rate (HR) and skin conductance (SC), were taken. Films were created from the time-series of all self-reports as a synopsis for seven musical pieces heard by all participants. This technique of data visualization allows a very aesthetic method of data analysis and provides the opportunity to investigate commonalities of emotional self-reports as well as difvii x-psychoakustischenEigenschaften von Musik korrelieren. Die Vpn gaben außerdem an, welche Kriterien sie benutzt hatten, um zu ihrer Einschätzung des Chillpotentials der Musik zu kommen und wie sie die einzelnen Kriterien gewichten würden. Harmonie, Melodie, Klimax und Lautstärkeverlauf waren die vier wichtigsten Parameter. Außerdem wurde ein Geschlechtsunterschied in den Bewertungen festgestellt: Frauen gaben mehr Chills an als Männer (Frauen: 32.1 Chills pro Person, Männer: 15.2 Chills pro Person), Aufregung, Gefallen und Chillpotential wurden von den Frauen allerdings auch systematisch höher bewertet als von den Männern.

USING A THREE-DIMENSIONAL MODEL of affect, we compared the affective consequences of manipulating intensity, rate, and pitch height in music and speech. Participants rated 64 music and 64 speech excerpts on valence (pleasant-unpleasant), energy arousal (awaketired), and tension arousal (tense-relaxed). For music and speech, loud excerpts were judged as more pleasant, energetic, and tense than soft excerpts. Manipulations of rate had overlapping effects on music and speech. Fast music and speech were judged as having greater energy than slow music and speech. However, whereas fast speech was judged as less pleasant than slow speech, fast music was judged as having greater tension than slow music. Pitch height had opposite consequences for music and speech, with high-pitched speech but lowpitched music associated with higher ratings of valence (more pleasant). Interactive effects on judgments were also observed. We discuss similarities and differences between vocal and musical communication of affect, and the need to distinguish between two types of arousal: energy and tension.

Proceeding of the 40th International Computer Music Conference (ICMC)

Music conveys emotions by means of pitch, rhythm, loudness, and many other musical qualities. It was recently confirmed that timbre also has direct association with emotion, for example, that a horn is perceived as sad and a trumpet heroic in even isolated instrument tones. As previous work has mainly focused on sustaining instruments such as bowed strings and winds, this paper presents an experiment with non-sustaining instruments, using a similar approach with pairwise comparisons of tones for emotion categories. Plucked string, mallet percussion, and keyboard instrument tones were investigated for eight emotions: Happy, Sad, Heroic, Scary, Comic, Shy, Joyful, and Depressed. We found that plucked string tones tended to be Sad and Depressed, while harpsichord and mallet percussion tones induced positive emotions such as Happy and Heroic. The piano was emotionally neutral. Beyond spectral centroid and its deviation, which are important features in sustaining tones, decay slope was also significantly correlated with emotion in non-sustaining tones.

2015

The goal of this study was to evaluate the impact of the inclusion of listener-related factors (individual differences) on the prediction of music induced affect. A group of 24 subjects listened to a set of music excerpts previously demonstrated to express specific emotional characteristics (in terms of Arousal and Valence), and we collected information related to listeners' stable (personality, emotional intelligence, attentiveness, music preferences) and transient (mood, and physiological activity) states. Through a series of regression analysis we identified those factors which have a significant explanatory power over the affective states induced in the listeners. Our results show that incorporating information related to individual differences permits to identify more accurately the affective states induced in the listeners, which differ from those expressed by the music.