Opinion and emotion in movies: a modular perspective to annotation

2018

We present results of a sentiment annotation study in the context of historical German plays. Our annotation corpus consists of 200 representative speeches from the German playwright Gotthold Ephraim Lessing. Six annotators, five non-experts and one expert in the domain, annotated the speeches according to different sentiment annotation schemes. They had to annotate the differentiated polarity (very negative, negative, neutral, mixed, positive, very positive), the binary polarity (positive/negative) and the occurrence of eight basic emotions. After the annotation, the participants completed a questionnaire about their experience of the annotation process; additional feedback was gathered in a closing interview. Analysis of the annotations shows that the agreement among annotators ranges from low to mediocre. The non-expert annotators perceive the task as very challenging and report different problems in understanding the language and the context. Although fewer problems occur for th...

Movie ratings and reviews at sites such as IMDb or Amazon are commonly used by moviegoers to decide which movie to watch or buy next. Currently, moviegoers base their decisions as to which movie to watch by looking at the ratings of movies as well as reading some of the reviews at IMDb or Amazon. This paper argues that there is a better way: reviewers movie scores and reviews can be analyzed with respect to their emotion content, aggregated and projected onto a movie, resulting in an emotion map for a movie. One can then make a decision on which movie to watch next by selecting those movies having emotion maps with certain emotion map patterns desirable for him/her. This paper is a first step towards the above-listed scenario.

2006

Research on emotional real-life data has to tackle the problem of their annotation. The annotation of emotional corpora raises the issue of how different coders perceive the same multimodal emotional behaviour. The long-term goal of this paper is to produce a guideline for the selection of annotators. The LIMSI team is working towards the definition of a coding scheme integrating emotion, context and multimodal annotations. We present the current defined coding scheme for emotion annotation, and the use of soft vectors for representing a mixture of emotions. This paper describes a perceptive test of emotion annotations and the results obtained with 40 different coders on a subset of complex real-life emotional segments selected from the EmoTV Corpus collected at LIMSI. The results of this first study validate previous annotations of emotion mixtures and highlight the difference of annotation between male and female coders.

Natural Language Processing and Information Systems, 2020

In this paper, we use movies and series subtitles and applied text mining and Natural Language Processing methods to evaluate emotions in videos. Three different word lexicons were used and one of the outcomes of this research is the generation of a secondary dataset with more than 3658 records which can be used for other data analysis and data mining research. We used our secondary dataset to find and display correlations between different emotions on the videos and the correlation between emotions on the movies and users’ scores on IMDb using the Pearson correlation method and found some statistically significant correlations.

Zenodo (CERN European Organization for Nuclear Research), 2023

Proceedings of the 9th Workshop on Slavic Natural Language Processing 2023 (SlavicNLP 2023)

This paper introduces Cro-FiReDa, a sentimentannotated dataset for Croatian in the domain of movie reviews. The dataset, which contains over 10,000 sentences, has been annotated at the sentence level. In addition to presenting the overall annotation process, we also present benchmark results based on the transformerbased fine-tuning approach.

Interspeech 2007, 2007

Corpus annotation is an important aspect in speech applications where stochastic models need to be trained and evaluated. Multimodal corpora are also annotated. Moreover, corpus annotation is an essential phase in the construction of emotion recognizer engines. Large corpora, as they are essential to construct representative knowledge bases, have been a problem for corpus annotators. Time consumed for labeling such corpora is very significant. Furthermore, manageability becomes more arduous and tedious. In this paper, we propose a semi-automatic tool, called BECAM tool, that will help corpus annotators in managing and annotating large sample emotion corpora.

Language Resources and Evaluation, 2010

In the context of Systemic Functional Linguistics, Appraisal is a theory describing the types of language utilised in communicating emotion and opinion. Robust automatic analyses of Appraisal could contribute in a number of ways to computational sentiment analysis by: distinguishing various types of evaluation, for example affect, ethics or aesthetics; discriminating between an author's opinions and the opinions of authors referenced by the author and determining the strength of evaluations. This paper reviews the typology described by Appraisal, presents a methodology for annotating Appraisal, and the use of this to annotate a corpus of book reviews. It discusses an inter-annotator agreement study, and considers instances of systematic disagreement that indicate areas in which Appraisal may be refined or clarified. Although the annotation task is difficult, there are many instances where the annotators agree; these are used to create a gold-standard corpus for future experimentation with Appraisal.

2003

Effectively labelingthe visual content ofmovies is essentialfor annotation. Wepresent theinteractive andadaptive i-Notationsystem, whichdescribes actors’names, automaticallyprocessesmultimodalinformation sources,and deals withavailable sources’varying quality. Itprovides the basisfor intelligentinteraction anddemonstratessignificantimprovements inannotationefficiency.

2021

We present first results of an exploratory study about sentiment analysis via different media channels on a German historical play. We propose the exploration of other media channels than text for sentiment analysis on plays since the auditory and visual channel might offer important cues for sentiment analysis. We perform a case study and investigate how textual, auditory (voice-based), and visual (face-based) sentiment analysis perform compared to human annotations and how these approaches differ from each other. As use case we chose Emilia Galotti by the famous German playwright Gotthold Ephraim Lessing. We acquired a video recording of a 2002 theater performance of the play at the "Wiener Burgtheater". We evaluate textual lexicon-based sentiment analysis and two state-of-the-art audio and video sentiment analysis tools. As gold standard we use speech-based annotations of three expert annotators. We found that the audio and video sentiment analysis do not perform better...