Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Eye-Tracking Databases for Emotion Recognition
Methods
Participants
Materials and Setup
Experimental Procedure
Data Analysis Methodology
Raw Eye-Tracking Data
Pupil Estimation Implications
Results
Machine Learning Analysis
Discussion
Conclusions
References
All Topics
Psychology
Experimental and Cognitive Psychology

eSEE-d: Emotional State Estimation Based on Eye-Tracking Dataset

Profile image of Dimitris ManousosDimitris Manousos

Brain Sciences

https://doi.org/10.3390/BRAINSCI13040589
visibility

…

description

22 pages

link

1 file

Abstract

Affective state estimation is a research field that has gained increased attention from the research community in the last decade. Two of the main catalysts for this are the advancement in the data analysis using artificial intelligence and the availability of high-quality video. Unfortunately, benchmarks and public datasets are limited, thus making the development of new methodologies and the implementation of comparative studies essential. The current work presents the eSEE-d database, which is a resource to be used for emotional State Estimation based on Eye-tracking data. Eye movements of 48 participants were recorded as they watched 10 emotion-evoking videos, each of them followed by a neutral video. Participants rated four emotions (tenderness, anger, disgust, sadness) on a scale from 0 to 10, which was later translated in terms of emotional arousal and valence levels. Furthermore, each participant filled three self-assessment questionnaires. An extensive analysis of the parti...

Related papers

Evaluation of Human Emotion from Eye Motions
Vidas Raudonis, Gintaras Dervinis, Andrius Vilkauskas

International Journal of Advanced Computer Science and Applications, 2013

The object of this paper is to develop an emotion recognition system that analysis the motion trajectory of the eye and gives the response on appraisal emotion. The emotion recognition solution is based on the data gathering using head mounted eye tracking device. The participants of experimental investigation were provided with a visual stimulus (PowerPoint slides) and the emotional feedback was determined by the combination of eye tracking device and emotion recognition software. The stimulus was divided in four groups by the emotion that should be triggered in the human, i.e., neutral, disgust, exhilaration and excited. Some initial experiments and the data on the recognition accuracy of the emotion from eye motion trajectory are provided along with the description of implemented algorithms.

View PDFchevron_right
Eye-Tracking Analysis for Emotion Recognition
Marcin Kołodziej

is article reports the results of the study related to emotion recognition by using eye-tracking. Emotions were evoked by presenting a dynamic movie material in the form of 21 video fragments. Eye-tracking signals recorded from 30 participants were used to calculate 18 features associated with eye movements (fixations and saccades) and pupil diameter. To ensure that the features were related to emotions, we investigated the influence of luminance and the dynamics of the presented movies. ree classes of emotions were considered: high arousal and low valence, low arousal and moderate valence, and high arousal and high valence. A maximum of 80% classification accuracy was obtained using the support vector machine (SVM) classifier and leave-one-subject-out validation method.

View PDFchevron_right
RESEARCH ARTICLE Predicting the Valence of a Scene from Observers ’ Eye Movements
Samia Nefti

2016

Multimedia analysis benefits from understanding the emotional content of a scene in a vari-ety of tasks such as video genre classification and content-based image retrieval. Recently, there has been an increasing interest in applying human bio-signals, particularly eye move-ments, to recognize the emotional gist of a scene such as its valence. In order to determine the emotional category of images using eye movements, the existing methods often learn a classifier using several features that are extracted from eye movements. Although it has been shown that eye movement is potentially useful for recognition of scene valence, the contribution of each feature is not well-studied. To address the issue, we study the contribu-tion of features extracted from eye movements in the classification of images into pleasant, neutral, and unpleasant categories. We assess ten features and their fusion. The features are histogram of saccade orientation, histogram of saccade slope, histogram of saccad...

View PDFchevron_right
Review of eye tracking metrics involved in emotional and cognitive processes
Giorgos Giannakakis

IEEE reviews in biomedical engineering, 2021

Eye behaviour provides valuable information revealing one's higher cognitive functions and state of affect. Although eye tracking is gaining ground in the research community, it is not yet a popular approach for the detection of emotional and cognitive states. In this paper, we present a review of eye and pupil tracking related metrics (such as gaze, fixations, saccades, blinks, pupil size variation, etc.) utilized towards the detection of emotional and cognitive processes, focusing on visual attention, emotional arousal and cognitive workload. Besides, we investigate their involvement as well as the computational recognition methods employed for the reliable emotional and cognitive assessment. The publicly available datasets employed in relevant research efforts were collected and their specifications and other pertinent details are described. The multimodal approaches which combine eye-tracking features with other modalities (e.g. biosignals), along with artificial intelligenc...

View PDFchevron_right
Integrated data analysis for the quantification of emotional responses during video observation
Margherita Pillan

2017 IEEE 3rd International Forum on Research and Technologies for Society and Industry (RTSI), 2017

The use of questionnaires at the end of a specific task only evaluates what is expressed by the conscious mind and, therefore, cannot give a complete representation of the individual's emotional state. By adding data from physiological measures, such as cerebral activity, skin conductance, heart rate and gaze position, more accurate information about cognitive engagement and emotional responses to a given task could be provided. This study aims to evaluate participants' emotional (arousal, valence) and cognitive (memorization effort, attention, pleasantness) responses toward two videos, through the integration of above cited measures. Our findings show that the two tested videos produce two different unconscious reactions (one video causes a significantly higher increase in heart rate and the other one requires higher memorization effort), while producing similar conscious responses (no statistically significant differences were found by analyzing questionnaires' answers). Further, eye tracking device provided a way to investigate reasons behind these differences. The results show that the integration of self-reported and biological measures with eye tracking data could effectively help to understand emotional and cognitive responses during video observation.

View PDFchevron_right
Neural Networks for Emotion Recognition Based on Eye Tracking Data
Juan Velasquez

2015 IEEE International Conference on Systems, Man, and Cybernetics, 2015

We present an approach for recognising emotions using information of the pupil. In last years, the pupil variables have been used as an assessment of emotional arousal. In this article, we generate signals of pupil size and gaze position monitored during image viewing. The emotions are provoked by visual stimuli of color images. Those images were taken from the International Affective Picture System which has been the reference for objective emotional assessment based on visual stimuli. For recognising the emotions we use the evolution of the eye tracking data during a window of time. The learning dataset is composed by the evolution of the pupil size and the gaze position, and labels associated to the emotional states. We study two kinds of learning tools based on Neural Networks. We obtain promising empirical results that show the potential of using temporal learning tools for emotion recognition.

View PDFchevron_right
Towards Estimation of Emotions From Eye Pupillometry With Low-Cost Devices
Mamoona Asghar

IEEE Access, 2021

Emotional care is important for some patients and their caregivers. Within a clinical or home care situation, technology can be employed to remotely monitor the emotional response of such people. This paper considers pupillometry as a non-invasive way of classifying an individual's emotions. Standardized audio signals were used to emotionally stimulate the test subjects. Eye pupil images of up to 32 subjects of different genders were captured as video images by low-cost, infrared, Raspberry Pi board cameras. By processing of the images, a dataset of pupil diameters according to gender and age characteristics was established. Appropriate statistical tests for inference of the emotional state were applied to that dataset to establish the subjects' emotional states in response to the audio stimuli. Results showed agreement between the test subjects' opinions of their emotional state and the classification of emotions according to the range of pupil diameters found using the described method.

View PDFchevron_right
Predicting the Valence of a Scene from Observers’ Eye Movements
Seppo Laukka

PLOS ONE, 2015

Multimedia analysis benefits from understanding the emotional content of a scene in a variety of tasks such as video genre classification and content-based image retrieval. Recently, there has been an increasing interest in applying human bio-signals, particularly eye movements, to recognize the emotional gist of a scene such as its valence. In order to determine the emotional category of images using eye movements, the existing methods often learn a classifier using several features that are extracted from eye movements. Although it has been shown that eye movement is potentially useful for recognition of scene valence, the contribution of each feature is not well-studied. To address the issue, we study the contribution of features extracted from eye movements in the classification of images into pleasant, neutral, and unpleasant categories. We assess ten features and their fusion. The features are histogram of saccade orientation, histogram of saccade slope, histogram of saccade length, histogram of saccade duration, histogram of saccade velocity, histogram of fixation duration, fixation histogram, top-ten salient coordinates, and saliency map. We utilize machine learning approach to analyze the performance of features by learning a support vector machine and exploiting various feature fusion schemes. The experiments reveal that 'saliency map', 'fixation histogram', 'histogram of fixation duration', and 'histogram of saccade slope' are the most contributing features. The selected features signify the influence of fixation information and angular behavior of eye movements in the recognition of the valence of images.

View PDFchevron_right
Real-time estimation of emotional experiences from facial expressions
Timo Partala

Interacting with Computers, 2006

The present aim was to develop methods that estimate emotional experiences in real time from the electromyographic activity of two facial muscles: zygomaticus major (activated when smiling) and corrugator supercilii (activated when frowning). Ten subjects were stimulated with a series of emotionally arousing pictures and videos. After each stimulus the subjects rated the valence of their emotional experience on a nine-point bipolar dimensional scale. At the same time the computer estimated the subjects' ratings on the basis of their electrical facial activity during each stimulation with 70 computational models. The models estimated the subjects' ratings either categorically or dimensionally with regression models. The best categorical models were able to estimate negative and positive ratings with an average accuracy of over 70 and 80% for pictures and videos, respectively. The best correlations between the human ratings and machine estimations formed with the regression models were high (rO0.9). These findings indicate that models estimating psychoemotional experiences on the basis of facial activity can be created successfully in several ways. q 2005 Published by Elsevier B.V.

View PDFchevron_right
Towards Automated Inferencing of Emotional State from Face Images
G. Tsihrintzis

International Conference on Software and Data Technologies, 2007

Automated facial expression classification is very important in the design of new human-computer interaction modes and multimedia interactive services and arises as a difficult, yet crucial, pattern recognition problem. Recently, we have been building such a system, called NEU-FACES, which processes multiple camera images of computer user faces with the ultimate goal of determining their affective state. In here, we present results from an empirical study we conducted on how humans classify facial expressions, corresponding error rates, and to which degree a face image can provide emotion recognition from the perspective of a human observer. This study lays related system design requirements, quantifies statistical expression recognition performance of humans, and identifies quantitative facial features of high expression discrimination and classification power.

View PDFchevron_right

References (76)

  1. Ekman, P.; Friesen, W.V. Constants across cultures in the face and emotion. J. Personal. Soc. Psychol. 1971, 17, 124-129. [CrossRef] [PubMed]
  2. Ekman, P.; Friesen, W.V. Facial Action Coding System: A Technique for the Measurement of Facial Movement; Consulting Psychologists Press: Palo Alto, CA, USA, 1978.
  3. Toisoul, A.; Kossaifi, J.; Bulat, A.; Tzimiropoulos, G.; Pantic, M. Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nat. Mach. Intell. 2021, 3, 42-50. [CrossRef]
  4. Russell, J.A. A circumplex model of affect. J. Personal. Soc. Psychol. 1980, 39, 1161-1178. [CrossRef]
  5. Aracena, C.; Basterrech, S.; Snáel, V.; Velásquez, J.D. Neural Networks for Emotion Recognition Based on Eye Tracking Data. In Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China, 9-12 October 2015; pp. 2632-2637.
  6. Di Luzio, F.; Rosato, A.; Panella, M. A randomized deep neural network for emotion recognition with landmarks detection. Biomed. Signal Process. Control. 2023, 81, 104418. [CrossRef]
  7. Zhai, J.; Barreto, A.; Chin, C.; Li, C. Realization of stress detection using psychophysiological signals for improvement of human-computer interactions. In Proceedings of the IEEE SoutheastCon, 2005, Ft. Lauderdale, FL, USA, 8-10 April 2005; pp. 415-420. [CrossRef]
  8. Lin, Z.; Liu, Y.; Wang, H.; Liu, Z.; Cai, S.; Zheng, Z.; Zhou, Y.; Zhang, X. An eye tracker based on webcam and its preliminary application evaluation in Chinese reading tests. Biomed. Signal Process. Control 2022, 74, 103521. [CrossRef]
  9. JothiPrabha, A.; Bhargavi, R.; Deepa Rani, B.V. Prediction of dyslexia severity levels from fixation and saccadic eye movement using machine learning. Biomed. Signal Process. Control 2023, 79, 104094. [CrossRef]
  10. Ktistakis, E.; Skaramagkas, V.; Manousos, D.; Tachos, N.S.; Tripoliti, E.; Fotiadis, D.I.; Tsiknakis, M. COLET: A dataset for COgnitive workLoad estimation based on eye-tracking. Comput. Methods Programs Biomed. 2022, 224, 106989. [CrossRef]
  11. Alghowinem, S.; Goecke, R.; Wagner, M.; Parker, G.; Breakspear, M. Eye movement analysis for depression detection. In Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, VIC, Australia, 15-18 September 2013; pp. 4220-4224. [CrossRef]
  12. Al-gawwam, S.; Benaissa, M. Depression Detection From Eye Blink Features. In Proceedings of the 2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Louisville, KY, USA, 6-8 December 2018; pp. 388-392. [CrossRef]
  13. Zheng, L.; Mountstephens, J.; Teo, J. Multiclass Emotion Classification Using Pupil Size in VR: Tuning Support Vector Machines to Improve Performance. J. Physics Conf. Ser. 2020, 1529. [CrossRef]
  14. Tarnowski, P.; Kołodziej, M.; Majkowski, A.; Rak, R.J. Eye-Tracking Analysis for Emotion Recognition. Comput. Intell. Neurosci. 2020, 2020, 2909267. [CrossRef]
  15. Zheng, W.L.; Dong, B.N.; Lu, B.L. Multimodal emotion recognition using EEG and eye tracking data. In Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014, Chicago, IL, USA, 26-30 August 2014; pp. 5040-5043. [CrossRef]
  16. Skaramagkas, V.; Ktistakis, E.; Manousos, D.; Tachos, N.S.; Kazantzaki, E.; Tripoliti, E.E.; Fotiadis, D.I.; Tsiknakis, M. A machine learning approach to predict emotional arousal and valence from gaze extracted features. In Proceedings of the 2021 IEEE 21st International Conference on Bioinformatics and Bioengineering (BIBE), Kragujevac, Serbia, 25-27 October 2021; pp. 1-5. [CrossRef]
  17. Raudonis, V.; Dervinis, G.; Vilkauskas, A.; Paulauskaite, A.; Kersulyte, G. Evaluation of Human Emotion from Eye Motions. Int. J. Adv. Comput. Sci. Appl. 2013, 4, 79-84. [CrossRef]
  18. Skaramagkas, V.; Giannakakis, G.; Ktistakis, E.; Manousos, D.; Karatzanis, I.; Tachos, N.; Tripoliti, E.E.; Marias, K.; Fotiadis, D.I.; Tsiknakis, M. Review of eye tracking metrics involved in emotional and cognitive processes. IEEE Rev. Biomed. Eng. 2021, 16, 260-277.
  19. Kimble, M.O.; Fleming, K.; Bandy, C.; Kim, J.; Zambetti, A. Eye tracking and visual attention to threating stimuli in veterans of the Iraq war. J. Anxiety Disord. 2010, 24, 293-299. [CrossRef] [PubMed]
  20. Quigley, L.; Nelson, A.L.; Carriere, J.; Smilek, D.; Purdon, C. The effects of trait and state anxiety on attention to emotional images: an eye-tracking study. Cogn. Emot. 2012, 26, 1390-1411. [CrossRef] [PubMed]
  21. Holmqvist, K.; Nystrom, M.; Andersson, R.; Dewhurst, R.; Jarodzka, H.; Weijer, J.V.D. Eye Tracking: A Comprehensive Guide to Methods and Measures; Reprint Edition; Oxford University Press: Oxford, UK, 2015; ISBN 9780198738596.
  22. Mollahosseini, A.; Hasani, B.; Mahoor, M.H. AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild. IEEE Trans. Affect. Comput. 2019, 10, 18-31. [CrossRef]
  23. Giannopoulos, P.; Perikos, I.; Hatzilygeroudis, I., Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013. In Advances in Hybridization of Intelligent Methods: Models, Systems and Applications; Hatzilygeroudis, I., Palade, V., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 1-16.
  24. Kosti, R.; Alvarez, J.M.; Recasens, A.; Lapedriza, A. EMOTIC: Emotions in Context Dataset. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, 21-26 July 2017; pp. 2309-2317.
  25. Xue, J.; Wang, J.; Hu, S.; Bi, N.; Lv, Z. Ovpd: Odor-video elicited physiological signal database for emotion recognition. IEEE Trans. Instrum. Meas. 2022, 71, 1-12. [CrossRef]
  26. Wu, M.; Teng, W.; Fan, C.; Pei, S.; Li, P.; Lv, Z. An investigation of olfactory-enhanced video on eeg-based emotion recognition. IEEE Trans. Neural Syst. Rehabil. Eng. 2023, 31, 1602-1613. [CrossRef]
  27. Zlatintsi, A.; Koutras, P.; Evangelopoulos, G.; Malandrakis, N.; Efthymiou, N.; Pastra, K.; Potamianos, A.; Maragos, P. COGN- IMUSE: a multimodal video database annotated with saliency, events, semantics and emotion with application to summarization. EURASIP J. Image Video Process. 2017, 2017. [CrossRef]
  28. Fan, S.; Shen, Z.; Jiang, M.; Koenig, B.L.; Xu, J.; Kankanhalli, M.S.; Zhao, Q. Emotional Attention: A Study of Image Sentiment and Visual Attention. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18-23 June 2018; pp. 7521-7531. [CrossRef]
  29. Ramanathan, S.; Katti, H.; Sebe, N.; Kankanhalli, M.; Chua, T.-S. An Eye Fixation Database for Saliency Detection in Images. In Proceedings of the Computer Vision-ECCV 2010; Daniilidis, K., Maragos, P., Paragios, N., Eds.; Springer: Berlin/Heidelberg, Germany, 2010; pp. 30-43.
  30. Borji, A.; Itti, L. CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research. arXiv 2015, arXiv:1505.03581.
  31. Katsigiannis, S.; Ramzan, N. Dreamer: A database for emotion recognition through eeg and ecg signals from wireless low-cost off-the-shelf devices. IEEE J. Biomed. Health Inform. 2018, 22, 98-107. [CrossRef]
  32. Song, T.; Zheng, W.; Lu, C.; Zong, Y.; Zhang, X.; Cui, Z. Mped: A multi-modal physiological emotion database for discrete emotion recognition. IEEE Access 2019, 7, 12177-12191. [CrossRef]
  33. Schaefer, A.; Nils, F.; Sanchez, X.; Philippot, P. Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers. Cogn. Emot. Cogn. Emot. 2010, 24, 1153-1172. [CrossRef]
  34. Johnson-Laird, P.N.; Oatley, K. The language of emotions: An analysis of a semantic field. Cogn. Emot. 1989, 3, 81-123. [CrossRef]
  35. Ekman, P. An Argument for Basic Emotions. Cogn. Emot. 1992, 6, 169-200. [CrossRef]
  36. Quinto, L.; Thompson, W.F. Composers and performers have different capacities to manipulate arousal and valence. Psychomusicol. Music. Mind Brain 2013, 23, 137-150. [CrossRef]
  37. Zhao, G.; Zhang, Y.; Ge, Y. Frontal EEG asymmetry and middle line power difference in discrete emotions. Front. Behav. Neurosci. 2018, 12, 225. [CrossRef] [PubMed]
  38. Martínez-Rodrigo, A.; Fernandez-Aguilar, L.; Zangroniz, R.; Latorre, J.M.; Pastor, J.M.; Fernandez-Caballero, A. Film mood induction and emotion classification using physiological signals for health and wellness promotion in older adults living alone. Expert Syst. 2020, 37. [CrossRef]
  39. Gross, J.J.; Levenson, R.W. Emotion elicitation using films. Cogn. Emot. 1995, 9, 87-108. [CrossRef]
  40. Radloff, L.S. The CES-D Scale: A Self-Report Depression Scale for Research in the General Population. Appl. Psychol. Meas. 1977, 1, 385-401. [CrossRef]
  41. Fountoulakis, K.; Iacovides, A.; Kleanthous, S.; Samolis, S.; Kaprinis, S.G.; Sitzoglou, K.; Kaprinis, G.S.; Bech, P. Reliability, Validity and Psychometric Properties of the Greek Translation of the Center for Epidemiological Studies-Depression (CES-D) Scale. BMC Psychiatry 2001, 1, 3. [CrossRef]
  42. Spielberger, C.D.; Gorsuch, R.L.; Lushene, R.; Vagg, P.R.; Jacobs, G.A. Manual for the State-Trait Anxiety Inventory; Consulting Psychologists Press: Palo Alto, CA, USA, 1983.
  43. Fountoulakis, K.N.; Papadopoulou, M.; Kleanthous, S.; Papadopoulou, A.; Bizeli, V.; Nimatoudis, I.; Iacovides, A.; Kaprinis, G.S. Reliability and psychometric properties of the Greek translation of the State-Trait Anxiety Inventory form Y: Preliminary data. Ann. Gen. Psychiatry 2006, 5, 2. [CrossRef]
  44. Jolliffe, D.; Farrington, D.P. Development and validation of the Basic Empathy Scale. J. Adolesc. 2006, 29, 589-611. [CrossRef] [PubMed]
  45. Stavrinides, P.; Georgiou, S.; Theofanous, V. Bullying and empathy: A short-term longitudinal investigation. Educ. Psychol. 2010, 30, 793-802. [CrossRef]
  46. Kassner, M.; Patera, W.; Bulling, A. Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-Based Interaction. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, Seattle, WA, USA, 13-17 September 2014; pp. 1151-1160. [CrossRef]
  47. Plainis, S.; Orphanos, Y.; Tsilimbaris, M.K. A Modified ETDRS Visual Acuity Chart for European-Wide Use. Optom Vis Sci. 2007, 84, 647-653. [CrossRef] [PubMed]
  48. How Is the Sample Percentage Calculated in Studio, Lab and Controller? Available online: https://connect.tobii.com/s/article/ Sample-percentage-calculated-in-Studio-Lab-and-Controller? (accessed on 28 March 2023).
  49. Hollander, J.; Huette, S. Extracting blinks from continuous eye-tracking data in a mind wandering paradigm. Conscious. Cogn. 2022, 100, 103303. [CrossRef]
  50. Faber, M.; Bixler, R.; D'Mello, S.K. An automated behavioral measure of mind wandering during computerized reading. Behav. Res. Methods 2018, 50, 134-150. [CrossRef]
  51. Killingsworth, M.A.; Gilbert, D.T. A wandering mind is an unhappy mind. Science 2010, 330, 932. [CrossRef]
  52. Chartier, S.; Renaud, P. An Online Noise Filter for Eye-Tracker Data Recorded in a Virtual Environment. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications; Association for Computing Machinery: New York, NY, USA, 26 March 2008; pp. 153-156.
  53. Duchowski, A.T. Eye Movement Analysis. In Eye Tracking Methodology: Theory and Practice; Springer London: London, UK, 2003; pp. 111-128.
  54. Salvucci, D.D.; Goldberg, J.H. Identifying Fixations and Saccades in Eye-Tracking Protocols. In Proceedings of the 2000 Sympo- sium on Eye Tracking Research & Applications; Association for Computing Machinery: New York, NY, USA, 8 November 2000; pp. 71-78.
  55. Andersson, R.; Larsson, L.; Holmqvist, K.; Stridh, M.; Nyström, M. One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms. Behav. Res. Methods 2016, 49, 616-637. . [CrossRef]
  56. Zaidawi, S.M.K.A.; Prinzler, M.H.U.; Lührs, J.; Maneth, S. An Extensive Study of User Identification via Eye Movements across Multiple Datasets. arXiv 2021, arXiv:2111.05901.
  57. Ellis, C.J. The pupillary light reflex in normal subjects. Br. J. Ophthalmol. 1981, 65, 754-759. [CrossRef]
  58. Atchison, D.A.; Girgenti, C.C.; Campbell, G.M.; Dodds, J.P.; Byrnes, T.M.; Zele, A.J. Influence of field size on pupil diameter under photopic and mesopic light levels. Clin. Exp. Optom. 2011, 94, 545-548. [CrossRef]
  59. Watson, A.B.; Yellott, J.I. A unified formula for light-adapted pupil size. J. Vis. 2012, 12, 12. [CrossRef] [PubMed]
  60. Sun, X.; Hong, T.; Li, C.; Ren, F. Hybrid spatiotemporal models for sentiment classification via galvanic skin response. Neurocomputing 2019, 358, 385-400. [CrossRef]
  61. Meghanathan, N. Assortativity Analysis of Real-World Network Graphs based on Centrality Metrics. Comput. Inf. Sci. 2016, 9, 7. [CrossRef]
  62. Philippot, P. Inducing and assessing differentiated emotion-feeling states in the laboratory. Cogn. Emot. 1993, 7, 171-193.
  63. Singmann, H.; Kellen, D. An Introduction to Mixed Models for Experimental Psychology. In New Methods in Cognitive Psychology; Routledge: Oxfordshire, UK, 2019; p. 28, ISBN 9780429318405.
  64. Chen, Z.; Pang, M.; Zhao, Z.; Li, S.; Miao, R.; Zhang, Y.; Feng, X.; Feng, X.; Zhang, Y.; Duan, M.; et al. Feature selection may improve deep neural networks for the bioinformatics problems. Bioinformatics 2019, 36, 1542-1552. [CrossRef] [PubMed]
  65. Gholamy, A.; Kreinovich, V.; Kosheleva, O. Why 70/30 or 80/20 Relation between Training and Testing Sets: A Pedagogical Explanation; ScholarWorks: Allendale Charter Twp, MI, USA, 2018.
  66. Nair, V.; Hinton, G.E. Rectified Linear Units Improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning; Omnipress: Madison, WI, USA, 21 June 2010; pp. 807-814.
  67. Narayan, S. The generalized sigmoid activation function: Competitive supervised learning. Inf. Sci. 1997, 99, 69-82. [CrossRef]
  68. Bouma, H. Size of the Static Pupil as a Function of Wave-length and Luminosity of the Light Incident on the Human Eye. Nature 1962, 193, 690-691. [CrossRef]
  69. Carle, C.F.; James, A.C.; Maddess, T. The Pupillary Response to Color and Luminance Variant Multifocal Stimuli. Investig. Ophthalmol. Vis. Sci. 2013, 54, 467-475. [CrossRef]
  70. Davidson, R.J. Affective Style and Affective Disorders: Perspectives from Affective Neuroscience. Cogn. Emot. 1998, 12, 307-330.
  71. Verduyn, P.; Delaveau, P.; Rotge, J.Y.; Fossati, P.; Mechelen, I.V. Determinants of Emotion Duration and Underlying Psychological and Neural Mechanisms. Emot. Rev. 2015, 7, 330-335. [CrossRef]
  72. Mechelen, I.V.; Verduyn, P.; Brans, K. The Duration of Emotional Episodes. In Changing Emotions; Psychology Press: London, UK, 2013; ISBN 9780203075630.
  73. Frijda, N.H. The laws of emotion. Am. Psychol. 1988, 43 , 349-358. [CrossRef] [PubMed]
  74. Verduyn, P.; van Mechelen, I.; Tuerlinckx, F. The relation between event processing and the duration of emotional experience. Emotion 2011, 11, 20-28. [CrossRef] [PubMed]
  75. Sbarra, D.A. Predicting the onset of emotional recovery following nonmarital relationship dissolution: Survival analyses of sadness and anger. Personal. Soc. Psychol. Bull. 2006, 32, 298-312. [CrossRef] [PubMed]
  76. Disclaimer/Publisher's Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Related papers

Exploring Eye Activity as an Indication of Emotional States Using an Eye-Tracking Sensor
Roland Goecke

Studies in Computational Intelligence, 2014

The automatic detection of human emotional states has been of great interest lately for its applications not only in the Human-Computer Interaction field, but also for its applications in psychological studies. Using an emotion elicitation paradigm, we investigate whether eye activity holds discriminative power for detecting affective states. Our emotion elicitation paradigm includes induced emotions by watching emotional movie clips and spontaneous emotions elicited by interviewing participants about emotional events in their life. To reduce gender variability, the selected participants were 60 female native Arabic speakers (30 young adults, and 30 mature adults). In general, the automatic classification results using eye activity were reasonable, giving 66% correct recognition rate on average. Statistical measures show statistically significant differences in eye activity patterns between positive and negative emotions. We conclude that eye activity, including eye movement, pupil dilation and pupil invisibility could be used as a complementary cues for the automatic recognition of human emotional states.

View PDFchevron_right
A machine learning approach to predict emotional arousal and valence from gaze extracted features
Dimitris Manousos

2021 IEEE 21st International Conference on Bioinformatics and Bioengineering (BIBE), 2021

In the last years, many studies have been investigating emotional arousal and valence. Most of them have focused on the use of physiological signals such as EEG or EMG, cardiovascular measures or skin conductance. However, eye related features have proven to be very helpful and easy to use metrics, especially pupil size and blink activity. The aim of this study is to predict emotional arousal and valence levels which are induced during emotionally charged situations from eye related features. For this reason, we performed an experimental study where the participants watched emotion-eliciting videos and self-assessed their emotions, while their eye movements were being recorded. In this work, several classifiers such as KNN, SVM, Naive Bayes, Trees and Ensemble methods were trained and tested. Finally, emotional arousal and valence levels were predicted with 85 and 91% efficiency, respectively.

View PDFchevron_right
An exploratory study of detecting emotion states using eye-tracking technology
Sharifa Alghowinem

Studying eye movement has proven to be useful in the study of detecting and understanding human emotional states. This paper aims to investigate eye movement features: pupil size, time of first fixation, first fixation duration, fixation duration and fixation count in clips emotional stimulation. Thirty seven subjects' pupil responses were measured while watching two pleasant and unpleasant emotional clips. The results showed that the fixation duration and fixation count significantly different between pleasant and unpleasant clip arousal. These results suggest that the measurement of eye fixation may be a potentially useful computer input for detecting positive and negative emotional state.

View PDFchevron_right
Emotion Recognition Using Eye-Tracking: Taxonomy, Review and Current Challenges
James Mountstephens, JIA ZHENG LIM

Sensors, 2020

The ability to detect users’ emotions for the purpose of emotion engineering is currently one of the main endeavors of machine learning in affective computing. Among the more common approaches to emotion detection are methods that rely on electroencephalography (EEG), facial image processing and speech inflections. Although eye-tracking is fast in becoming one of the most commonly used sensor modalities in affective computing, it is still a relatively new approach for emotion detection, especially when it is used exclusively. In this survey paper, we present a review on emotion recognition using eye-tracking technology, including a brief introductory background on emotion modeling, eye-tracking devices and approaches, emotion stimulation methods, the emotional-relevant features extractable from eye-tracking data, and most importantly, a categorical summary and taxonomy of the current literature which relates to emotion recognition using eye-tracking. This review concludes with a dis...

View PDFchevron_right
Eye gaze patterns in emotional pictures
Enzo Scilingo, Antonio Lanatà

Journal of Ambient Intelligence and Humanized Computing, 2013

This paper reports on a preliminary study aiming at investigating the eye gaze pattern and pupil size variation to discriminate emotional states induced by looking at pictures having different arousal content. A wearable and wireless eye gaze tracking system, hereinafter called HATCAM, which was able to robustly detect eye tracking and pupil area was used. A group of ten volunteers was presented with a set of neutral and arousal pictures extracted from the International Affective Picture System according to an ad-hoc experimental protocol. A set of features was extracted from eye gaze patterns and pupil size variations and used to classify the two classes of pictures. Although preliminary, results are very promising for affective computing applications.

View PDFchevron_right

Related topics

  • Psychology
  • Eye tracking
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved