A Real-Time Facial Expression Recognition System for Online Games

International Journal of Technology and Educational Marketing, 2014

Enabling computer systems to track and recognize facial expressions and then infer emotions from about real time video is a challenging research topic. In this work, a real time approach to emotion recognition through facial expression in live video is introduced. Several automatic methods for face localization, facial feature tracker, and facial expression recognition are employed. A robust tracking is achieved by using a face mask to resolve mismatches that could be generated during the tracking process. Action units (AUs) are then built to recognize the facial expression in each frame. The main objective of this work is to provide a prediction ability of a human behavior such as a crime, angry or for being nervous.

… and Workshops, 2009. …, 2009

2009

Cognitive Technologies, 2006

In current dialogue systems the use of speech as an input modality is common. But this modality is only one of those human beings use. In human-human interaction people use gestures to point or facial expressions to show their moods as well. To give modern systems a chance to read information from all modalities used by humans, these systems must have multimodal user interfaces. The SMARTKOM system has such a multimodal interface that analyzes facial expression, speech and gesture simultaneously. Here we present the module that fulfills the task of facial expression analysis in order to identify the internal state of a user. In the following we first describe the state of the art in emotion and user state recognition by analyzing facial expressions. Next, we describe the facial expression recognition module. After that we present the experiments and results for recognition of user states. We summarize our results in the last section.

2005

Automatic facial expression recognition is a research topic with interesting applications in the field of human-computer interaction, psychology and product marketing. The classification accuracy for an automatic system which uses static images as input is however largely limited by the image quality, lighting conditions and the orientation of the depicted face. These problems can be partially overcome by using a holistic model based approach called the Active Appearance Model. A system will be described that can classify expressions from one of the emotional categories joy, anger, sadness, surprise, fear and disgust with remarkable accuracy. It is also able to detect smaller, local facial features based on minimal muscular movements described by the Facial Action Coding System (FACS). Finally, we show how the system can be used for expression analysis and synthesis.

Proceedings of the 2007 international conference on Computer systems and technologies - CompSysTech '07, 2007

The paper highlights the performance of video sequence-oriented facial expression recognition using Active Appearance Model-AAM, in a comparison with the analysis based on still pictures. The AAM is used to extract relevant information regarding the shapes of the faces to be analyzed. Specific key points from a Facial Characteristic Point-FCP model are used to derive the set of features. These features are used for the classification of the expressions of a new face sample into the prototypic emotions. The classification method uses Support Vector Machines.

Face and Gesture 2011, 2011

We present the Computer Expression Recognition Toolbox (CERT), a software tool for fully automatic real-time facial expression recognition, and officially release it for free academic use. CERT can automatically code the intensity of 19 different facial actions from the Facial Action Unit Coding System (FACS) and 6 different protoypical facial expressions. It also estimates the locations of 10 facial features as well as the 3-D orientation (yaw, pitch, roll) of the head. On a database of posed facial expressions, Extended Cohn-Kanade (CK+ [1]), CERT achieves an average recognition performance (probability of correctness on a two-alternative forced choice (2AFC) task between one positive and one negative example) of 90.1% when analyzing facial actions. On a spontaneous facial expression dataset, CERT achieves an accuracy of nearly 80%. In a standard dual core laptop, CERT can process 320 × 240 video images in real time at approximately 10 frames per second.

In Proceedings of IEEE …, 2003

Most facial expression analysis systems attempt to recognize facial expressions from data collected in a highly controlled laboratory with very high resolution frontal faces (face regions greater than 200 x 200 pixels) and cannot handle large head motions. In real environments such as smart meetings, a facial expression analysis system must be able to automatically recognize expressions at lower resolution and handle the full range of head motion. This paper describes a real-time system to automatically recognize facial expressions in relatively low resolution face images (around 50x70 to 75x100 pixels). To handle the full range of head motion, we detect the head instead of the face. Then the head pose is estimated based on the detected head. For frontal and near frontal views of the face, the location and shape features are computed for expression recognition. Our system successfully deals with complex real world interactions, as demonstrated on the PETS2003 dataset.

In human – human communication we use verbal, vocal and non-verbal signals to communicate with others. Facial expressions are a form of non-verbal communication, recognizing them helps to improve the human – machine interaction. This paper proposes an approach for recognizing the facial expressions using camera images. The proposed system tracks the human face using high-speed cameras, extracts the pose-normalized image and it is able to learn a dynamic model of the facial expressions.

2008

The recognition of emotional information is a key step toward giving computers the ability to interact more naturally and intelligently with people. We present a simple and computationally feasible method to perform automatic emotional classification of facial expressions. We propose the use of a set of characteristic facial points (that are part of the MPEG4 feature points) to extract relevant emotional information (basically five distances, presence of wrinkles in the eyebrow and mouth shape). The method defines and detects the six basic emotions (plus the neutral one) in terms of this information and has been fine-tuned with a database of more than 1500 images. The system has been integrated in a 3D engine for managing virtual characters, allowing the exploration of new forms of natural interaction.