EXPERTNet

2017

Facial expression recognition (FER) is a research area that consists of classifying the human emotions through the expressions on their face. It can be used in applications such as biometric security, intelligent human-computer interaction, robotics, and clinical medicine for autism, depression, pain and mental health problems. This dissertation investigates the advanced technologies for facial expression analysis and develops the artificial intelligent systems for practical applications. The first part of this work applies geometric and texture domain feature extractors along with various machine learning techniques to improve FER. Advanced 2D and 3D facial processing techniques such as Edge Oriented Histograms (EOH) and Facial Mesh Distances (FMD) are then fused together using a framework designed to investigate their individual and combined domain performances. Following these tests, the face is then broken down into facial parts using advanced facial alignment and localising techniques. Deep learning in the form of Convolutional Neural Networks (CNNs) is also explored also FER. A novel approach is used for the deep network architecture design, to learn the facial parts jointly, showing an improvement over using the whole face. Joint Bayesian is also adapted in the form of metric learning, to work with deep feature representations of the facial parts. This provides a further improvement over using the deep network alone. Dynamic emotion content is explored as a solution to provide richer information than still images. The motion occurring across the content is initially captured using the Motion History Histogram descriptor (MHH) and is critically evaluated. Based on this observation, several improvements are proposed through extensions such as Average Spatial Pooling Multi-scale Motion History Histogram (ASMMHH). This extension adds two modifications, first is to view the content in different spatial dimensions through spatial pooling; influenced by the structure of CNNs. The other modification is to capture motion at different speeds. Combined, they have provided better performance over MHH, and other popular techniques like Local Binary Patterns-Three Orthogonal Planes (LBP-TOP). Finally, the dynamic emotion content is observed in the feature space, with sequences of images represented as sequences of extracted features. A novel technique called Facial Dynamic History Histogram (FDHH) is developed to capture patterns of variations within the sequence of features; an approach not seen before. FDHH is applied in an end to end framework for applications in Depression analysis and evaluating the induced emotions through a large set of video clips from various movies. With the combination of deep learning techniques and FDHH, state-of-the-art results are achieved for Depression analysis.

Pakistan Journal of Engineering and Technology

Human convey their message in different forms. Expressing their emotions and moods through their facial expression is one of them. In this work, to avoid the traditional process of feature extraction (Geometry based method, Template based method, and Appearance based method), CNN model is used as a feature extractor for emotion detection using facial expression. In this study we also used three pre-trained models VGG-16, ResNet-50, Inception-V3. This Experiment is done on Fer-2013 facial expression dataset and Cohn Extended (CK+) dataset. By using FER-2013 dataset the accuracy rates for CNN, ResNet-50, VGG-16 and Inception-V3 are 76.74%, 85.71%, 85.78%s, 97.93% respectively. Similarly, the experimental results using CK+ dataset showed the accuracy rates for CNN, ResNet- 50, VGG-16 and Inception-V3 are 84.18%, 92.91%, 91.07%, and 73.16% respectively. The experimental results showed exceptional results for Inception-V3 with 97.93% using FER-2013 dataset and ResNet-50 with 91.92% using...

Facial expressions reflect human emotions and an individual's intentions. To detect facial expressions by human beings is a very easy task whereas it’s a very difficult task using computers. They perform a vigorous part in everyday life. It is a non-verbal mode that may include feelings, opinions, and thoughts without speaking. Deep neural networks, Convolutional Neural Networks, Neural networks, Artificial Intelligence, Fuzzy Logic, and Machine Learning are the different technologies used to detect facial expressions. To detect facial expressions, static images, video, webcam data, or real-time images can be used. This research paper focused on developing the SMM Facial Expression dataset and proposes a convolutional neural network model to identify facial expressions. The proposed method was tested on two different benchmarked datasets namely FER2013 and CK+ for facial expression detection. We have explored the proposed model on CK+ and achieved 93.94% accuracy and 67.18 % for...

Revue d'Intelligence Artificielle

Facial Expression Recognition (FER) systems are helpful in a wide range of industries, including healthcare, social marketing, targeted advertising, the music industry, school counseling systems, and detection in the police sector. In this research, using Deep Convolutional Neural Networks (DCNN) architecture, specifically from the EfficientNet family (EfficientNet-B0, Efficient-Net-B01, EfficientNet-B02, EfficientNet-B03, EfficientNet-B04, EfficientNet-B05, EfficientNet-B06, and EfficientNet-B07) has previously gone through a combined scaling process of combined dept, width and resolution. First, the previously frozen sublayer EfficientNet model was used for feature extraction. Next, the layer closer to the output layer is melted by several layers to be retrained in order to recognize the pattern of the CK+ and JAFFE data sets. This process is called the transfer learning technique. This technique is very powerful for working on relatively small data sets, namely CK+ and JAFFE. The main of this research is to improve the accuracy and performance of facial expression recognition models with a transfer learning approach using EfficientNet pre-trained with fine-tuning strategy. Our proposed method, specifically using the EfficientNet-B0 architecture, achieves superior performance for each of the CK+ and JAFFE datasets, achieving 99.57% and 100% accuracy in the test set respectively.

International Journal of Engineering Trends and Technology, 2019

Facial expression recognition (FER) is now getting extensively popular because of its ability to predict an unknown data-set, and to its extent with some accuracy. An average human being possesses or shows seven different expressions based on the situation, namely anger, sad, happy, surprise, disgust, neutral and scared. Each individual has a unique way to express the afore-mentioned expressions and hence the term "an unknown data-set". To identify human's present mindset through facial expressions, many data sets are prepared based on face components (such as lips, cheek, nose, eyes and eye brows etc.,) dislocations and elasticity of all the facial parts. Many facial recognition systems are functioning on muscle distribution analysis from the mother image set's pixel parameters. This research paper is going to present about image pre processing, facial expression learning methods, classification methods and implementation of FaceEx algorithm for facial expression analysis through FER2013 CNN data sets and Viola-Jones Principle.

ArXiv, 2018

Facial expression recognition has been an active area in computer vision with application areas including animation, social robots, personalized banking, etc. In this study, we explore the problem of image classification for detecting facial expressions based on features extracted from pre-trained convolutional neural networks trained on ImageNet database. Features are extracted and transferred to a Linear Support Vector Machine for classification. All experiments are performed on two publicly available datasets such as JAFFE and CK+ database. The results show that representations learned from pre-trained networks for a task such as object recognition can be transferred, and used for facial expression recognition. Furthermore, for a small dataset, using features from earlier layers of the VGG19 network provides better classification accuracy. Accuracies of 92.26% and 92.86% were achieved for the CK+ and JAFFE datasets respectively.

Biochemical and Biophysical Research Communications, 2021

Automatic face expression recognition is an exigent research subject and a challenge in computer vision. It is an interdisciplinary domain standing at the crossing of behavioural science, psychology, neurology, and artificial intelligence. Human-robot interaction is getting more significant with the automation of every field, like treating autistic patients, child therapy, babysitting, etc. In all the cases robots need to understand the present state of mind for better decision making. It is difficult for machine learning techniques to recognize the expressions of people since there will be significant changes in the way of their expressions. The emotions expressed through the human face have its importance in making arguments and decisions on different subjects. Machine Learning with Computer Vision and Deep Learning can be used to recognize facial expressions from the preloaded or real time images with human faces. DNN (Deep Neural Networking) is one among the hottest areas of research and is found to be very effective in classification of images with a high degree of accuracy. In the proposed work, the popular dataset CK+ is analysed for comparison. The dataset FER 2013 and home-brewed data sets are used in the work for calculating the accuracy of the model created. The results are obtained in such a way that DCNN approach is very efficient in facial emotion recognition. Experiments and study show that the dataset, FER 2013 is a high-quality dataset with equal efficiency as the other two popular datasets. This paper aims to ameliorate the accuracy of classification of facial emotion.

International Journal of Engineering and Advanced Technology, 2019

In this paper we are proposing a compact CNN model for facial expression recognition. Expression recognition on the low quality images are much more challenging and interesting due to the presence of low-intensity expressions. These low intensity expressions are difficult to distinguish with insufficient image resolution. Data collection for FER is expensive and time-consuming. Researches indicates the fact that downloaded images from the Internet is very useful to model and train expression recognition problem. We use extra datasets to improve the training of facial expression recognition, each representing specific data source. Moreover, to prevent subjective annotation, each dataset is labeled with different approaches to ensure annotation qualities. Recognizing the precise and exact expression from a variety of expressions of different people is a huge problem. To solve this problem, we proposed an Emotion Detection Model to extract emotions from the given input image. This work mainly focuses on the psychological approach of color circle-emotion relation[1] to find the accurate emotion from the input image. Initially the whole image is preprocessed and pixel by pixel data is studied. And the combinations of the circles based on combined data will result in a new color. This resulted color will be directly correlated to a particular emotion. Based on the psychological aspects the output will be of reasonable accuracy. The major application of our work is to predict a person's emotion based on his face images or video frames This can even be applied for evaluating the public opinion relating to a particular movie, form the video reaction posts on social Medias. One of the diverse applications of our system is to understand the students learning from their emotions. Human beings shows their emotional states and intentions through facial expressions.. Facial expressions are powerful and natural methods that emphasize the emotional status of humans .The approach used in this work successfully exploits temporal information and it improves the accuracies on the public benchmarking databases. The basic facial expressions are happiness, fear, anger, disgust sadness, and surprise[2]. Contempt was subsequently added as one of the basic emotions. Having sufficient well labeled training data with variations of the populations and environments is important for the design of a deep expression recognition system .Behaviors, poses, facial expressions, actions and speech are considered as channels, which convey human emotions. Lot of research works are going on in this field to explore the correlation between the above mentioned channels and emotions. This paper highlights on the development of a system which automatically recognizes the

Facial Expression evaluation has become necessary for human machine interaction, behavior analysis and also forensic and clinical evaluation. Deep convolutional neural networks (CNN) have been largely used for facial expression recognition but due to locality of convolution, CNNs results in lower accuracy when trained with facial expression data of varying ethnicity and emotion intensity. Recurrent neural networks (RNN) are used to work with sequential data and used to predict the sequences. We propose CNN-RNN network approach, a hybrid network, wherein the outputs from CNN and RNN have been concatenated to predict the final emotion, similarly a CNN model followed by a RNN layers has been designed that gives promising results. The proposed hybrid models are evaluated on two publically available datasets CK+ and JAFFE which provide us with variation in ethnicity and emotion intensity. Promising results have been obtained with this hybrid approach when compared to various machine learning and deep learning methods.

International Journal of Information Technology and Computer Science, 2015

Facial Expression is a key component in evaluating a person's feelings, intentions and characteristics. Facial Expression is an important part of human-computer interaction and has the potential to play an equal important role in humancomputer interaction. The aim of this paper is bring together two areas in which are Artificial Neural Network (ANN) and K-Nearest Neighbor (K-NN) applying for facial expression classification. We propose the ANN_KNN model using ANN and K-NN classifier. ICA is used to extract facial features. The ratios feature is the input of K-NN classifier. We apply ANN_KNN model for seven basic facial expression classifications (anger, fear, surprise, sad, happy, disgust and neutral) on JAFEE database. The classifying precision 92.38% has been showed the feasibility of our proposal model.