Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Materials and Conventional Neural Network
Dataset
Results and Discussion
Conclusion
Acknowledgement
References

Thermal Face Authentication with Convolutional Neural Network

Profile image of Mohamed SayedMohamed SayedProfile image of Faris BakerFaris Baker

2018, Journal of Computer Science

https://doi.org/10.3844/JCSSP.2018.1627.1637
visibility

…

description

11 pages

link

1 file

Abstract

Matching thermal face images as a method of biometric authentication has gained increasing interest because of its advantage of tracking a target object at night and in total darkness. Therefore, for security purposes, it has become highly favourable and has extensive applications, for instance, in video surveillance at night. The aim of this study is to present a simple and efficient deep learning model, which accurately predicts person identification. A pre-trained Convolutional Neural Network (CNN) is employed to extract the features of the multiple convolution layers of the low resolutions' thermal infrared images. To run the program and evaluate the performance, we use a sample of 1500 resized thermal images, each with resolution 181×161 pixels. The sample comprises of images that were captured within different time-lapse and with diverse emotions, poses and lighting conditions. The proposed approach is effective compared to the state-of-the-art thermal face recognition algorithms and achieves impressive accuracy of 99.6% with less processing and training times.

Related papers

Implementation challenge and analysis of thermal image degradation on R-CNN face detection
Ilija Popadic

Telfor Journal, 2020

Face detection systems with color cameras were rapidly evolving and have been well researched. In environments with good visibility they can reach excellent accuracy. But changes in illumination conditions can result in performance degradation, which is the one of the major limitations in visible light face detection systems. The solution to this problem could be in using thermal infrared cameras, since their operation doesn't depend on illumination. Recent studies have shown that deep learning methods can achieve an impressive performance on object detection tasks, and face detection in particular. The goal of this paper is to find an effective way to take advantages from thermal infrared spectra and provide an analysis of various image degradation influence on thermal face detection performance in a system based on R-CNN with special accent on implementation on a hardware platform for video signal processing that institute Vlatacom has developed, called vVSP.

View PDFchevron_right
Thermal Infrared Face Recognition
Vincent Weidlich

Cureus

The technology for deep learning in the field of thermal infrared face recognition has recently become more available for use in research, therefore allowing for the many groups working on this subject to achieve many novel findings. Thermal infrared face recognition helps recognize faces that are not able to be recognized in visible light and can additionally recognize facial blood vessel structure. Previous research regarding temperature variations, mathematical formulas, wave types, and methods in thermal infrared face recognition is reviewed.

View PDFchevron_right
Facial recognition and body temperature measurements based on thermal images using a deep-learning algorithm
Muhammad Reza Ramadhan

IAES International Journal of Artificial Intelligence (IJ-AI)

Recognizing the early symptoms of the SARS-CoV-2 virus (COVID-19) is essential for minimizing its spread. One of the typical symptoms of a person infected with COVID-19 is increased body temperature beyond the normal range. Facial recognition can be used to separate healthy people from those with high body temperatures based on thermal images of the faces. In this study, the XEAST XE-27 thermal imager modes 2, 3, and 4 comprising 1500 thermal images each were compared. The facial recognition was performed using a convolutional neural network. Additionally, body temperatures were extracted from thermal images using matrix laboratory (MATLAB) by considering the minimum and maximum temperatures of each mode and class. The network training results indicate that the accuracies achieved by the proposed facial recognition system in modes 2, 3, and 4 are 87.33%, 92.33%, and 91.66%, respectively. Furthermore, the accuracies of body temperature extraction in modes 2, 3, and 4 are 70%, 60%, an...

View PDFchevron_right
Thermal to Visible Face Recognition Using Deep Autoencoders
Alperen KANTARCI

2019 International Conference of the Biometrics Special Interest Group (BIOSIG), 2019

Visible face recognition systems achieve nearly perfect recognition accuracies using deep learning. However, in lack of light, these systems perform poorly. A way to deal with this problem is thermal to visible cross-domain face matching. This is a desired technology because of its usefulness in night time surveillance. Nevertheless, due to differences between two domains, it is a very challenging face recognition problem. In this paper, we present a deep autoencoder based system to learn the mapping between visible and thermal face images. Also, we assess the impact of alignment in thermal to visible face recognition. For this purpose, we manually annotate the facial landmarks on the Carl and EURECOM datasets. The proposed approach is extensively tested on three publicly available datasets: Carl, UND-X1, and EURECOM. Experimental results show that the proposed approach improves the state-of-the-art significantly. We observe that alignment increases the performance by around 2%. Ann...

View PDFchevron_right
Detection of 3D face masks with thermal infrared imaging and deep learning techniques
Krzysztof Mierzejewski

Photonics Letters of Poland, 2021

Biometric systems are becoming more and more efficient due to increasing performance of algorithms. These systems are also vulnerable to various attacks. Presentation of falsified identity to a biometric sensor is one the most urgent challenges for the recent biometric recognition systems. Exploration of specific properties of thermal infrared seems to be a comprehensive solution for detecting face presentation attacks. This letter presents outcome of our study on detecting 3D face masks using thermal infrared imaging and deep learning techniques. We demonstrate results of a two-step neural network-featured method for detecting presentation attacks. Full Text: PDF ReferencesS.R. Arashloo, J. Kittler, W. Christmas, "Face Spoofing Detection Based on Multiple Descriptor Fusion Using Multiscale Dynamic Binarized Statistical Image Features", IEEE Trans. Inf. Forensics Secur. 10, 11 (2015). CrossRef A. Anjos, M.M. Chakka, S. Marcel, "Motion-based counter-measures to photo a...

View PDFchevron_right
Thermal Face Verification through Identification
Artur Grudzień

Sensors (Basel, Switzerland), 2021

This paper reports on a new approach to face verification in long-wavelength infrared radiation. Two face images were combined into one double image, which was then used as an input for a classification based on neural networks. For testing, we exploited two external and one homemade thermal face databases acquired in various variants. The method is reported to achieve a true acceptance rate of about 83%. We proved that the proposed method outperforms other studied baseline methods by about 20 percentage points. We also analyzed the issue of extending the performance of algorithms. We believe that the proposed double image method can also be applied to other spectral ranges and modalities different than the face.

View PDFchevron_right
Illumination-invariant Face recognition by fusing thermal and visual images via gradient transfer
Sumit Agarwal

Face recognition in real life situations like low illumination condition is still an open challenge in biometric security. It is well established that the state-of-the-art methods in face recognition provide low accuracy in the case of poor illumination. In this work, we propose an algorithm for a more robust illumination invariant face recognition using a multi-modal approach. We propose a new dataset consisting of aligned faces of thermal and visual images of a hundred subjects. We then apply face detection on thermal images using the biggest blob extraction method and apply them for fusing images of different modalities for the purpose of face recognition. An algorithm is proposed to implement fusion of thermal and visual images. We reason for why relying on only one modality can give erroneous results. We use a lighter and faster CNN model called MobileNet for the purpose of face recognition with faster inferencing and to be able to be use it in real time biometric systems. We test our proposed method on our own created dataset to show that real-time face recognition on fused images shows far better results than using visual or thermal images separately.

View PDFchevron_right
High-resolution thermal face dataset for face and expression recognition
Artur Grudzień

2018

Development of facial recognition or expression recognition algorithms requires input data to thoroughly test the performance of algorithms in various conditions. Researchers are developing various methods to face challenges like illumination, pose and expression changes, as well as facial disguises. In this paper, we propose and establish a dataset of thermal facial images, which contains a set of neutral images in various poses as well as a set of facial images with different posed expressions collected with a thermal infrared camera. Since the properties of face in the thermal domain strongly depend on time, in order to show the impact of aging, collection of the dataset has been repeated and a corresponding set of data is provided. The paper describes the measurement methodology and database structure. We present baseline results of processing using state-of-the-art facial descriptors combined with distance metrics for thermal face reidentification. Three selected local descript...

View PDFchevron_right
A deep learning model for classifying human facial expressions from infrared thermal images
Dmitrii I Kaplun

Scientific Reports

The analysis of human facial expressions from the thermal images captured by the Infrared Thermal Imaging (IRTI) cameras has recently gained importance compared to images captured by the standard cameras using light having a wavelength in the visible spectrum. It is because infrared cameras work well in low-light conditions and also infrared spectrum captures thermal distribution that is very useful for building systems like Robot interaction systems, quantifying the cognitive responses from facial expressions, disease control, etc. In this paper, a deep learning model called IRFacExNet (InfraRed Facial Expression Network) has been proposed for facial expression recognition (FER) from infrared images. It utilizes two building blocks namely Residual unit and Transformation unit which extract dominant features from the input images specific to the expressions. The extracted features help to detect the emotion of the subjects in consideration accurately. The Snapshot ensemble technique...

View PDFchevron_right
Facial Expressions Recognition in Thermal Images based on Deep Learning Techniques
Neveen Ghali

International Journal of Image, Graphics and Signal Processing, 2019

Facial expressions are undoubtedly the best way to express human attitude which is crucial in social communications. This paper gives attention for exploring the human sentimental state in thermal images through Facial Expression Recognition (FER) by utilizing Convolutional Neural Network (CNN). Most traditional approaches largely depend on feature extraction and classification methods with a big pre-processing level but CNN as a type of deep learning methods, can automatically learn and distinguish influential features from the raw data of images through its own multiple layers. Obtained experimental results over the IRIS database show that the use of CNN architecture has a 96.7% recognition rate which is high compared with Neural Networks (NN), Autoencoder (AE) and other traditional recognition methods as Local Standard Deviation (LSD), Principle Component Analysis (PCA) and K-Nearest Neighbor (KNN).

View PDFchevron_right

References (36)

  1. Abadi, M., P. Barham, J. Chen, Z. Chen and A. Davis et al., 2016. TensorFlow: A system for large-scale machine learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, (SDI' 16), Google Brain.
  2. Chollet, F., 2015. Keras. https://github.com/fchollet/keras
  3. Chollet, F., (n.d.). Keras: Deep learning library for theano and tensorflow. https://keras.io/ CNN, (n.d.). Convolutional Neural Network. https://en.wikipedia.org/wiki/Convolutional_neural_ network
  4. Goodfellow, I., Y. Bengio and A. Courville, 2016. Deep Learning. 1st Edn., MIT Press, Cambridge, ISBN-10: 0262337371, pp: 800.
  5. Hadji, I. and R. Wildes, 2018. What do we understand about convolutional networks? arXiv preprint arXiv:1803.08834.
  6. He, K., X. Zhang, S. Ren and J. Sun, 2016. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 27-30, IEEE Xplore Press, Las Vegas, NV, USA, pp: 770-778. DOI: 10.1109/CVPR.2016.90
  7. Hinton, G., N. Srivastava, A. Krizh, I. Sutskever and R. Salakhutdinov, 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580.
  8. Hoo-Chang, S., H. Roth, M. Gao, L. Lu and Z. Xu et al., 2016. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging, 35: 1285-1298. DOI: 10.1109/TMI.2016.2528162
  9. Huang, G., Z. Liu, L. Van Der Maaten and K. Weinberger, 2017. Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jul. 21-26, IEEE Xplore Press, Honolulu, HI, USA, pp: 2261-2269. DOI: 10.1109/CVPR.2017.243
  10. Kandpal, A., 2017. Medium.com: https://codeburst.io/machine-learning-day-1- 60bd231d0660
  11. Karpathy, A., G. Toderici, S. Shetty, T. Leung and S. Rahul et al., 2014. Large-scale video classification with convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 23-28, IEEE Xplore Press, Columbus, OH, USA, pp: 1725-1732. DOI: 10.1109/CVPR.2014.223
  12. Khanal, S., J. Fulton and S. Shearer, 2017. An overview of current and potential applications of thermal remote sensing in precision agriculture. Comput. Electron. Agric., 139: 22-32. DOI: 10.1016/j.compag.2017.05.001
  13. Kramer, O., 2016. Scikit-learn. Mach. Learn. Evolut. Strategies, 20: 45-53. DOI: 10.1007/978-3-319-33383-0_5
  14. Krizhevsky, A., I. Sutskever and G.E. Hinton, 2012. Imagenet classification with deep convolutional neural networks. Adv. Neural Inform. Process. Syst., 25: 1097-1105. DOI: 10.1145/3065386
  15. Krizhevsky, A., I. Sutskever and G.E. Hinton, 2017. ImageNet classification with deep convolutional neural networks. Commun. ACM, 60: 84-90. DOI: 10.1145/3065386
  16. Lahiri, B., S. Bagavathiappan, T. Jayakumar and J. Philip, 2012. Medical applications of infrared thermography: A review. Infrared Phys. Technol., 55: 221-235. DOI: 10.1016/j.infrared.2012.03.007
  17. LeCun, Y., L. Bottou, Y. Bengio and P. Haffner, 1998. Gradient-based learning applied to document recognition. Proc. IEEE, 86: 2278-2324. DOI: 10.1109/5.726791
  18. LeCun, Y., K. Kavukcuoglu and C. Farabet, 2010. Convolutional networks and applications in vision. Proceedings of IEEE International Symposium on Circuits and Systems, May 30-Jun. 2, IEEE Xplore Press, Paris, France, pp: 253-256. DOI: 10.1109/ISCAS.2010.5537907
  19. MathWorks, 2015. Image processing toolbox. MathWorks. https://www.mathworks.com
  20. Ng, H.W., V. Dung Nguyen, V. Vonikakis and S. Winkler, 2015. Deep learning for emotion recognition on small datasets using transfer learning. Proceedings of the ACM on International Conference on Multimodal Interaction, Nov. 09-13, ACM, Seattle, Washington, USA, pp: 443-449. DOI: 10.1145/2818346.2830593
  21. Nixon, M.S. and A.S. Aguado, 2002. Feature Extraction and Image Processing. 1st Edn., Newnes, Oxford, ISBN-10: 0750650788, pp: 350.
  22. Oquab, M., L. Bottou, I. Laptev and J. Sivic, 2014. Learning and transferring mid-level image representations using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 23-28, IEEE Xplore Press, Columbus, OH, USA., pp: 1717-1724. DOI: 10.1109/CVPR.2014.222
  23. Pedregosa, F., G. Varoquaux, A. Gramf, V. Michel and B. Thirion et al., 2011. Scikit-learn: Machine learning in python. J. Mach. Learn. Res., 12: 2825-2830.
  24. Peng, M., C. Wang, T. Chen and G. Liu, 2016. NIRFaceNet: A convolutional neural network for near-infrared face identification. Information, 61: 1-14. DOI: 10.3390/info7040061
  25. Sayed, M., 2018a. Biometric gait recognition based on machine learning. J. Comput. Sci., 14: 1064-1073. DOI: 10.3844/jcssp.2018.1064.1073
  26. Sayed, M., 2018b. Performance of convolutional neural networks for human identification by gait recognition. J. Artif. Intell., 11: 30-38. DOI: 10.3923/jai.2018.30.38
  27. Simonyan, K. and A. Zisserman, 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.
  28. Smith, S. and Q. Le, 2018. A Bayesian perspective on generalization and stochastic gradient descent. ICLR.
  29. Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov, 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Machine Learn. Res., 15: 1929-1958.
  30. Szegedy, C., S. Ioffe, V. Vanhoucke and A. Alemi, 2017. Inception-v4, inception-ResNet and the impact of residual connections on learning. Proceedings of the 31th AAAI Conference on Artificial Intelligence, (CAI' 17), pp: 12-12.
  31. Szegedy, C., W. Liu, Y. Jia, P. Sermanet and S. Reed et al., 2015. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 7-12, IEEE Xplore Press, Boston, MA, USA, pp: 1-9. DOI: 10.1109/CVPR.2015.7298594
  32. Tajbakhsh, N., J. Shin, S. Gurudu, R. Todd Hurst and C. Kendall et al., 2017. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE Trans. Med. Imag., 35: 1299-1312. DOI: 10.1109/TMI.2016.2535302
  33. Vadivambal, R. and D. Jayas, 2011. Applications of thermal imaging in agriculture and food industry-a review. Food Bioprocess Technol., 4: 186-199. DOI: 10.1007/s11947-010-0333-5
  34. Wang, M. and W. Deng, 2018. Deep visual domain adaptation: A survey. Neurocomputing, 312: 135-153. DOI: 10.1016/j.neucom.2018.05.083
  35. Wu, Z., M. Peng and T. Chen, 2016. Thermal face recognition using convolutional neural network. Proceedings of the International Conference on Optoelectronics and Image Processing, Jun. 10-12, IEEE Xplore Press, Warsaw, Poland, pp: 6-9. DOI: 10.1109/OPTIP.2016.7528489
  36. Zaeri, N., F. Baker and R. Dip, 2015. Thermal face recognition using moments invariants. Int. J. Signal Process. Syst., 3: 94-99. DOI: 10.12720/ijsps.3.2.94-99

Related papers

Real-Time Thermal Face Identification System for Low Memory Vision Applications Using CNN
Alessandro Ortis

Pattern Recognition. ICPR International Workshops and Challenges

Image based face identification systems have attained optimal performance. However, the design of such systems often involves some issues related to extreme light conditions and privacy protection, among others. Since several years, Face Identification (FI) based on thermal images using deep neural networks (DNN) has received significant attention. Yet, the majority of the FI systems developed through DNN's need huge computational power; those systems are not suitable for the devices with memory limitations. In this paper, we proposed a new CNN framework based on depthwise separable convolutions for real-time face identification for low memory vision applications. The lack of publicly available thermal datasets makes very hard the research and developing of new techniques. In this work, we further present a new large-scale thermal face database called "ST UNICT Thermal Face". As per our analysis, the evaluation of the learnt model using the data obtained in the single-day (without temporal variations), it might not stable over time. One of the main reasons behind the development of this database for the real-time evaluation of the proposed model depends on the fact that most thermal face identification systems are not stable over time and climate due to insufficient time data. The evaluation results exhibit that the proposed framework is suitable for the devices having limited memory and which is stable over time and different indoor environmental conditions.

View PDFchevron_right
An efficient deep learning model for classification of thermal face images
Ahmed Sedik

Journal of Enterprise Information Management, 2020

Purpose The objective of this paper is to perform infrared (IR) face recognition efficiently with convolutional neural networks (CNNs). The proposed model in this paper has several advantages such as the automatic feature extraction using convolutional and pooling layers and the ability to distinguish between faces without visual details. Design/methodology/approach A model which comprises five convolutional layers in addition to five max-pooling layers is introduced for the recognition of IR faces. Findings The experimental results and analysis reveal high recognition rates of IR faces with the proposed model. Originality/value A designed CNN model is presented for IR face recognition. Both the feature extraction and classification tasks are incorporated into this model. The problems of low contrast and absence of details in IR images are overcome with the proposed model. The recognition accuracy reaches 100% in experiments on the Terravic Facial IR Database (TFIRDB).

View PDFchevron_right
An overview of thermal face recognition methods
Marina Ivasic-Kos

2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 2018

The popularity of surveillance systems grows as well as a need for better security systems particularly in a bad lighting conditions or at night. The aim of a security system is to collect as many details as possible to enable a better recognition of persons. In this paper, a comparison of representative thermal face recognition methods will be given, emphasizing their strengths and weaknesses. Then, trends in the development of surveillance and security systems will be outlined such as fusion of visible and thermal images and use of convolutional neural networks. Also, existing challenges of thermal facial recognition and its applications in a real world will be pointed out.

View PDFchevron_right
Thermal–Visible Face Recognition Based on CNN Features and Triple Triplet Configuration for On-the-Move Identity Verification
Krzysztof Mierzejewski

Sensors, 2022

Face recognition operating in visible domains exists in many aspects of our lifes, while the remaining parts of the spectrum including near and thermal infrared are not sufficiently explored. Thermal–visible face recognition is a promising biometric modality that combines affordable technology and high imaging qualities in the visible domain with low-light capabilities of thermal infrared. In this work, we present the results of our study in the field of thermal–visible face verification using four different algorithm architectures tested using several publicly available databases. The study covers Siamese, Triplet, and Verification Through Identification methods in various configurations. As a result, we propose a triple triplet face verification method that combines three CNNs being used in each of the triplet branches. The triple triplet method outperforms other reference methods and achieves TAR @FAR 1% values up to 90.61%.

View PDFchevron_right
Thermal Image Generation for Robust Face Recognition
vicente pavez

Applied Sciences, 2022

This article shows how to create a robust thermal face recognition system based on the FaceNet architecture. We propose a method for generating thermal images to create a thermal face database with six different attributes (frown, glasses, rotation, normal, vocal, and smile) based on various deep learning models. First, we use StyleCLIP, which oversees manipulating the latent space of the input visible image to add the desired attributes to the visible face. Second, we use the GANs N’ Roses (GNR) model, a multimodal image-to-image framework. It uses maps of style and content to generate thermal imaging from visible images, using generative adversarial approaches. Using the proposed generator system, we create a database of synthetic thermal faces composed of more than 100k images corresponding to 3227 individuals. When trained and tested using the synthetic database, the Thermal-FaceNet model obtained a 99.98% accuracy. Furthermore, when tested with a real database, the accuracy was...

View PDFchevron_right

Cited by

Real-Time Thermal Face Identification System for Low Memory Vision Applications Using CNN
Alessandro Ortis

Pattern Recognition. ICPR International Workshops and Challenges

Image based face identification systems have attained optimal performance. However, the design of such systems often involves some issues related to extreme light conditions and privacy protection, among others. Since several years, Face Identification (FI) based on thermal images using deep neural networks (DNN) has received significant attention. Yet, the majority of the FI systems developed through DNN's need huge computational power; those systems are not suitable for the devices with memory limitations. In this paper, we proposed a new CNN framework based on depthwise separable convolutions for real-time face identification for low memory vision applications. The lack of publicly available thermal datasets makes very hard the research and developing of new techniques. In this work, we further present a new large-scale thermal face database called "ST UNICT Thermal Face". As per our analysis, the evaluation of the learnt model using the data obtained in the single-day (without temporal variations), it might not stable over time. One of the main reasons behind the development of this database for the real-time evaluation of the proposed model depends on the fact that most thermal face identification systems are not stable over time and climate due to insufficient time data. The evaluation results exhibit that the proposed framework is suitable for the devices having limited memory and which is stable over time and different indoor environmental conditions.

View PDFchevron_right
Infrared Thermal Image Gender Classifier Based on the Deep ResNet Model
alyaa jalil

Advances in Human-Computer Interaction

Gender classification from human face images has attracted researchers over the past decade. It has great impact in different fields including defense, human-computer interaction, surveillance industry, and mobile applications. Many methods and techniques have been proposed depending on clear digital images and complex feature extraction preprocessing. However, most recent critical real systems use thermal cameras. This paper has the novelty of utilizing thermal images in gender classification. It proposes a unique approach called IRT_ResNet that adopts residual network (ResNet) model with different layer configurations: 18, 50, and 101. Two different datasets of thermal images have been leveraged to train and test these models. The proposed approach has been compared with convolutional neural network (CNN), principal component analysis (PCA), local binary pattern (LBP), and scale invariant feature transform (SIFT). The experimental results show that the proposed model has higher ov...

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved