Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Proposed Methods
Experiments A. Datasets
Results on the WFLW Dataset
Results on the JD-Landmark Dataset
Conclusions
References
All Topics
Computer Science
Artificial Intelligence

LOTR: Face Landmark Localization Using Localization Transformer

Profile image of Sanjana Prakash JainSanjana Prakash Jain

2022, IEEE Access

https://doi.org/10.1109/ACCESS.2022.3149380
visibility

…

description

15 pages

link

1 file

Abstract

This paper presents a novel Transformer-based facial landmark localization network named Localization Transformer (LOTR). The proposed framework is a direct coordinate regression approach leveraging a Transformer network to better utilize the spatial information in a feature map. An LOTR model consists of three main modules: 1) a visual backbone that converts an input image into a feature map, 2) a Transformer module that improves the feature representation from the visual backbone, and 3) a landmark prediction head that directly predicts coordinates from the Transformer's representation. Given cropped-and-aligned face images, the proposed LOTR can be trained end-to-end without requiring any postprocessing steps. This paper also introduces a loss function named smooth-Wing loss, which addresses the gradient discontinuity of the Wing loss, leading to better convergence than standard loss functions such as L1, L2, and Wing loss. Experimental results on the JD landmark dataset provided by the First Grand Challenge of 106-Point Facial Landmark Localization indicate the superiority of LOTR over the existing methods on the leaderboard and two recent heatmap-based approaches. On the WFLW dataset, the proposed LOTR framework demonstrates promising results compared with several state-of-the-art methods. Additionally, we report an improvement in the performance of state-of-the-art face recognition systems when using our proposed LOTRs for face alignment.

Related papers

Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression
Fuxin Li

arXiv (Cornell University), 2019

Heatmap regression with a deep network has become one of the mainstream approaches to localize facial landmarks. However, the loss function for heatmap regression is rarely studied. In this paper, we analyze the ideal loss function properties for heatmap regression in face alignment problems. Then we propose a novel loss function, named Adaptive Wing loss, that is able to adapt its shape to different types of ground truth heatmap pixels. This adaptability penalizes loss more on foreground pixels while less on background pixels. To address the imbalance between foreground and background pixels, we also propose Weighted Loss Map, which assigns high weights on foreground and difficult background pixels to help training process focus more on pixels that are crucial to landmark localization. To further improve face alignment accuracy, we introduce boundary prediction and CoordConv with boundary coordinates. Extensive experiments on different benchmarks, including COFW, 300W and WFLW, show our approach outperforms the state-of-the-art by a significant margin on various evaluation metrics. Besides, the Adaptive Wing loss also helps other heatmap regression tasks. Code will be made publicly available at https://github.com/ protossw512/AdaptiveWingLoss.

View PDFchevron_right
A Detailed Look At CNN-based Approaches In Facial Landmark Detection
Chin-Laung Lei

arXiv (Cornell University), 2020

Facial landmark detection has been studied over decades. Numerous neural network (NN)-based approaches have been proposed for detecting landmarks, especially the convolutional neural network (CNN)-based approaches. In general, CNN-based approaches can be divided into regression and heatmap approaches. However, no research systematically studies the characteristics of different approaches. In this paper, we investigate both CNN-based approaches, generalize their advantages and disadvantages, and introduce a variation of the heatmap approach, a pixel-wise classification (PWC) model. To the best of our knowledge, using the PWC model to detect facial landmarks have not been comprehensively studied. We further design a hybrid loss function and a discrimination network for strengthening the landmarks' interrelationship implied in the PWC model to improve the detection accuracy without modifying the original model architecture. Six common facial landmark datasets, AFW, Helen, LFPW, 300-W, IBUG, and COFW are adopted to train or evaluate our model. A comprehensive evaluation is conducted and the result shows that the proposed model outperforms other models in all tested datasets.

View PDFchevron_right
Sub-pixel face landmarks using heatmaps and a bag of tricks
Pavit Noinongyao

ArXiv, 2021

Accurate face landmark localization is an essential part of face recognition, reconstruction and morphing. To accurately localize face landmarks, we present our heatmap regression approach. Each model consists of a MobileNetV2 backbone followed by several upscaling layers, with different tricks to optimize both performance and inference cost. We use five naïve face landmarks from a publicly available face detector to position and align the face instead of using the bounding box like traditional methods. Moreover, we show by adding random rotation, displacement and scaling—after alignment—that the model is more sensitive to the face position than orientation. We also show that it is possible to reduce the upscaling complexity by using a mixture of deconvolution and pixel-shuffle layers without impeding localization performance. We present our state-of-the-art face landmark localization model (ranking second on The 2nd Grand Challenge of 106-Point Facial Landmark Localization validati...

View PDFchevron_right
Facial point localization via neural networks in a cascade regression framework
Anwar Saeed

Multimedia Tools and Applications, 2017

Facial point detection gains an increasing importance in computer vision as it plays a vital role in several applications such as facial expression recognition and human behavior analysis. In this work, we propose an approach to locate 49 facial points via neural networks in a cascade regression fashion. The localization process starts by detecting the face, followed by a face cropping refinement task and lastly arriving at the facial point location through five cascades of regressors. In particular, we perform a guided initialization using holistic features extracted from the entire face patch. Then, the points location is refined in the next four cascades using local features extracted from patches enclosing the prior estimates of the points. The generalization capability was improved by performing feature selection at each cascade. By evaluating our approach on samples gathered from four challenging databases, we achieved a location average error for each point ranging between 0.72 % and 1.57 % of the face width. The proposed approach was further evaluated according to the 300-w challenge, where we achieved competitive results to those obtained by state-of-the-art approaches and commercial software packages. Moreover, our approach showed better generalization capability. Finally, we validated the proposed enhancements by studying the impact of several factors on the point localization accuracy.

View PDFchevron_right
68 landmarks are efficient for 3D face alignment: what about more?
ALI WALI

Multimedia Tools and Applications

This paper proposes a 3D face alignment of 2D face images in the wild with noisy landmarks. The objective is to recognize individuals from their single profile image. We first proceed by extracting more than 68 landmarks using a bag of features. This allows us to obtain a bag of visible and invisible facial keypoints. Then, we reconstruct a 3D face model and get a triangular mesh by meshing the obtained keypoints. For each face, the number of keypoints is not the same, which makes this step very challenging. Later, we process the 3D face using butterfly and BPA algorithms to make correlation and regularity between 3D face regions. Indeed, 2D-to-3D annotations give much higher quality to the 3D reconstructed face model without the need for any additional 3D Morphable models. Finally, we carry out alignment and pose correction steps to get frontal pose by fitting the rendered 3D reconstructed face to 2D face and performing pose normalization to achieve good rates in face recognition. The recognition step is based on deep learning and it is performed using DCNNs, which are very powerful and modern, for feature learning and face identification. To verify the proposed method, three popular benchmarks, YTF, LFW, and BIWI databases, are tested. Compared to the best recognition results reported on these benchmarks, our proposed method achieves comparable or even better recognition performances.

View PDFchevron_right
A lightweight 3D dense facial landmark estimation model from position map data
Michael Schukat

Zenodo (CERN European Organization for Nuclear Research), 2023

The incorporation of 3D data in facial analysis tasks has gained popularity in recent years. Though it provides a more accurate and detailed representation of the human face, accruing 3D face data is more complex and expensive than 2D face images. Either one has to rely on expensive 3D scanners or depth sensors which are prone to noise. An alternative option is the reconstruction of 3D faces from uncalibrated 2D images in an unsupervised way without any ground truth 3D data. However, such approaches are computationally expensive and the learned model size is not suitable for mobile or other edge device applications. Predicting dense 3D landmarks over the whole face can overcome this issue. As there is no public dataset available containing dense landmarks, we propose a pipeline to create a dense keypoint training dataset containing 520 key points across the whole face from an existing facial position map data. We train a lightweight MobileNet-based regressor model with the generated data. As we do not have access to any evaluation dataset with dense landmarks in it we evaluate our model against the 68 keypoint detection task. Experimental results show that our trained model outperforms many of the existing methods in spite of its lower model size and minimal computational cost. Also, the qualitative evaluation shows the efficiency of our trained models in extreme head pose angles as well as other facial variations and occlusions. Code is

View PDFchevron_right
Head Pose Estimation in the Wild Assisted by Facial Landmarks Based on Convolutional Neural Networks
Jiacai Liao

IEEE Access, 2019

Convolutional neural networks (CNNs) exhibit excellent performance on the head pose estimation problem under controllable conditions, but their generalization ability in the wild needs to be improved. To address this issue, we propose an approach involving the introduction of facial landmark information into the task simplifier and landmark heatmap generator constructed before the feed-forward neural network, which can use this information to normalize the face shape into a canonical shape and generate a landmark heatmap based on the transformed facial landmarks to assist in feature extraction, for enhancing generalization ability in the wild. Our method was trained on 300W-LP and tested on AFLW2000-3D. The result shows that for the same feed-forward neural network, when our method is used to introduce facial landmark information into a CNN, accuracy improves from 88.5% to 99.0% and mean average error decreases from 5.94 • to 1.46 • on AFLW2000-3D. Furthermore, we evaluate our method on several datasets used for pose estimation and compare the result with AFLW2000-3D, finding that the features extracted by a CNN could not reflect the head pose efficiently, which limits the performance of the CNN on the head pose estimation problem in wild. By introducing facial landmarks, the CNN could extract features that reflect head pose more efficiently, thereby significantly improving the accuracy of head pose estimation in the wild. INDEX TERMS head pose estimation, convolutional neural network, landmark heatmap, faceshape normalization

View PDFchevron_right
Lightweight and Effective Facial Landmark Detection using Adversarial Learning with Face Geometric Map Generative Network
Kyung Hee University

Facial landmark detection plays an important role in face analysis tasks. Moreover, it is used as a prerequisite in many facial related applications, the simplicity, as well as effectiveness, is essential in the facial landmark detection. In this paper, we propose an effective facial landmark detection network and an associated learning framework with the geometric prior-generative adversarial network. The geometric prior-generative adversarial network consists of one generator and two discriminators. The generator consists of an encoder and two decoders. The encoder predicts facial landmark points. The decoders generate a facial inner and contour geometric map from predicted landmark points. Generating face geometric maps from predicted landmark points helps the predicted landmark points to represent the face geometric information, including shape and configuration. The discriminators determine that the given geometric maps are generated from actual landmark points or estimated landmark points. Our proposed network is end-to-end trainable, and only the encoder part is used simply as the facial landmark detector in the testing stage. To verify the effectiveness of the proposed method, we have conducted comprehensive experiments with benchmark data sets. The results have shown that the proposed method achieves comparable performances over recently proposed facial landmark detection methods with a simple and effective facial landmark detection network.

View PDFchevron_right
Simultaneous regression and feature learning for facial landmarking
Simon Dobrišek

arXiv (Cornell University), 2019

timal for the alignment tasks and in addition to competitive landmarking results also ensures extremely rapid processing. We evaluate both approaches in rigorous experiments on several popular datasets of 3D face images, i.e., the FRGCv2 and Bosphorus 3D Face datasets and image collections F and G from the University of Notre Dame. The results of our evaluation show that both approaches are competitive in comparison to the state-of-the-art, while exhibiting considerable robustness to pose variations.

View PDFchevron_right
Facial landmark detection by semi-supervised deep learning
Jianbing Shen

Neurocomputing, 2018

In this paper, we propose a semi-supervised facial landmark detection algorithm (SEMI) based on convolutional neural network (CNN), which can detect facial components and landmarks simultaneously. Unlike previous coarse-to-fine algorithms, our model does not need extra input such as initial landmark prediction. It also solves the large area occlusion problem by detecting the visible facial components while existing face detectors failed to detect faces in the picture. Semi-supervised learning algorithm is also an effective data augmentation method. In our experiment, each image has two types of ground truth, one is bounding-box related (classification and coordinates) and the other is landmark coordinates inside the bounding-box. The supervised data have both two types of ground truth while the semi-supervised data only have the bounding-box. Our model was trained by the merge of two parts of data. Extensive evaluations on Helen, LFPW and 300-W show that our algorithm is able to complete the landmark task and performs better than many state-of-art facial landmark detecting algorithms.

View PDFchevron_right

References (75)

  1. S. W. Earp, A. Samacoits, S. Jain, P. Noinongyao, and S. Boonpun- mongkol, "Sub-pixel face landmarks using heatmaps and a bag of tricks," arXiv preprint arXiv:2103.03059, 2021.
  2. N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar, "Attribute and simile classifiers for face verification," in 2009 IEEE 12th International Conference on Computer Vision, 2009, pp. 365-372.
  3. L. Wolf, T. Hassner, and Y. Taigman, "Similarity scores based on back- ground samples," in Computer Vision -ACCV 2009, H. Zha, R.-i. Taniguchi, and S. Maybank, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 88-97.
  4. O. M. Parkhi, A. Vedaldi, and A. Zisserman, "Deep face recognition," in Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, September 2015, pp. 41.1-41.12.
  5. F. Schroff, D. Kalenichenko, and J. Philbin, "Facenet: A unified embed- ding for face recognition and clustering," CoRR, vol. abs/1503.03832, 2015.
  6. W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song, "Sphereface: Deep hypersphere embedding for face recognition," CoRR, vol. abs/1704.08063, 2017.
  7. H. Wang, Y. Wang, Z. Zhou, X. Ji, Z. Li, D. Gong, J. Zhou, and W. Liu, "Cosface: Large margin cosine loss for deep face recognition," CoRR, vol. abs/1801.09414, 2018.
  8. J. Deng, J. Guo, and S. Zafeiriou, "Arcface: Additive angular margin loss for deep face recognition," CoRR, vol. abs/1801.07698, 2018.
  9. X. An, X. Zhu, Y. Xiao, L. Wu, M. Zhang, Y. Gao, B. Qin, D. Zhang, and Y. Fu, "Partial FC: Training 10 million identities on a single machine," arXiv e-prints, p. arXiv:2010.05222, Oct. 2020.
  10. P. Barra, C. Bisogni, M. Nappi, and S. Ricciardi, "Fast quadtree-based pose estimation for security applications using face biometrics," in Network and System Security, M. H. Au, S. M. Yiu, J. Li, X. Luo, C. Wang, A. Castiglione, and K. Kluczniak, Eds. Cham: Springer International Publishing, 2018, pp. 160-173.
  11. V. Kazemi and J. Sullivan, "One millisecond face alignment with an ensemble of regression trees," in 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1867-1874.
  12. C. Cao, Y. Weng, S. Lin, and K. Zhou, "3d shape regression for real-time facial animation," vol. 32, no. 4, Jul 2013.
  13. J. Roth, Y. Tong, and X. Liu, "Unconstrained 3D face reconstruction," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 07-12-June-2015. IEEE Computer Society, Oct 2015, pp. 2606-2615.
  14. P. Dou, S. K. Shah, and I. A. Kakadiaris, "End-to-end 3D face reconstruc- tion with deep neural networks," Proceedings -30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, vol. 2017-January, pp. 1503-1512, Apr 2017.
  15. Y. Feng, F. Wu, X. Shao, Y. Wang, and X. Zhou, "Joint 3D face reconstruc- tion and dense alignment with position map regression Network," Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11218 LNCS, pp. 557-574, Mar 2018.
  16. X. Yang, Y. Li, H. Qi, and S. Lyu, "Exposing gan-synthesized faces using landmark locations," CoRR, vol. abs/1904.00167, 2019.
  17. M. Day, "Exploiting facial landmarks for emotion recognition in the wild," CoRR, vol. abs/1603.09129, 2016.
  18. M. I. N. P. Munasinghe, "Facial expression recognition using facial land- marks and random forest classifier," in 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS), 2018, pp. 423- 427.
  19. S. Hinduja and S. Canavan, "Facial action unit detection using 3D facial landmarks," arXiv e-prints, p. arXiv:2005.08343, May 2020.
  20. H. Dibeklioglu, A. A. Salah, and L. Akarun, "3d facial landmarking under expression, pose, and occlusion variations," in 2008 IEEE Second Interna- tional Conference on Biometrics: Theory, Applications and Systems, 2008, pp. 1-6.
  21. T. Cootes, C. Taylor, D. Cooper, and J. Graham, "Active shape models- their training and application," Computer Vision and Image Understand- ing, vol. 61, no. 1, pp. 38 -59, 1995.
  22. T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active appearance models," in IEEE Transactions on Pattern Analysis and Machine Intelligence. Springer, 1998, pp. 484-498.
  23. L. Liang, R. Xiao, F. Wen, and J. Sun, "Face alignment via component- based discriminative search," 10 2008, pp. 72-85.
  24. X. Zhu and D. Ramanan, "Face detection, pose estimation, and landmark localization in the wild," in 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2879-2886.
  25. B. Amberg and T. Vetter, "Optimal landmark detection using shape models and branch and bound," 11 2011, pp. 455-462.
  26. P. Belhumeur, D. Jacobs, D. Kriegman, and N. Kumar, "Localizing parts of faces using a consensus of exemplars," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, pp. 2930-40, 12 2013.
  27. B. A. Efraty, M. Papadakis, A. Profitt, S. Shah, and I. A. Kakadiaris, "Facial component-landmark detection," in 2011 IEEE International Con- ference on Automatic Face Gesture Recognition (FG), 2011, pp. 278-285.
  28. Y. Sun, X. Wang, and X. Tang, "Deep convolutional network cascade for facial point detection," in 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3476-3483.
  29. E. Zhou, H. Fan, Z. Cao, Y. Jiang, and Q. Yin, "Extensive facial landmark localization with coarse-to-fine convolutional network cascade," in 2013 IEEE International Conference on Computer Vision Workshops, 2013, pp. 386-391.
  30. J. Zhang, S. Shan, M. Kan, and X. Chen, "Coarse-to-fine auto-encoder networks (cfan) for real-time face alignment," in Computer Vision -ECCV 2014, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. Cham: Springer International Publishing, 2014, pp. 1-16.
  31. Z. Zhang, P. Luo, C. C. Loy, and X. Tang, "Learning deep representation for face alignment with auxiliary attributes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 5, pp. 918-930, 2016.
  32. R. Ranjan, V. M. Patel, and R. Chellappa, "Hyperface: A deep multi- task learning framework for face detection, landmark localization, pose estimation, and gender recognition," CoRR, vol. abs/1603.01249, 2016.
  33. M. Kowalski, J. Naruniec, and T. Trzcinski, "Deep alignment network: A convolutional neural network for robust face alignment," CoRR, vol. abs/1706.01789, 2017.
  34. Y. Xiong, Z. Zhou, Y. Dou, and Z. Su, "Gaussian vector: An efficient solution for facial landmark detection," in Computer Vision -ACCV 2020, H. Ishikawa, C.-L. Liu, T. Pajdla, and J. Shi, Eds. Cham: Springer International Publishing, 2021, pp. 70-87.
  35. S. Mahpod, R. Das, E. Maiorana, Y. Keller, and P. Campisi, "Facial landmark point localization using coarse-to-fine deep recurrent neural network," arXiv, vol. abs/1805.01760, 2018.
  36. A. Vaswani, G. Brain, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," Tech. Rep., 2017.
  37. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, "End-to-end object detection with transformers." in ECCV (1), ser. Lecture Notes in Computer Science, A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, Eds., vol. 12346. Springer, 2020, pp. 213- 229.
  38. W. Wu, C. Qian, S. Yang, Q. Wang, Y. Cai, and Q. feng Zhou, "Look at boundary: A boundary-aware face alignment algorithm," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2129-2138, 2018.
  39. Z.-H. Feng, J. Kittler, M. Awais, P. Huber, and X.-J. Wu, "Wing loss for robust facial landmark localisation with convolutional neural networks," Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2235-2245, Nov 2017.
  40. S. Sengupta, J. Chen, C. Castillo, V. M. Patel, R. Chellappa, and D. W. Jacobs, "Frontal to profile face verification in the wild," in 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), March 2016, pp. 1-9.
  41. T. Zheng and W. Deng, "Cross-pose lfw: A database for studying cross- pose face recognition in unconstrained environments," Beijing University of Posts and Telecommunications, Tech. Rep, vol. 5, 2018.
  42. C. Whitelam, E. Taborsky, A. Blanton, B. Maze, J. Adams, T. Miller, N. Kalka, A. K. Jain, J. A. Duncan, K. Allen, J. Cheney, and P. Grother, "Iarpa janus benchmark-b face dataset," in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 592-600.
  43. B. Maze, J. Adams, J. A. Duncan, N. Kalka, T. Miller, C. Otto, A. K. Jain, W. T. Niggel, J. Anderson, J. Cheney, and P. Grother, "Iarpa janus bench- mark -c: Face dataset and protocol," in 2018 International Conference on Biometrics (ICB), Feb 2018, pp. 158-165.
  44. Z. Zhang, P. Luo, C. C. Loy, and X. Tang, "Facial landmark detection by deep multi-task learning," in European Conference on Computer Vision. Springer, 2014, pp. 94-108.
  45. X. Dong, Y. Yan, W. Ouyang, and Y. Yang, "Style aggregated network for facial landmark detection," CoRR, vol. abs/1803.04108, 2018.
  46. K. Sun, Y. Zhao, B. Jiang, T. Cheng, B. Xiao, D. Liu, Y. Mu, X. Wang, W. Liu, and J. Wang, "High-resolution representations for labeling pixels and regions," arXiv, vol. abs/1904.04514, 2019.
  47. A. Kumar, T. K. Marks, W. Mou, Y. Wang, M. Jones, A. Cherian, T. Koike- Akino, X. Liu, and C. Feng, "Luvli face alignment: Estimating landmarks' location, uncertainty, and visibility likelihood," 2020 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pp. 8233- 8243, 2020.
  48. A. Newell, K. Yang, and J. Deng, "Stacked hourglass networks for human pose estimation," Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformat- ics), vol. 9912 LNCS, pp. 483-499, Mar 2016.
  49. S. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, "Convolutional pose machines," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4724-4732.
  50. B. Xiao, H. Wu, and Y. Wei, "Simple baselines for human pose estimation and tracking," arXiv, vol. abs/1804.06208, 2018.
  51. F. Zhang, X. Zhu, H. Dai, M. Ye, and C. Zhu, "Distribution-aware coordinate representation for human pose estimation," arXiv preprint arXiv:1910.06278, 2019.
  52. Y. Liu, H. Shen, Y. Si, X. Wang, X. Zhu, H. Shi, Z. Hong, H. Guo, Z. Guo, Y. Chen, B. Li, T. Xi, J. Yu, H. Xie, G. Xie, M. Li, Q. Lu, Z. Wang, S. Lai, Z. Chai, and X. Wei, "Grand challenge of 106-point facial landmark localization," arXiv e-prints, p. arXiv:1905.03469, May 2019.
  53. X. Lan, Q. Hu, and J. Cheng, "Hih: Towards more accurate face alignment via heatmap in heatmap," arXiv preprint arXiv:2104.03100, 2021.
  54. X. Wang, L. Bo, and L. Fuxin, "Adaptive wing loss for robust face alignment via heatmap regression," Proceedings of the IEEE International Conference on Computer Vision, pp. 6970-6980, Apr 2019.
  55. X. P. Burgos-Artizzu, P. Perona, and P. Dollar, "Robust face landmark estimation under occlusion," in Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2013.
  56. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 580-587.
  57. T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollár, "Microsoft COCO: Common Objects in Context," arXiv e-prints, p. arXiv:1405.0312, May 2014.
  58. R. Girshick, "Fast R-CNN," arXiv e-prints, p. arXiv:1504.08083, Apr 2015.
  59. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "Mo- bileNetV2: Inverted residuals and linear bottlenecks," Jan 2018.
  60. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778.
  61. J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, D. Liu, Y. Mu, M. Tan, X. Wang, W. Liu, and B. Xiao, "Deep high-resolution repre- sentation learning for visual recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 10, pp. 3349-3364, 2021.
  62. C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, "300 faces in-the-wild challenge: The first facial landmark localization challenge," in 2013 IEEE International Conference on Computer Vision Workshops, 2013, pp. 397-403.
  63. C. Sagonas, E. Antonakos, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, "300 faces in-the-wild challenge: Database and results." Image Vis. Com- put., vol. 47, pp. 3-18, 2016.
  64. X. Zhu and D. Ramanan, "Face detection, pose estimation, and landmark localization in the wild," in 2012 IEEE Conference on Computer Vision and Pattern Recognition, June 2012, pp. 2879-2886.
  65. V. Le, J. Brandt, Z. Lin, L. Bourdev, and T. S. Huang, "Interactive facial feature localization," Computer Vision -ECCV 2012 Lecture Notes in Computer Science, p. 679-692, 2012.
  66. C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, "A semi- automatic methodology for facial landmark annotation." in CVPR Work- shops. IEEE Computer Society, 2013, pp. 896-903.
  67. S. Yang, P. Luo, C. C. Loy, and X. Tang, "Wider face: A face detection benchmark," in IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), 2016.
  68. J. Deng, J. Guo, Z. Yuxiang, J. Yu, I. Kotsia, and S. Zafeiriou, "Retinaface: Single-stage dense face localisation in the wild," in arxiv, 2019.
  69. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "ImageNet: A Large-Scale Hierarchical Image Database," in CVPR09, 2009.
  70. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdi- nov, "Dropout: A simple way to prevent neural networks from overfitting," Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929-1958, 2014.
  71. K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1026-1034.
  72. Y. You, J. Li, S. Reddi, J. Hseu, S. Kumar, S. Bhojanapalli, X. Song, J. Demmel, K. Keutzer, and C.-J. Hsieh, "Large batch optimization for deep learning: Training BERT in 76 minutes," Tech. Rep., 2019.
  73. T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, and Z. Zhang, "MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems," arXiv e-prints, p. arXiv:1512.01274, Dec. 2015.
  74. J. Guo, H. He, T. He, L. Lausen, M. Li, H. Lin, X. Shi, C. Wang, J. Xie, S. Zha, A. Zhang, H. Zhang, Z. Zhang, Z. Zhang, S. Zheng, and Y. Zhu, "Gluoncv and gluonnlp: Deep learning in computer vision and natural language processing," Journal of Machine Learning Research, vol. 21, no. 23, pp. 1-7, 2020.
  75. S. W. F. Earp, P. Noinongyao, J. A. Cairns, and A. Ganguly, "Face detection with feature pyramids and landmarks," arXiv e-prints, p. arXiv:1912.00596, Dec. 2019.

Related papers

H.: Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization
Paul Wohlhart

2011

Face alignment is a crucial step in face recognition tasks. Especially, using landmark localization for geometric face normalization has shown to be very effective, clearly improving the recognition results. However, no adequate databases exist that provide a sufficient number of annotated facial landmarks. The databases are either limited to frontal views, provide only a small number of annotated images or have been acquired under controlled conditions. Hence, we introduce a novel database overcoming these limitations: Annotated Facial Landmarks in the Wild (AFLW). AFLW provides a large-scale collection of images gathered from Flickr, exhibiting a large variety in face appearance (e.g., pose, expression, ethnicity, age, gender) as well as general imaging and environmental conditions. In total 25,993 faces in 21,997 real-world images are annotated with up to 21 landmarks per image. Due to the comprehensive set of annotations AFLW is well suited to train and test algorithms for multi...

View PDFchevron_right
Annotated Facial Landmarks in the Wild: A Large-scale, Real-world Database for Facial Landmark Localization
Paul Wohlhart

lrs.icg.tugraz.at

Face alignment is a crucial step in face recognition tasks. Especially, using landmark localization for geometric face normalization has shown to be very effective, clearly improving the recognition results. However, no adequate databases exist that provide a sufficient number of annotated facial landmarks. The databases are either limited to frontal views, provide only a small number of annotated images or have been acquired under controlled conditions. Hence, we introduce a novel database overcoming these limitations: Annotated Facial Landmarks in the Wild (AFLW). AFLW provides a large-scale collection of images gathered from Flickr, exhibiting a large variety in face appearance (e.g., pose, expression, ethnicity, age, gender) as well as general imaging and environmental conditions. In total 25,993 faces in 21,997 real-world images are annotated with up to 21 landmarks per image. Due to the comprehensive set of annotations AFLW is well suited to train and test algorithms for multi-view face detection, facial landmark localization and face pose estimation. Further, we offer a rich set of tools that ease the integration of other face databases and associated annotations into our joint framework.

View PDFchevron_right
Approaching human level facial landmark localization by deep learning
nathik azad

In this paper we present our solution to the 300 Faces in the Wild Facial Landmark Localization Challenge. We demonstrate how to achieve very competitive localization performance with a simple deep learning based system. Human study is conducted to show that the accuracy of our system has been very close to human performance. We discuss how this finding would affect our future direction to improve our system.

View PDFchevron_right
Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning
Jianguo Zhang

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Heatmap regression methods have dominated face alignment area in recent years while they ignore the inherent relation between different landmarks. In this paper, we propose a Sparse Local Patch Transformer (SLPT) for learning the inherent relation. The SLPT generates the representation of each single landmark from a local patch and aggregates them by an adaptive inherent relation based on the attention mechanism. The subpixel coordinate of each landmark is predicted independently based on the aggregated feature. Moreover, a coarse-to-fine framework is further introduced to incorporate with the SLPT, which enables the initial landmarks to gradually converge to the target facial landmarks using fine-grained features from dynamically resized local patches. Extensive experiments carried out on three popular benchmarks, including WFLW, 300W and COFW, demonstrate that the proposed method works at the state-of-the-art level with much less computational complexity by learning the inherent relation between facial landmarks. The code is available at the project website 1 .

View PDFchevron_right
Coarse-to-fine Face Alignment with Multi-Scale Local Patch Regression
Zhiao Huang

2015

Facial landmark localization plays an important role in face recognition and analysis applications. In this paper, we give a brief introduction to a coarse-to-fine pipeline with neural networks and sequential regression. First, a global convolutional network is applied to the holistic facial image to give an initial landmark prediction. A pyramid of multi-scale local image patches is then cropped to feed to a new network for each landmark to refine the prediction. As the refinement network outputs a more accurate position estimation than the input, such procedure could be repeated several times until the estimation converges. We evaluate our system on the 300-W dataset [11] and it outperforms the recent state-of-the-arts.

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Computer Vision
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved