Academia.eduAcademia.edu

Outline

Title
Abstract
Key Takeaways
Introduction
Methodology
Training Dataset
Results
Baseline Results
Results on JD-Landmark Datasets
Conclusions
References
All Topics
Computer Science

Sub-pixel face landmarks using heatmaps and a bag of tricks

Profile image of Pavit NoinongyaoPavit Noinongyao

2021, ArXiv

Abstract

Accurate face landmark localization is an essential part of face recognition, reconstruction and morphing. To accurately localize face landmarks, we present our heatmap regression approach. Each model consists of a MobileNetV2 backbone followed by several upscaling layers, with different tricks to optimize both performance and inference cost. We use five naïve face landmarks from a publicly available face detector to position and align the face instead of using the bounding box like traditional methods. Moreover, we show by adding random rotation, displacement and scaling—after alignment—that the model is more sensitive to the face position than orientation. We also show that it is possible to reduce the upscaling complexity by using a mixture of deconvolution and pixel-shuffle layers without impeding localization performance. We present our state-of-the-art face landmark localization model (ranking second on The 2nd Grand Challenge of 106-Point Facial Landmark Localization validati...

Key takeaways
sparkles

AI

  1. The proposed model ranks second in the JD-landmark-2 challenge, utilizing a MobileNetV2 backbone.
  2. Heatmap regression improves landmark localization accuracy compared to traditional bounding box methods.
  3. Random rotation enhances model sensitivity to face position, while scaling and repositioning reduce performance.
  4. Combining pixel-shuffle and deconvolution layers minimizes FLOPS with minimal impact on localization accuracy.
  5. Facial landmark adjustments improve face recognition performance by 0.07% to 0.44% on specific benchmarks.

Related papers

Approaching human level facial landmark localization by deep learning
nathik azad

In this paper we present our solution to the 300 Faces in the Wild Facial Landmark Localization Challenge. We demonstrate how to achieve very competitive localization performance with a simple deep learning based system. Human study is conducted to show that the accuracy of our system has been very close to human performance. We discuss how this finding would affect our future direction to improve our system.

View PDFchevron_right
Head Pose Estimation in the Wild Assisted by Facial Landmarks Based on Convolutional Neural Networks
Jiacai Liao

IEEE Access, 2019

Convolutional neural networks (CNNs) exhibit excellent performance on the head pose estimation problem under controllable conditions, but their generalization ability in the wild needs to be improved. To address this issue, we propose an approach involving the introduction of facial landmark information into the task simplifier and landmark heatmap generator constructed before the feed-forward neural network, which can use this information to normalize the face shape into a canonical shape and generate a landmark heatmap based on the transformed facial landmarks to assist in feature extraction, for enhancing generalization ability in the wild. Our method was trained on 300W-LP and tested on AFLW2000-3D. The result shows that for the same feed-forward neural network, when our method is used to introduce facial landmark information into a CNN, accuracy improves from 88.5% to 99.0% and mean average error decreases from 5.94 • to 1.46 • on AFLW2000-3D. Furthermore, we evaluate our method on several datasets used for pose estimation and compare the result with AFLW2000-3D, finding that the features extracted by a CNN could not reflect the head pose efficiently, which limits the performance of the CNN on the head pose estimation problem in wild. By introducing facial landmarks, the CNN could extract features that reflect head pose more efficiently, thereby significantly improving the accuracy of head pose estimation in the wild. INDEX TERMS head pose estimation, convolutional neural network, landmark heatmap, faceshape normalization

View PDFchevron_right
A lightweight 3D dense facial landmark estimation model from position map data
Michael Schukat

Zenodo (CERN European Organization for Nuclear Research), 2023

The incorporation of 3D data in facial analysis tasks has gained popularity in recent years. Though it provides a more accurate and detailed representation of the human face, accruing 3D face data is more complex and expensive than 2D face images. Either one has to rely on expensive 3D scanners or depth sensors which are prone to noise. An alternative option is the reconstruction of 3D faces from uncalibrated 2D images in an unsupervised way without any ground truth 3D data. However, such approaches are computationally expensive and the learned model size is not suitable for mobile or other edge device applications. Predicting dense 3D landmarks over the whole face can overcome this issue. As there is no public dataset available containing dense landmarks, we propose a pipeline to create a dense keypoint training dataset containing 520 key points across the whole face from an existing facial position map data. We train a lightweight MobileNet-based regressor model with the generated data. As we do not have access to any evaluation dataset with dense landmarks in it we evaluate our model against the 68 keypoint detection task. Experimental results show that our trained model outperforms many of the existing methods in spite of its lower model size and minimal computational cost. Also, the qualitative evaluation shows the efficiency of our trained models in extreme head pose angles as well as other facial variations and occlusions. Code is

View PDFchevron_right
Lightweight and Effective Facial Landmark Detection using Adversarial Learning with Face Geometric Map Generative Network
Kyung Hee University

Facial landmark detection plays an important role in face analysis tasks. Moreover, it is used as a prerequisite in many facial related applications, the simplicity, as well as effectiveness, is essential in the facial landmark detection. In this paper, we propose an effective facial landmark detection network and an associated learning framework with the geometric prior-generative adversarial network. The geometric prior-generative adversarial network consists of one generator and two discriminators. The generator consists of an encoder and two decoders. The encoder predicts facial landmark points. The decoders generate a facial inner and contour geometric map from predicted landmark points. Generating face geometric maps from predicted landmark points helps the predicted landmark points to represent the face geometric information, including shape and configuration. The discriminators determine that the given geometric maps are generated from actual landmark points or estimated landmark points. Our proposed network is end-to-end trainable, and only the encoder part is used simply as the facial landmark detector in the testing stage. To verify the effectiveness of the proposed method, we have conducted comprehensive experiments with benchmark data sets. The results have shown that the proposed method achieves comparable performances over recently proposed facial landmark detection methods with a simple and effective facial landmark detection network.

View PDFchevron_right
A Detailed Look At CNN-based Approaches In Facial Landmark Detection
Chin-Laung Lei

arXiv (Cornell University), 2020

Facial landmark detection has been studied over decades. Numerous neural network (NN)-based approaches have been proposed for detecting landmarks, especially the convolutional neural network (CNN)-based approaches. In general, CNN-based approaches can be divided into regression and heatmap approaches. However, no research systematically studies the characteristics of different approaches. In this paper, we investigate both CNN-based approaches, generalize their advantages and disadvantages, and introduce a variation of the heatmap approach, a pixel-wise classification (PWC) model. To the best of our knowledge, using the PWC model to detect facial landmarks have not been comprehensively studied. We further design a hybrid loss function and a discrimination network for strengthening the landmarks' interrelationship implied in the PWC model to improve the detection accuracy without modifying the original model architecture. Six common facial landmark datasets, AFW, Helen, LFPW, 300-W, IBUG, and COFW are adopted to train or evaluate our model. A comprehensive evaluation is conducted and the result shows that the proposed model outperforms other models in all tested datasets.

View PDFchevron_right
Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning
Jianguo Zhang

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Heatmap regression methods have dominated face alignment area in recent years while they ignore the inherent relation between different landmarks. In this paper, we propose a Sparse Local Patch Transformer (SLPT) for learning the inherent relation. The SLPT generates the representation of each single landmark from a local patch and aggregates them by an adaptive inherent relation based on the attention mechanism. The subpixel coordinate of each landmark is predicted independently based on the aggregated feature. Moreover, a coarse-to-fine framework is further introduced to incorporate with the SLPT, which enables the initial landmarks to gradually converge to the target facial landmarks using fine-grained features from dynamically resized local patches. Extensive experiments carried out on three popular benchmarks, including WFLW, 300W and COFW, demonstrate that the proposed method works at the state-of-the-art level with much less computational complexity by learning the inherent relation between facial landmarks. The code is available at the project website 1 .

View PDFchevron_right
Localizing Parts of Faces Using a Consensus of Exemplars
Peter Belhumeur

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000

We present a novel approach to localizing parts in images of human faces. The approach combines the output of local detectors with a nonparametric set of global models for the part locations based on over 1,000 hand-labeled exemplar images. By assuming that the global models generate the part locations as hidden variables, we derive a Bayesian objective function. This function is optimized using a consensus of models for these hidden variables. The resulting localizer handles a much wider range of expression, pose, lighting, and occlusion than prior ones. We show excellent performance on real-world face datasets such as Labeled Faces in the Wild (LFW) and a new Labeled Face Parts in the Wild (LFPW) and show that our localizer achieves state-of-the-art performance on the less challenging BioID dataset.

View PDFchevron_right
68 landmarks are efficient for 3D face alignment: what about more?
ALI WALI

Multimedia Tools and Applications

This paper proposes a 3D face alignment of 2D face images in the wild with noisy landmarks. The objective is to recognize individuals from their single profile image. We first proceed by extracting more than 68 landmarks using a bag of features. This allows us to obtain a bag of visible and invisible facial keypoints. Then, we reconstruct a 3D face model and get a triangular mesh by meshing the obtained keypoints. For each face, the number of keypoints is not the same, which makes this step very challenging. Later, we process the 3D face using butterfly and BPA algorithms to make correlation and regularity between 3D face regions. Indeed, 2D-to-3D annotations give much higher quality to the 3D reconstructed face model without the need for any additional 3D Morphable models. Finally, we carry out alignment and pose correction steps to get frontal pose by fitting the rendered 3D reconstructed face to 2D face and performing pose normalization to achieve good rates in face recognition. The recognition step is based on deep learning and it is performed using DCNNs, which are very powerful and modern, for feature learning and face identification. To verify the proposed method, three popular benchmarks, YTF, LFW, and BIWI databases, are tested. Compared to the best recognition results reported on these benchmarks, our proposed method achieves comparable or even better recognition performances.

View PDFchevron_right
Simultaneous regression and feature learning for facial landmarking
Peter Peer

2019

Face alignment (or facial landmarking) is an important task in many face-related applications, ranging from registration, tracking and animation to higher-level classification problems such as face, expression or attribute recognition. While several solutions have been presented in the literature for this task so far, reliably locating salient facial features across a wide range of posses still remains challenging. To address this issue, we propose in this paper a novel method for automatic facial landmark localization in 3D face data designed specifically to address appearance variability caused by significant pose variations. Our method builds on recent cascaded-regression-based methods to facial landmarking and uses a gating mechanism to incorporate multiple linear cascaded regression models each trained for a limited range of poses into a single powerful landmarking model capable of processing arbitrary posed input data. We develop two distinct approaches around the proposed gat...

View PDFchevron_right
ACE-Net: Fine-Level Face Alignment through Anchors and Contours Estimation
Amir Tamrakar

ArXiv, 2020

We propose a novel facial Anchors and Contours Estimation framework, ACE-Net, for fine-level face alignment tasks. ACE-Net predicts facial anchors and contours that are richer than traditional facial landmarks and more accurate than facial boundaries. In addition, it does not suffer from the ambiguities and inconsistencies in facial-landmarks definitions. We introduce a weakly supervised loss enabling ACE-Net to learn from existing facial landmarks datasets without the need for extra annotations. Synthetic data is also used during training to bridge the density gap between landmarks annotation and true facial contours. We evaluate ACE-Net on commonly used face alignment datasets 300-W and HELEN, and show that ACE-Net achieves significantly higher fine-level face alignment accuracy than landmarks based models, without compromising its performance at the landmarks level. The proposed ACE-Net framework does not rely on any specific network architecture and thus can be applied on top of...

View PDFchevron_right

References (76)

  1. B. Amberg and T. Vetter. Optimal landmark detection us- ing shape models and branch and bound. pages 455-462, 11 2011. doi: 10.1109/ICCV.2011.6126275.
  2. X. An, X. Zhu, Y. Xiao, L. Wu, M. Zhang, Y. Gao, B. Qin, D. Zhang, and Y. Fu. Partial FC: Training 10 Mil- lion Identities on a Single Machine. arXiv e-prints, art. arXiv:2010.05222, Oct. 2020.
  3. M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele. 2d human pose estimation: New benchmark and state of the art analysis. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 3686-3693, 2014. doi: 10.1109/CVPR.2014.471.
  4. P. Belhumeur, D. Jacobs, D. Kriegman, and N. Kumar. Lo- calizing parts of faces using a consensus of exemplars. IEEE transactions on pattern analysis and machine in- telligence, 35:2930-40, 12 2013. doi: 10.1109/TPAMI. 2013.23.
  5. C. Cao, Y. Weng, S. Lin, and K. Zhou. 3d shape re- gression for real-time facial animation. ACM Trans. Graph., 32(4), July 2013. ISSN 0730-0301. doi: 10.1145/2461912.2462012. URL https://doi.org/ 10.1145/2461912.2462012.
  6. T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, and Z. Zhang. MXNet: A Flexi- ble and Efficient Machine Learning Library for Het- erogeneous Distributed Systems. arXiv e-prints, art. arXiv:1512.01274, Dec. 2015.
  7. T. Chen, T. Moreau, Z. Jiang, L. Zheng, E. Yan, M. Cowan, H. Shen, L. Wang, Y. Hu, L. Ceze, C. Guestrin, and A. Krishnamurthy. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning. arXiv e-prints, art. arXiv:1802.04799, Feb. 2018.
  8. Y. Chen and T. Pock. Trainable nonlinear reaction diffu- sion: A flexible framework for fast and effective im- age restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6):1256-1272, 2017. doi: 10.1109/TPAMI.2016.2596743.
  9. T. Cootes, C. Taylor, D. Cooper, and J. Graham. Ac- tive shape models-their training and application. Com- puter Vision and Image Understanding, 61(1):38 -59, 1995. ISSN 1077-3142. doi: https://doi.org/10.1006/ cviu.1995.1004. URL http://www.sciencedirect. com/science/article/pii/S1077314285710041.
  10. T. F. Cootes, G. J. Edwards, and C. J. Taylor. Active ap- pearance models. In IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, pages 484-498. Springer, 1998.
  11. M. Dantone, J. Gall, C. Leistner, and L. van Gool. Human pose estimation using body parts dependent joint regres- sors. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3041-3048, Portland, OR, USA, June 2013. IEEE.
  12. M. Day. Exploiting facial landmarks for emotion recog- nition in the wild. CoRR, abs/1603.09129, 2016. URL http://arxiv.org/abs/1603.09129.
  13. J. Deng, J. Guo, and S. Zafeiriou. Arcface: Additive an- gular margin loss for deep face recognition. CoRR, abs/1801.07698, 2018. URL http://arxiv.org/ abs/1801.07698.
  14. J. Deng, J. Guo, Z. Yuxiang, J. Yu, I. Kotsia, and S. Zafeiriou. Retinaface: Single-stage dense face lo- calisation in the wild. In arxiv, 2019a.
  15. J. Deng, A. T. Roussos, G. Chrysos, E. Ververas, I. Kotsia, J. Shen, and S. Zafeiriou. The menpo benchmark for multi-pose 2d and 3d facial landmark localisation and tracking. International Journal of Computer Vision, 127, 06 2019b. doi: 10.1007/s11263-018-1134-y.
  16. H. Dibeklioglu, A. A. Salah, and L. Akarun. 3d facial landmarking under expression, pose, and occlusion vari- ations. In 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems, pages 1-6, 2008. doi: 10.1109/BTAS.2008.4699324.
  17. C. Dong, C. C. Loy, K. He, and X. Tang. Image super- resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 38(2):295-307, 2016. doi: 10.1109/TPAMI.2015. 2439281.
  18. P. Dou, S. K. Shah, and I. A. Kakadiaris. End-to-end 3D face reconstruction with deep neural networks. Proceed- ings -30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-January:1503- 1512, apr 2017. URL http://arxiv.org/abs/1704. 05020.
  19. S. W. F. Earp, P. Noinongyao, J. A. Cairns, and A. Ganguly. Face Detection with Feature Pyramids and Landmarks. arXiv e-prints, art. arXiv:1912.00596, Dec. 2019.
  20. B. A. Efraty, M. Papadakis, A. Profitt, S. Shah, and I. A. Kakadiaris. Facial component-landmark detection. In 2011 IEEE International Conference on Automatic Face Gesture Recognition (FG), pages 278-285, 2011. doi: 10.1109/FG.2011.5771411.
  21. M. Eichner and V. Ferrari. Better appearance models for pictorial structures. 23, 01 2009. doi: 10.5244/C.23.3.
  22. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 32(9):1627- 1645, 2010. doi: 10.1109/TPAMI.2009.167.
  23. Y. Feng, F. Wu, X. Shao, Y. Wang, and X. Zhou. Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioin- formatics), 11218 LNCS:557-574, mar 2018. URL http://arxiv.org/abs/1803.07835.
  24. M. A. Fischler and R. A. Elschlager. The representa- tion and matching of pictorial structures. IEEE Trans- actions on Computers, C-22(1):67-92, 1973. doi: 10.1109/T-C.1973.223602.
  25. R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker. Multi-pie. In 2008 8th IEEE International Conference on Automatic Face Gesture Recognition, pages 1-8, 2008. doi: 10.1109/AFGR.2008.4813399.
  26. J. Guo, H. He, T. He, L. Lausen, M. Li, H. Lin, X. Shi, C. Wang, J. Xie, S. Zha, A. Zhang, H. Zhang, Z. Zhang, Z. Zhang, S. Zheng, and Y. Zhu. Gluoncv and gluonnlp: Deep learning in computer vision and natural language processing. Journal of Machine Learning Research, 21(23):1-7, 2020. URL http://jmlr.org/papers/ v21/19-429.html.
  27. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn- ing for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016. doi: 10.1109/CVPR.2016.90.
  28. S. Hinduja and S. Canavan. Facial Action Unit Detec- tion using 3D Facial Landmarks. arXiv e-prints, art. arXiv:2005.08343, May 2020.
  29. A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan, Q. V. Le, and H. Adam. Searching for MobileNetV3. may 2019. URL http://arxiv.org/abs/1905.02244.
  30. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. apr 2017. URL http://arxiv.org/abs/1704.04861.
  31. G. Huang, M. Mattar, T. Berg, and E. Learned-Miller. La- beled faces in the wild: A database forstudying face recognition in unconstrained environments. Tech. rep., 10 2008a.
  32. G. B. Huang, M. Mattar, T. Berg, and E. Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In Workshop on Faces in 'Real-Life' Images: Detection, Alignment, and Recognition, Marseille, France, Oct. 2008b. Erik Learned-Miller and Andras Ferencz and Frédéric Jurie. URL https://hal.inria.fr/inria-00321923.
  33. J. Huang, Z. Zhu, G. Huang, and D. Du. Aid: Pushing the performance boundary of human pose estimation with information dropping augmentation. arXiv preprint arXiv:2008.07139, 2020.
  34. D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. arXiv e-prints, art. arXiv:1412.6980, Dec. 2014.
  35. M. Kowalski, J. Naruniec, and T. Trzcinski. Deep align- ment network: A convolutional neural network for ro- bust face alignment. CoRR, abs/1706.01789, 2017. URL http://arxiv.org/abs/1706.01789.
  36. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Ima- genet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25, pages 1097-1105. Curran Associates, Inc., 2012. URL https: //proceedings.neurips.cc/paper/2012/file/ c399862d3b9d6b76c8436e924a68c45b-Paper. pdf. N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar. Attribute and simile classifiers for face ver- ification. In 2009 IEEE 12th International Confer- ence on Computer Vision, pages 365-372, 2009. doi: 10.1109/ICCV.2009.5459250.
  37. L. Liang, R. Xiao, F. Wen, and J. Sun. Face align- ment via component-based discriminative search. pages 72-85, 10 2008. ISBN 978-3-540-88685-3. doi: 10.1007/978-3-540-88688-4_6.
  38. T. Lin, M. Maire, S. J. Belongie, L. D. Bourdev, R. B. Girshick, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft COCO: common ob- jects in context. CoRR, abs/1405.0312, 2014. URL http://arxiv.org/abs/1405.0312.
  39. W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song. Sphereface: Deep hypersphere embedding for face recognition. CoRR, abs/1704.08063, 2017. URL http://arxiv.org/abs/1704.08063.
  40. Y. Liu, H. Shen, Y. Si, X. Wang, X. Zhu, H. Shi, Z. Hong, H. Guo, Z. Guo, Y. Chen, B. Li, T. Xi, J. Yu, H. Xie, G. Xie, M. Li, Q. Lu, Z. Wang, S. Lai, Z. Chai, and X. Wei. Grand Challenge of 106-Point Facial Landmark Localization. arXiv e-prints, art. arXiv:1905.03469, May 2019.
  41. S. Mahpod, R. Das, E. Maiorana, Y. Keller, and P. Campisi. Facial landmark point localization using coarse-to-fine deep recurrent neural network. ArXiv, abs/1805.01760, 2018.
  42. M. I. N. P. Munasinghe. Facial expression recognition using facial landmarks and random forest classifier. In 2018 IEEE/ACIS 17th International Conference on Com- puter and Information Science (ICIS), pages 423-427, 2018. doi: 10.1109/ICIS.2018.8466510.
  43. A. Newell, K. Yang, and J. Deng. Stacked hourglass net- works for human pose estimation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioin- formatics), 9912 LNCS:483-499, mar 2016. ISSN 16113349. doi: 10.1007/978-3-319-46484-8_29. URL http://arxiv.org/abs/1603.06937.
  44. C. Osendorfer, H. Soyer, and P. van der Smagt. Image super-resolution with fast approximate convolutional sparse coding. 11 2014. ISBN 978-3-319-12642-5. doi: 10.1007/978-3-319-12643-2_31.
  45. O. M. Parkhi, A. Vedaldi, and A. Zisserman. Deep face recognition. In Proceedings of the British Ma- chine Vision Conference (BMVC), pages 41.1-41.12. BMVA Press, September 2015. ISBN 1-901725-53-7. doi: 10.5244/C.29.41. URL https://dx.doi.org/ 10.5244/C.29.41.
  46. R. Ranjan, V. M. Patel, and R. Chellappa. Hyperface: A deep multi-task learning framework for face detec- tion, landmark localization, pose estimation, and gen- der recognition. CoRR, abs/1603.01249, 2016. URL http://arxiv.org/abs/1603.01249.
  47. J. Roth, Y. Tong, and X. Liu. Unconstrained 3D face recon- struction. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recogni- tion, volume 07-12-June-2015, pages 2606-2615. IEEE Computer Society, oct 2015. ISBN 9781467369640. doi: 10.1109/CVPR.2015.7298876.
  48. C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pan- tic. 300 faces in-the-wild challenge: The first facial landmark localization challenge. In 2013 IEEE Inter- national Conference on Computer Vision Workshops, pages 397-403, 2013. doi: 10.1109/ICCVW.2013.59.
  49. C. Sagonas, E. Antonakos, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic. 300 faces in-the-wild challenge: database and results. Image and Vision Computing, 47, 01 2016. doi: 10.1016/j.imavis.2016.01.002.
  50. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. MobileNetV2: Inverted Residuals and Linear Bot- tlenecks. jan 2018. URL http://arxiv.org/abs/ 1801.04381.
  51. F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clus- tering. CoRR, abs/1503.03832, 2015. URL http: //arxiv.org/abs/1503.03832.
  52. S. Sengupta, J. Chen, C. Castillo, V. M. Patel, R. Chellappa, and D. W. Jacobs. Frontal to profile face verification in the wild. In 2016 IEEE Winter Conference on Appli- cations of Computer Vision (WACV), pages 1-9, March 2016. doi: 10.1109/WACV.2016.7477558.
  53. W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang. Real-time single image and video super-resolution using an ef- ficient sub-pixel convolutional neural network. In 2016 IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 1874-1883, 2016. doi: 10.1109/CVPR.2016.207.
  54. K. Sun, B. Xiao, D. Liu, and J. Wang. Deep high-resolution representation learning for human pose estimation. In CVPR, 2019.
  55. X. Sun, S. Jiaxiang, S. Liang, and Y. Wei. Compo- sitional human pose regression. Computer Vision and Image Understanding, 176-177, 04 2017. doi: 10.1016/j.cviu.2018.10.006.
  56. Y. Sun, X. Wang, and X. Tang. Deep convolutional network cascade for facial point detection. In 2013 IEEE Con- ference on Computer Vision and Pattern Recognition, pages 3476-3483, 2013. doi: 10.1109/CVPR.2013.446.
  57. J. Tompson, A. Jain, Y. Lecun, and C. Bregler. Joint train- ing of a convolutional network and a graphical model for human pose estimation. 06 2014a.
  58. J. Tompson, A. Jain, Y. LeCun, and C. Bregler. Joint train- ing of a convolutional network and a graphical model for human pose estimation. CoRR, abs/1406.2984, 2014b. URL http://arxiv.org/abs/1406.2984.
  59. A. Toshev and C. Szegedy. Deeppose: Human pose es- timation via deep neural networks. pages 1653-1660, 2014. doi: 10.1109/CVPR.2014.214.
  60. H. Wang, Y. Wang, Z. Zhou, X. Ji, Z. Li, D. Gong, J. Zhou, and W. Liu. Cosface: Large margin cosine loss for deep face recognition. CoRR, abs/1801.09414, 2018. URL http://arxiv.org/abs/1801.09414.
  61. Z. Wang, D. Liu, J. Yang, W. Han, and T. Huang. Deeply improved sparse coding for image super-resolution. 07 2015.
  62. S. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh. Con- volutional pose machines. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4724-4732, 2016. doi: 10.1109/CVPR.2016.511.
  63. L. Wolf, T. Hassner, and Y. Taigman. Similarity Scores Based on Background Samples. In H. Zha, R.-i. Taniguchi, and S. Maybank, editors, Computer Vision -ACCV 2009, pages 88-97, Berlin, Heidelberg, 2010. Springer Berlin Heidelberg. ISBN 978-3-642-12304-7.
  64. B. Xiao, H. Wu, and Y. Wei. Simple baselines for human pose estimation and tracking. ArXiv, abs/1804.06208, 2018.
  65. X. Xiong and F. De la Torre. Supervised descent method and its applications to face alignment. In 2013 IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 532-539, 06 2013. doi: 10.1109/CVPR. 2013.75.
  66. Y. Xiong, Z. Zhou, Y. Dou, and Z. Su. Gaussian vector: An efficient solution for facial landmark detection, 2020.
  67. X. Yang, Y. Li, H. Qi, and S. Lyu. Exposing gan- synthesized faces using landmark locations. CoRR, abs/1904.00167, 2019. URL http://arxiv.org/ abs/1904.00167.
  68. Y. Yang and D. Ramanan. Articulated human detection with flexible mixtures of parts. IEEE transactions on pattern analysis and machine intelligence, 35:2878-90, 12 2013. doi: 10.1109/TPAMI.2012.261.
  69. F. Zhang, X. Zhu, H. Dai, M. Ye, and C. Zhu. Distribution- aware coordinate representation for human pose estima- tion, 2019.
  70. F. Zhang, X. Zhu, H. Dai, M. Ye, and C. Zhu. Distribution- aware coordinate representation for human pose estima- tion. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  71. J. Zhang, S. Shan, M. Kan, and X. Chen. Coarse-to-fine auto-encoder networks (cfan) for real-time face align- ment. In D. Fleet, T. Pajdla, B. Schiele, and T. Tuyte- laars, editors, Computer Vision -ECCV 2014, pages 1-16, Cham, 2014. Springer International Publishing. ISBN 978-3-319-10605-2.
  72. Z. Zhang, P. Luo, C. C. Loy, and X. Tang. Learn- ing deep representation for face alignment with aux- iliary attributes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5):918-930, 2016. doi: 10.1109/TPAMI.2015.2469286.
  73. T. Zheng and W. Deng. Cross-pose lfw: A database for studying cross-pose face recognition in unconstrained environments. Beijing University of Posts and Telecom- munications, Tech. Rep, 5, 2018.
  74. T. Zheng, W. Deng, and J. Hu. Cross-age lfw: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.08197, 2017.
  75. E. Zhou, H. Fan, Z. Cao, Y. Jiang, and Q. Yin. Extensive facial landmark localization with coarse-to-fine convo- lutional network cascade. In 2013 IEEE International Conference on Computer Vision Workshops, pages 386- 391, 2013. doi: 10.1109/ICCVW.2013.58.
  76. X. Zhu and D. Ramanan. Face detection, pose estimation, and landmark localization in the wild. In 2012 IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 2879-2886, 2012. doi: 10.1109/CVPR.2012. 6248014.

Related papers

Annotated Facial Landmarks in the Wild: A Large-scale, Real-world Database for Facial Landmark Localization
Paul Wohlhart

lrs.icg.tugraz.at

Face alignment is a crucial step in face recognition tasks. Especially, using landmark localization for geometric face normalization has shown to be very effective, clearly improving the recognition results. However, no adequate databases exist that provide a sufficient number of annotated facial landmarks. The databases are either limited to frontal views, provide only a small number of annotated images or have been acquired under controlled conditions. Hence, we introduce a novel database overcoming these limitations: Annotated Facial Landmarks in the Wild (AFLW). AFLW provides a large-scale collection of images gathered from Flickr, exhibiting a large variety in face appearance (e.g., pose, expression, ethnicity, age, gender) as well as general imaging and environmental conditions. In total 25,993 faces in 21,997 real-world images are annotated with up to 21 landmarks per image. Due to the comprehensive set of annotations AFLW is well suited to train and test algorithms for multi-view face detection, facial landmark localization and face pose estimation. Further, we offer a rich set of tools that ease the integration of other face databases and associated annotations into our joint framework.

View PDFchevron_right
H.: Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization
Paul Wohlhart

2011

Face alignment is a crucial step in face recognition tasks. Especially, using landmark localization for geometric face normalization has shown to be very effective, clearly improving the recognition results. However, no adequate databases exist that provide a sufficient number of annotated facial landmarks. The databases are either limited to frontal views, provide only a small number of annotated images or have been acquired under controlled conditions. Hence, we introduce a novel database overcoming these limitations: Annotated Facial Landmarks in the Wild (AFLW). AFLW provides a large-scale collection of images gathered from Flickr, exhibiting a large variety in face appearance (e.g., pose, expression, ethnicity, age, gender) as well as general imaging and environmental conditions. In total 25,993 faces in 21,997 real-world images are annotated with up to 21 landmarks per image. Due to the comprehensive set of annotations AFLW is well suited to train and test algorithms for multi...

View PDFchevron_right
Face Sketch Landmarks Localization in the Wild
Heng Yang

In this paper we propose a method for facial landmarks localization in face sketch images. As recent approaches and the corresponding datasets are designed for ordinary face photos, the performance of such models drop significantly when they are applied on face sketch images. We first propose a scheme to synthesize face sketches from face photos based on randomforests edge detection and local face region enhancement. Then we jointly train a Cascaded Pose Regression based method for facial landmarks localization for both face photos and sketches. We build an evaluation dataset, called Face Sketches in the Wild (FSW), with 450 face sketch images collected from the Internet and with the manual annotation of 68 facial landmark locations on each face sketch. The proposed multi-modality facial landmark localization method shows competitive performance on both face sketch images (the FSW dataset) and face photo images (the Labeled Face Parts in the Wild dataset), despite the fact that we do not use extra annotation of face sketches for model building.

View PDFchevron_right
Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression
Fuxin Li

arXiv (Cornell University), 2019

Heatmap regression with a deep network has become one of the mainstream approaches to localize facial landmarks. However, the loss function for heatmap regression is rarely studied. In this paper, we analyze the ideal loss function properties for heatmap regression in face alignment problems. Then we propose a novel loss function, named Adaptive Wing loss, that is able to adapt its shape to different types of ground truth heatmap pixels. This adaptability penalizes loss more on foreground pixels while less on background pixels. To address the imbalance between foreground and background pixels, we also propose Weighted Loss Map, which assigns high weights on foreground and difficult background pixels to help training process focus more on pixels that are crucial to landmark localization. To further improve face alignment accuracy, we introduce boundary prediction and CoordConv with boundary coordinates. Extensive experiments on different benchmarks, including COFW, 300W and WFLW, show our approach outperforms the state-of-the-art by a significant margin on various evaluation metrics. Besides, the Adaptive Wing loss also helps other heatmap regression tasks. Code will be made publicly available at https://github.com/ protossw512/AdaptiveWingLoss.

View PDFchevron_right
LOTR: Face Landmark Localization Using Localization Transformer
Pavit Noinongyao

ArXiv, 2021

This paper presents a novel Transformer-based facial landmark localization network named Localization Transformer (LOTR). The proposed framework is a direct coordinate regression approach leveraging a Transformer network to better utilize the spatial information in the feature map. An LOTR model consists of three main modules: 1) a visual backbone that converts an input image into a feature map, 2) a Transformer module that improves the feature representation from the visual backbone, and 3) a landmark prediction head that directly predicts the landmark coordinates from the Transformer’s representation. Given cropped-and-aligned face images, the proposed LOTR can be trained end-to-end without requiring any post-processing steps. This paper also introduces the smooth-Wing loss function, which addresses the gradient discontinuity of the Wing loss, leading to better convergence than standard loss functions such as L1, L2, and Wing loss. Experimental results on the JD landmark dataset p...

View PDFchevron_right

Related topics

  • Computer Science
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved