Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
The Nature of Dataset Bias
Bias in Cross-Dataset Training
Bias in Dataset's Feature Space
Datasets
Dataset Pre-Processing
Cross-Dataset Training
Dataset's Feature Space
Experimental Results
Looking from the Dataset Side
Conclusions
References
All Topics
Computer Science
Computer Vision

Dataset bias exposed in face verification

Profile image of Xosé PardoXosé Pardo

2019, IET Biometrics

https://doi.org/10.1049/IET-BMT.2018.5224
visibility

…

description

15 pages

link

1 file

Abstract

Most facial verification methods assume that training and testing sets contain independent and identically distributed samples, although, in many real applications, this assumption does not hold. Whenever gathering a representative dataset in the target domain is unfeasible, it is necessary to choose one of the already available (source domain) datasets. In this paper, a study was performed over the differences among six public datasets, and how this impacts on the performance of the learned methods. In the considered scenario of mobile devices, the individual of interest is enrolled using a few facial images taken in the operational domain, while training impostors are drawn from one of the public available datasets. This work tried to shed light on the inherent differences among the datasets, and potential harms that should be considered when they are combined for training and testing. Results indicate that a drop in performance occurs whenever training and testing are done on different datasets compared to the case of using the same dataset in both phases. However, the decay strongly depends on the kind of features. Besides, the representation of samples in the feature space reveals insights into to what extent bias is an endogenous or an exogenous factor.

Related papers

Bias Mitigation of Face Recognition Models Through Calibration
Vikram Voleti

ArXiv, 2021

Face recognition models suffer from bias: for example, the probability of a false positive (incorrect face match) strongly depends on sensitive attributes like ethnicity. As a result, these models may disproportionately and negatively impact minority groups when used in law enforcement. In this work, we introduce the Bias Mitigation Calibration (BMC) method, which (i) increases model accuracy (improving the state-of-the-art), (ii) produces fairly-calibrated probabilities, (iii) significantly reduces the gap in the false positive rates, and (iv) does not require knowledge of the sensitive attribute.

View PDFchevron_right
Face system evaluation toolkit: Recognition is harder than it seems
Walter Scheirer

2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), 2010

Challenges for face recognition still exist in factors such as pose, blur and distance. Many current datasets containing mostly frontal images are regarded as being too easy. With obviously unsolved problems researchers are in need of datasets that test these remaining challenges. There are quite a few datasets in existence to study pose. Datasets to study blur and distance are almost non-existent. Datasets allowing for the study of these variables would prove to be useful to researchers in biometric surveillance applications. However, until now there has been no effective way to create datasets that encompass these three variables in a controlled fashion.

View PDFchevron_right
Biometric interoperability across training, enrollment, and testing for face authentication
Marilena De Marsico

2012 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications (BIOMS) Proceedings, 2012

The paper focuses on "personally identifiable information" (PII) and interoperability. The emphasis is on quantitative performance analysis and validation for uncontrolled operational settings and image quality, and variable demographics and gallery composition. Biometric face authentication involves three distinct operational stages, those of face space learning ("training"), gallery enrollment, and testing ("querying"). The authentication tasks considered here are face identification and verification. Performance evaluation involves k-fold cross-validation using both PCA and LDA for face space representation. Our basic findings indicate that (a) training to learn the face space is less important than the quality of images during enrollment and testing;(b) exclusion of first eigenvectors in defining the face space improves performance particularly for the PCA face space and lesser quality data; (c) the size of the subject gallery affects performance; and (d) it does not make much difference if the face space is derived from biometric data coming from the same dataset source as that used for enrollment and testing. Possible solutions to enhance overall performance and cope with adversarial (impostor) behavior during mass screening include non-inductive learning settings, e.g., transduction and transfer learning, using both labeled and unlabeled examples.

View PDFchevron_right
Human-Machine Comparison for Cross-Race Face Verification: Race Bias at the Upper Limits of Performance?
Prajay Mehta

arXiv (Cornell University), 2023

Face recognition algorithms perform more accurately than humans in some cases-though humans and machines both show race-based accuracy differences. As algorithms continue to improve, it is important to continually assess their race bias relative to humans. We constructed a challenging test of "cross-race" face verification and used it to compare humans and two state-of-the-art face recognition systems. Pairs of same-and different-identity faces of White and Black individuals were selected to be difficult for humans and an open-source implementation of the ArcFace face recognition algorithm from 2019 [5]. Human participants (54 Black; 51 White) judged whether face pairs showed the same identity or different identities on a 7-point Likert-type scale. Two top-performing face recognition systems from the Face Recognition Vendor Testongoing [7] performed the same test. By design, the test proved challenging for humans as a group, who performed above chance, but far less than perfect. Both state-of-theart face recognition systems scored perfectly (no errors), consequently with equal accuracy for both races. We conclude that state-of-the-art systems for identity verification between two frontal face images of Black and White individuals can surpass the general population. Whether this result generalizes to challenging in-the-wild images is a pressing concern for deploying face recognition systems in unconstrained environments.

View PDFchevron_right
Evaluation of Face Datasets as Tools for Assessing the Performance of Face Recognition Methods
Lior Shamir

International Journal of Computer Vision, 2008

Face datasets are considered a primary tool for evaluating the efficacy of face recognition methods.

View PDFchevron_right
A Study of Face Databases used as Benchmarks in Face Recognition
Sheela Kore

Face recognition has become one of the robust means of authentication and hence lots of research has been carried on in this regard. For any face recognition system, the availability of a standard database consisting of appropriate face image samples is very important, since it serves as a benchmark for testing and comparing the results directly for the face recognition algorithms. From the last few decades, the creation of face database by proper acquisition of face images, has been an interesting research topic among research community. While there are many face databases available, the appropriate choice should be based on the task given (age, lighting, poses, expression, etc.). This paper makes a scrutinizing study of the existing face databases. The aim here is to give a clear picture to the researchers regarding the selection of the face databases to build effective face recognition systems.

View PDFchevron_right
Face Recognition: Too Bias, or Not Too Bias?
Gennady Livitz

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020

We reveal critical insights into problems of bias in state-of-the-art facial recognition (FR) systems using a novel Balanced Faces In the Wild (BFW) dataset: data balanced for gender and ethnic groups. We show variations in the optimal scoring threshold for face-pairs across different subgroups. Thus, the conventional approach of learning a global threshold for all pairs results in performance gaps between subgroups. By learning subgroup-specific thresholds, we reduce performance gaps, and also show a notable boost in overall performance. Furthermore, we do a human evaluation to measure bias in humans, which supports the hypothesis that an analogous bias exists in human perception. For the BFW database, source code, and more, visit https://github.com/visionjo/facerec-bias-bfw.

View PDFchevron_right
State-of-the-art face recognition performance using publicly available software and datasets
Dijana Petrovska-delacrétaz

2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), 2018

We are interested in the reproducibility of face recognition systems. By reproducibility we mean: is the scientific community, and are the researchers from different sides, capable of reproducing the last published results by a big company, that has at its disposal huge computational power and huge proprietary databases? With the constant advancements in GPU computation power and availability of open-source software, the reproducibility of published results should not be a problem. But, if architectures of the systems are private and databases are proprietary, the reproducibility of published results can not be easily attained. To tackle this problem, we focus on training and evaluation of face recognition systems on publicly available data and software. We are also interested in comparing the best Deep Neural Net (DNN) based results with a baseline "classical" system. This paper exploits the OpenFace open-source system to generate a deep convolutional neural network model using publicly available datasets. We study the impact of the size of the datasets, their quality and compare the performance to a classical face recognition approach. Our focus is to have a fully reproducible model. To this end, we used publicly available datasets (FRGC, MS-celeb-1M, MOBIO, LFW), as well publicly available software (OpenFace) to train our model in order to do face recognition. Our best trained model achieves 97.52% accuracy on the Labelled in the Wild dataset (LFW) dataset which is lower than Google's best reported results of 99.96% but slightly better than FaceBook's reported result of 97.35%. We also evaluated our best model on the challenging video dataset MOBIO and report competitive results with the best reported results on this database.

View PDFchevron_right
Characterizing the Variability in Face Recognition Accuracy Relative to Race
Kushal Vangara

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019

Many recent news headlines have labeled face recognition technology as "biased" or "racist". We report on a methodical investigation into differences in face recognition accuracy between African-American and Caucasian image cohorts of the MORPH dataset. We find that, for all four matchers considered, the impostor and the genuine distributions are statistically significantly different between cohorts. For a fixed decision threshold, the African-American image cohort has a higher false match rate and a lower false non-match rate. ROC curves compare verification rates at the same false match rate, but the different cohorts achieve the same false match rate at different thresholds. This means that ROC comparisons are not relevant to operational scenarios that use a fixed decision threshold. We show that, for the ResNet matcher, the two cohorts have approximately equal separation of impostor and genuine distributions. Using ICAO compliance as a standard of image quality, we find that the initial image cohorts have unequal rates of good quality images. The ICAO-compliant subsets of the original image cohorts show improved accuracy, with the main effect being to reducing the low-similarity tail of the genuine distributions.

View PDFchevron_right
Improving Face Authenticaion using Virtual Samples
Norman Poh

IEEE International Conference on …, 2003

In this paper, we present a simple yet effective way to improve a face verification system by generating multiple virtual samples from the unique image corresponding to an access request. These images are generated using simple geometric transformations. This method is often used during training to improve accuracy of a neural network model by making it robust against minor translation, scale and orientation change. The main contribution of this paper is to introduce such method during testing. By generating AE images from one single image and propagating them to a trained network model, one obtains AE scores. By merging these scores using a simple mean operator, we show that the variance of merged scores is decreased by a factor between 1 and AE. An experiment is carried out on the XM2VTS database which achieves new state-of-the-art performances.

View PDFchevron_right

References (45)

  1. S. Banerjee, J. Brogan, J. Krizaj, et al. To frontal- ize or not to frontalize: Do we really need elabo- rate pre-processing to improve face recognition? In Winter Conference on Applications of Com- puter Vision (WACV), pages 20-29, March 2018.
  2. J. Buolamwini and T. Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fair- ness, Accountability and Transparency, volume 81, pages 77-91. PMLR, 2018.
  3. F. Casado, C. Regueiro, R. Iglesias, X. Pardo, and E. López. Automatic selection of user samples for a non-collaborative face verification system. In ROBOT 2017: Third Iberian Robotics Conference, pages 555-566. Springer, 2018.
  4. D. Chen, X. Cao, L. Wang, F. Wen, and J. Sun. Bayesian face revisited: A joint formulation. In A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, and C. Schmid, editors, Computer Vision -ECCV 2012, pages 566-579, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg.
  5. D. Chen, X. Cao, D. Wipf, F. Wen, and J. Sun. An efficient joint formulation for bayesian face ver- ification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(1):32-46, Jan 2017.
  6. D. Crispell, O. Biris, N. Crosswhite, J. Byrne, and J. Mundy. Dataset augmentation for pose and lighting invariant face recognition. CoRR, abs/1704.04326, 2017.
  7. J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition (CVPR), pages 248-255. IEEE, 2009.
  8. B. Fernando, A. Habrard, M. Sebban, and T. Tuytelaars. Unsupervised visual domain adap- tation using subspace alignment. In International Conference on Computer Vision (ICCV), pages 2960-2967, 2013.
  9. T. Hassner. Viewing real-world faces in 3d. In International Conference on Computer Vision (ICCV), pages 3607-3614, Dec 2013.
  10. T. Hassner, S. Harel, E. Paz, and R. Enbar. Effec- tive face frontalization in unconstrained images. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4295-4304, June 2015.
  11. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 770-778, June 2016.
  12. G. Hu, X. Peng, Y. Yang, T. M. Hospedales, and J. Verbeek. Frankenstein: Learning deep face rep- resentations using small data. IEEE Transactions on Image Processing, 27(1):293-303, Jan 2018.
  13. G. Huang, M. Ramesh, T. Berg, and E. Learned- Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained envi- ronments. Technical report, Technical Report 07- 49, University of Massachusetts, Amherst, 2007.
  14. V. Kazemi and J. Sullivan. One millisecond face alignment with an ensemble of regression trees. In Computer Vision and Pattern Recogni- tion (CVPR), pages 1867-1874, 2014.
  15. A. Khosla, T. Zhou, T. Malisiewicz, A. Efros, and A. Torralba. Undoing the damage of dataset bias. In European Conference on Computer Vision (ECCV), pages 158-171. Springer, 2012.
  16. D. King. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10(Jul):1755-1758, 2009.
  17. B. F. Klare, B. Klein, E. Taborsky, et al. Pushing the frontiers of unconstrained face detection and recognition: Iarpa janus benchmark a. In Com- puter Vision and Pattern Recognition (CVPR), pages 1931-1939, 2015.
  18. A. Kortylewski, B. Egger, A. Schneider, T. Gerig, A. Morel-Forster, and T. Vetter. Empirically an- alyzing the effect of dataset biases on deep face recognition systems. In 2018 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2174-217409, June 2018.
  19. Y. LeCun, Y. Bengio, and G. Hinton. Deep learn- ing. Nature, 521(7553):436, 2015.
  20. Z. Lei, M. Pietikäinen, and S. Z. Li. Learn- ing discriminant face descriptor. IEEE Trans- actions on Pattern Analysis and Machine Intel- ligence, 36(2):289-302, Feb 2014.
  21. Z. Luo, J. Hu, W. Deng, and H. Shen. Deep unsupervised domain adaptation for face recogni- tion. In International Conference on Automatic Face Gesture Recognition (FG 2018), pages 453- 457, May 2018.
  22. L. v. d. Maaten and G. Hinton. Visualizing data using t-sne. Journal of Machine Learning Re- search, 9(Nov):2579-2605, 2008.
  23. U. Mahbub, S. Sarkar, V. M. Patel, and R. Chel- lappa. Active user authentication for smart- phones: A challenge data set and benchmark re- sults. In International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1-8, Sept 2016.
  24. N. C. Mithun, R. Panda, and A. K. Roy- Chowdhury. Generating diverse image datasets with limited labeling. In Proceedings of the 24th ACM International Conference on Multimedia, MM '16, pages 566-570, New York, NY, USA, 2016. ACM.
  25. H. Ng and S. Winkler. A data-driven approach to cleaning large face datasets. In International Con- ference on Image Processing (ICIP), pages 343- 347, Oct 2014.
  26. T. Ojala, M. Pietikainen, and T. Maenpaa. Mul- tiresolution gray-scale and rotation invariant tex- ture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 24(7):971-987, 2002.
  27. M. Parchami, S. Bashbaghi, and E. Granger. Video-based face recognition using ensemble of haar-like deep convolutional neural networks. In International Joint Conference on Neural Net- works (IJCNN), pages 4625-4632, 2017.
  28. O. M. Parkhi, A. Vedaldi, and A. Zisserman. Deep face recognition. In M. W. J. Xianghua Xie and G. K. L. Tam, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 41.1- 41.12. BMVA Press, September 2015.
  29. P. Phillips, H. Wechsler, J. Huang, and P. J. Rauss. The feret database and evaluation pro- cedure for face-recognition algorithms. Image and Vision Computing, 16(5):295-306, 1998.
  30. A. Ratner, H. Ehrenberg, Z. Hussain, J. Dunn- mon, and C. Ré. Learning to compose domain- specific transformations for data augmentation. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, ed- itors, Advances in Neural Information Processing Systems, volume 30, pages 3236-3246. Curran As- sociates, Inc., 2017.
  31. J. Ren, X. Jiang, and J. Yuan. A complete and fully automated face verification system on mobile devices. Pattern Recognition, 46(1):45 -56, 2013.
  32. K. Saenko, B. Kulis, M. Fritz, and T. Dar- rell. Adapting visual category models to new do- mains. In European Conference on Computer Vi- sion (ECCV), pages 213-226. Springer, 2010.
  33. F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. In 2015 IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 815-823, June 2015.
  34. A. F. Sequeira, J. C. Monteiro, A. Rebelo, and H. P. Oliveira. Mobbio: A multimodal database captured with a portable handheld device. In Con- ference on Computer Vision Theory and Applica- tions (VISAPP), volume 3, pages 133-139, Jan 2014.
  35. K. Sohn, S. Liu, G. Zhong, X. Yu, M. Yang, and M. Chandraker. Unsupervised domain adaptation for face recognition in unlabeled videos. In Inter- national Conference on Computer Vision (ICCV), pages 5917-5925, Oct 2017.
  36. B. Sun and K. Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Euro- pean Conference on Computer Vision Workshops (ECCVW), pages 443-450, Cham, 2016. Springer International Publishing.
  37. Y. Sun, X. Wang, and X. Tang. Deeply learned face representations are sparse, selective, and ro- bust. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2892-2900, June 2015.
  38. T. Tommasi, N. Patricia, B. Caputo, and T. Tuytelaars. A deeper look at dataset bias. In Domain Adaptation in Computer Vision Applica- tions, pages 37-55. Springer, 2017.
  39. A. Torralba and A. A. Efros. Unbiased look at dataset bias. In Computer Vision and Pat- tern Recognition (CVPR), pages 1521-1528, June 2011.
  40. E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell. Adversarial discriminative domain adaptation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2962-2971, July 2017.
  41. P. Viola and M. Jones. Rapid object detection us- ing a boosted cascade of simple features. In Com- puter Vision and Pattern Recognition (CVPR), volume 1, pages I-I. IEEE, 2001.
  42. G. Wen, H. Chen, D. Cai, and X. He. Improving face recognition with domain adaptation. Neuro- computing, 287(C):45-51, 2018.
  43. C. Whitelam, E. Taborsky, A. Blanton, et al. Iarpa janus benchmark-b face dataset. In Com- puter Vision and Pattern Recognition Workshops (CVPRW), pages 592-600, July 2017.
  44. X. Xing, G. Xu, B. Cai, C. Qing, and X. Xu. Face verification based on feature transfer via pca-svm framework. In 2017 IEEE International Confer- ence on Internet of Things (iThings) and IEEE Green Computing and Communications (Green- Com) and IEEE Cyber, Physical and Social Com- puting (CPSCom) and IEEE Smart Data (Smart- Data), pages 1086-1091, 2017.
  45. H. Xu, J. Zheng, A. Alavi, and R. Chellappa. Cross-domain visual recognition via domain adap- tive dictionary learning. CoRR, abs/1804.04687, 2018.

Related papers

A Comparison of Human and Automated Face Verification Accuracy on Unconstrained Image Sets
Kristen Allen

Automatic face recognition technologies have seen significant improvements in performance due to a combination of advances in deep learning and availability of larger datasets for training deep networks. Since recognizing faces is a task humans are believed to excel at, we use human performance on fully unconstrained facial imagery as a baseline against which to compare the relative performance of automated face recognition. In this work, we expand on previous studies of the recognition accuracy of humans and automated systems by performing several novel analyses utilizing unconstrained face imagery. We examine the impact on performance when human recognizers are presented with varying amounts of imagery per subject, immutable attributes such as gender, and circumstantial attributes such as occlusion, illumination, and pose. Results indicate that humans greatly outperform state-of-the-art automated face recognition algorithms on the challenging IJB-A dataset.

View PDFchevron_right
Fusing Face-Verification Algorithms and Humans
Alice O'Toole

IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), 2000

It has been demonstrated recently that state-of-theart face-recognition algorithms can surpass human accuracy at matching faces over changes in illumination. The ranking of algorithms and humans by accuracy, however, does not provide information about whether algorithms and humans perform the task comparably or whether algorithms and humans can be fused to improve performance. In this paper, we fused humans and algorithms using partial least square regression (PLSR). In the first experiment, we applied PLSR to face-pair similarity scores generated by seven algorithms participating in the Face Recognition Grand Challenge. The PLSR produced an optimal weighting of the similarity scores, which we tested for generality with a jackknife procedure. Fusing the algorithms' similarity scores using the optimal weights produced a twofold reduction of error rate over the most accurate algorithm. Next, human-subject-generated similarity scores were added to the PLSR analysis. Fusing humans and algorithms increased the performance to near-perfect classification accuracy. These results are discussed in terms of maximizing face-verification accuracy with hybrid systems consisting of multiple algorithms and humans.

View PDFchevron_right
SDFR: Synthetic Data for Face Recognition Competition
Bernardo Biesseck

arXiv (Cornell University), 2024

Large-scale face recognition datasets are collected by crawling the Internet and without individuals' consent, raising legal, ethical, and privacy concerns. With the recent advances in generative models, recently several works proposed generating synthetic face recognition datasets to mitigate concerns in web-crawled face recognition datasets. This paper presents the summary of the Synthetic Data for Face Recognition (SDFR) Competition held in conjunction with the 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2024) and established to investigate the use of synthetic data for training face recognition models. The SDFR competition was split into two tasks, allowing participants to train face recognition systems using new synthetic datasets and/or existing ones. In the first task, the face recognition backbone was fixed and the dataset size was limited, while the second task provided almost complete freedom on the model backbone, the dataset, and the training pipeline. The submitted models were trained on existing and also new synthetic datasets and used clever methods to improve training with synthetic data. The submissions were evaluated and ranked on a diverse set of seven benchmarking datasets. The paper gives an overview of the submitted face recognition models and reports achieved performance compared to baseline models trained on real and synthetic datasets. Furthermore, the evaluation of submissions is extended to bias assessment across different demography groups. Lastly, an outlook on the current state of the research in training face recognition models using synthetic data is presented, and existing problems as well as potential future directions are also discussed.

View PDFchevron_right
Face Verification Competition on the XM2VTS Database
Farzin Deravi

In the year 2000 a competition was organised to collect face verification results on an identical, publicly available data set using a standard evaluation protocol. The database used was the Xm2vts database along with the Lausanne protocol [14]. Four different institutions submitted results on the database which were subsequently published in [13]. Three years later, a second contest using the same dataset and protocol was organised as part of AVBPA 2003. This time round seven seperate institutions submitted results to the competition. This paper presents the results of the competition and shows that verification results on this protocol have increased in performance by a factor of 3.

View PDFchevron_right
Distill and De-bias: Mitigating Bias in Face Verification using Knowledge Distillation
Aniket Roy

arXiv (Cornell University), 2021

Face recognition networks generally demonstrate bias with respect to sensitive attributes like gender, skintone etc. For gender and skintone, we observe that the regions of the face that a network attends to vary by the category of an attribute. This might contribute to bias. Building on this intuition, we propose a novel distillation-based approach called Distill and De-bias (D&D) to enforce a network to attend to similar face regions, irrespective of the attribute category. In D&D, we train a teacher network on images from one category of an attribute; e.g. light skintone. Then distilling information from the teacher, we train a student network on images of the remaining category; e.g., dark skintone. A feature-level distillation loss constrains the student network to generate teacher-like representations. This allows the student network to attend to similar face regions for all attribute categories and enables it to reduce bias. We also propose a second distillation step on top of D&D, called D&D++. Here, we distill the 'un-biasedness' of the D&D network into a new student network, the D&D++ network, while training this new network on all attribute categories; e.g., both light and dark skintones. This helps us train a network that is less biased for an attribute, while obtaining higher face verification performance than D&D. We show that D&D++ outperforms existing baselines in reducing gender and skintone bias on the IJB-C dataset, while obtaining higher face verification performance than existing adversarial de-biasing methods. We evaluate the effectiveness of our proposed methods on two state-of-the-art face recognition networks: ArcFace and Crystalface.

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence and Mach...
  • Facial Images
  • Face Verification
  • Source Domain

    • Cited by

    Incremental Learning from Low-labelled Stream Data in Open-Set Video Face Recognition
    Xosé Pardo

    Pattern Recognition

    Deep Learning approaches have brought solutions, with impressive performance, to general classification problems where wealthy of annotated data are provided for training. In contrast, less progress has been made in continual learning of a set of non-stationary classes, mainly when applied to unsupervised problems with streaming data. Here, we propose a novel incremental learning approach which combines a deep features encoder with an Open-Set Dynamic Ensembles of SVM, to tackle the problem of identifying individuals of interest (IoI) from streaming face data. From a simple weak classifier trained on a few video-frames, our method can use unsupervised operational data to enhance recognition. Our approach adapts to new patterns avoiding catastrophic forgetting and partially heals itself from miss-adaptation. Besides, to better comply with real world conditions, the system was designed to operate in an open-set setting. Results show a benefit of up to 15% F1-score increase respect to non-adaptive state-of-theart methods.

    View PDFchevron_right
    Cross-sensor periocular biometrics in a global pandemic: Comparative benchmark and novel multialgorithmic approach
    Fernando Alonso-Fernandez

    Information Fusion, 2022

    The massive availability of cameras and personal devices results in a wide variability between imaging conditions, producing large intra-class variations and a significant performance drop if images from heterogeneous environments are compared for person recognition purposes. However, as biometric solutions are extensively deployed,

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved