Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Limitations in Operationalizing Race
Critical Race Methodologies
Disaggregated Analysis
Conclusion
References
All Topics
Sociology

Towards a critical race methodology in algorithmic fairness

Profile image of Jamila LoudJamila Loud

2020, Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

https://doi.org/10.1145/3351095.3372826
visibility

…

description

12 pages

link

1 file

Abstract

We examine the way race and racial categories are adopted in algorithmic fairness frameworks. Current methodologies fail to adequately account for the socially constructed nature of race, instead adopting a conceptualization of race as a fixed attribute. Treating race as an attribute, rather than a structural, institutional, and relational phenomenon, can serve to minimize the structural aspects of algorithmic unfairness. In this work, we focus on the history of racial categories and turn to critical race theory and sociological work on race and ethnicity to ground conceptualizations of race for fairness research, drawing on lessons from public health, biomedical research, and social survey research. We argue that algorithmic fairness researchers need to take into account the multidimensionality of race, take seriously the processes of conceptualizing and operationalizing race, focus on social processes which produce racial inequality, and consider perspectives of those most affected by sociotechnical systems.

Related papers

Intersectional Experiences of Unfair Treatment Caused by Automated Computational Systems
Mark Coté

Proceedings of the ACM on Human-Computer Interaction

This paper reports on empirical work conducted to study perceptions of unfair treatment caused by automated computational systems. While the pervasiveness of algorithmic bias has been widely acknowledged, and perceptions of fairness are commonly studied in Human Computer Interaction, there is a lack of research on how unfair treatment by automated computational systems is experienced by users from disadvantaged and marginalised backgrounds. There is a need for more diversification in terms of the investigated users, domains, and tasks, and regarding the strategies that users employ to reduce harm. To unpack these issues, we ran a prescreened survey of 663 participants, oversampling those with at-risk characteristics. We collected occurrences and types of conflicts regarding unfair and discriminatory treatment and systems, as well as the actions taken towards resolving these situations. Drawing on intersectional research, we combine qualitative and quantitative approaches in order to...

View PDFchevron_right
Fairness in Algorithmic Decision Making: An Excursion Through the Lens of Causality
Aria Khademi, Vasant G Honavar

2019

As virtually all aspects of our lives are increasingly impacted by algorithmic decision making systems, it is incumbent upon us as a society to ensure such systems do not become instruments of unfair discrimination on the basis of gender, race, ethnicity, religion, etc. We consider the problem of determining whether the decisions made by such systems are discriminatory, through the lens of causal models. We introduce two definitions of group fairness grounded in causality: fair on average causal effect (FACE), and fair on average causal effect on the treated (FACT). We use the Rubin-Neyman potential outcomes framework for the analysis of cause-effect relationships to robustly estimate FACE and FACT. We demonstrate the effectiveness of our proposed approach on synthetic data. Our analyses of two real-world data sets, the Adult income data set from the UCI repository (with gender as the protected attribute), and the NYC Stop and Frisk data set (with race as the protected attribute), show that the evidence of discrimination obtained by FACE and FACT, or lack thereof, is often in agreement with the findings from other studies. We further show that FACT, being somewhat more nuanced compared to FACE, can yield findings of discrimination that differ from those obtained using FACE.

View PDFchevron_right
AI's fairness problem: understanding wrongful discrimination in the context of automated decision-making
Hugo Cossette-Lefebvre

AI & Ethics, 2022

The use of predictive machine learning algorithms is increasingly common to guide or even take decisions in both public and private settings. Their use is touted by some as a potentially useful method to avoid discriminatory decisions since they are, allegedly, neutral, objective, and can be evaluated in ways no human decisions can. By (fully or partly) outsourcing a decision process to an algorithm, it should allow human organizations to clearly define the parameters of the decision and to, in principle, remove human biases. Yet, in practice, the use of algorithms can still be the source of wrongful discriminatory decisions based on at least three of their features: the data-mining process and the categorizations they rely on can reconduct human biases, their automaticity and predictive design can lead them to rely on wrongful generalizations, and their opaque nature is at odds with democratic requirements. We highlight that the two latter aspects of algorithms and their significance for discrimination are too often overlooked in contemporary literature. Though these problems are not all insurmountable, we argue that it is necessary to clearly define the conditions under which a machine learning decision tool can be used. We identify and propose three main guidelines to properly constrain the deployment of machine learning algorithms in society: algorithms should be vetted to ensure that they do not unduly affect historically marginalized groups; they should not systematically override or replace human decision-making processes; and the decision reached using an algorithm should always be explainable and justifiable.

View PDFchevron_right
An Intersectional Definition of Fairness
rashidul islam

2020 IEEE 36th International Conference on Data Engineering (ICDE), 2020

We propose differential fairness, a multi-attribute definition of fairness in machine learning which is informed by intersectionality, a critical lens arising from the humanities literature, leveraging connections between differential privacy and legal notions of fairness. We show that our criterion behaves sensibly for any subset of the set of protected attributes, and we prove economic, privacy, and generalization guarantees. We provide a learning algorithm which respects our differential fairness criterion. Experiments on the COMPAS criminal recidivism dataset and census data demonstrate the utility of our methods.

View PDFchevron_right
Review of "Algorithms of oppression: how search engines reinforce racism," by Noble, S. U. (2018). New York, New York: NYU Press
Gregory Zobel

Communication Design Quarterly, 2019

Read and considered thoughtfully, Safiya Umoja Noble's Algorithms of Oppression: How Search Engines Reinforce Racism is devastating. It reduces to rubble the notion that technology is neutral and ideology-free. Noble's crushing the neutrality myth does several things. First, this act lays foundations for her argument: only if you recognize and understand that technology is built with, and integrates, bias, can you then be open to her primary thesis: search engines advance discriminatory and often racist content. Second, it banishes a convenient response for many self-identified meritocratic Silicon Valley "winners" and their supporters. Post-reading, some individuals may retain their beliefs in a neutral and ideology-free technology in spite of the overwhelming evidence and citations Noble brings to bear. Effective countering of Noble's claims is unlikely to occur. For professionals working in technology, information, argumentation, and/or rhetorical studies, A...

View PDFchevron_right
Book Review: Algorithms of Oppression: How Search Engines Reinforce Racism
Michael L Rosino

Social Forces, 2019

View PDFchevron_right
Racial categories in machine learning
Bruce D. Haynes

Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019

Controversies around race and machine learning have sparked debate among computer scientists over how to design machine learning systems that guarantee fairness. These debates rarely engage with how racial identity is embedded in our social experience, making for sociological and psychological complexity. This complexity challenges the paradigm of considering fairness to be a formal property of supervised learning with respect to protected personal attributes. Racial identity is not simply a personal subjective quality. For people labeled "Black" it is an ascribed political category that has consequences for social differentiation embedded in systemic patterns of social inequality achieved through both social and spatial segregation. In the United States, racial classification can best be understood as a system of inherently unequal status categories that places whites as the most privileged category while signifying the Negro/black category as stigmatized. Social stigma is reinforced through the unequal distribution of societal rewards and goods along racial lines that is reinforced by state, corporate, and civic institutions and practices. This creates a dilemma for society and designers: be blind to racial group disparities and thereby reify racialized social inequality by no longer measuring systemic inequality, or be conscious of racial categories in a way that itself reifies race. We propose a third option. By preceding group fairness interventions with unsupervised learning to dynamically detect patterns of segregation, machine learning systems can mitigate the root cause of social disparities, social segregation and stratification, without further anchoring status categories of disadvantage. CCS CONCEPTS • Social and professional topics → Race and ethnicity; Systems analysis and design; • Applied computing → Sociology; • Computing methodologies → Dimensionality reduction and manifold learning;

View PDFchevron_right
Racist Algorithms or Systemic Problems? Risk Assessments and Racial Disparities
Gina Vincent

Criminal Justice and Behavior, 2020

As recent and historical events attest, racial and ethnic disparities are widely engrained into the justice system. Recently, scholars and policymakers have raised concerns that risk assessment instruments may exacerbate these disparities. While it is critical that risk instruments be scrutinized for racial bias, some concerns, though well-meaning, have gone beyond the evidence. This article explains what it means for an instrument to be “biased” and why instruments should not all be painted with the same brush (some will be more susceptible to bias than others). If some groups get apprehended more, those groups will score higher on non-biased, well-validated instruments derived to maximize prediction of recidivism because of mathematics. Thus, risk instruments shine a light on long-standing systemic problems of racial disparities. This article concludes with suggestions for research and for minimizing disparities by ensuring that systems use risk assessments to avoid unnecessary in...

View PDFchevron_right
Book Review of Algorithms of Oppression: How Search Engines Reinforce Racism
Yvonne Garrett

2019

View PDFchevron_right
An Epistemic Lens on Algorithmic Fairness
Elizabeth Edenberg

Equity and Access in Algorithms, Mechanisms, and Optimization

In this position paper, we introduce a new epistemic lens for analyzing algorithmic harm. We argue that the epistemic lens we propose herein has two key contributions to help reframe and address some of the assumptions underlying inquiries into algorithmic fairness. First, we argue that using the framework of epistemic injustice helps to identify the root causes of harms currently framed as instances of representational harm. We suggest that the epistemic lens offers a theoretical foundation for expanding approaches to algorithmic fairness in order to address a wider range of harms not recognized by existing technical or legal definitions. Second, we argue that the epistemic lens helps to identify the epistemic goals of inquiries into algorithmic fairness. There are two distinct contexts within which we examine algorithmic harm: at times, we seek to understand and describe the world as it is, and, at other times, we seek to build a more just future. The epistemic lens can serve to direct our attention to the epistemic frameworks that shape our interpretations of the world as it is and the ways we envision possible futures. Clarity with respect to which epistemic context is relevant in a given inquiry can further help inform choices among the different ways of measuring and addressing algorithmic harms. We introduce this framework with the goal of initiating new research directions bridging philosophical, legal, and technical approaches to understanding and mitigating algorithmic harms. CCS CONCEPTS • Computing methodologies → Philosophical/theoretical foundations of artificial intelligence.

View PDFchevron_right

References (125)

  1. J. Khadijah Abdurahman. Fat* be wilin'. 2019.
  2. Syed Mustafa Ali. A brief introduction to decolonial computing. XRDS: Crossroads, The ACM Magazine for Students, 22(4):16-21, 2016.
  3. Walter R Allen, Susan A Suh, Gloria González, and Joshua Yang. Qui bono? explaining-or defending-winners and losers in the competition for educational achievement. In Tukufu Zuberi and Eduardo Bonilla-Silva, editors, White logic, white methods : racism and methodology, chapter 13, pages 217-237. Rowman & Littlefield Publishers, 2008.
  4. Elizabeth Anderson. Toward a non-ideal, relational methodology for political philosophy: Comments on schwartzman's "challenging liberalism". Hypatia, 24(4):130-145, 2009.
  5. McKane Andrus and Thomas K Gilbert. Towards a just theory of measurement: A principled social measurement assurance program for machine learning. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 445-451. ACM, 2019.
  6. Julia Angwin and Jeff Larson. Propublica responds to company's critique of machine bias story. ProPublica, 29, 2016.
  7. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. ProPublica, 2016.
  8. Stanley R. Bailey, Mara Loveman, and Jeronimo O. Muniz. Measures of "race" and the analysis of racial inequality in brazil. Social Science Research, 42(1):106 -119, 2013.
  9. Chelsea Barabas, Karthik Dinakar, Joichi Ito, Madars Virza, and Jonathan Zittrain. Interventions over predictions: Reframing the ethical debate for actuarial risk assessment. arXiv preprint arXiv:1712.08238, 2017.
  10. Jan Baren-Nawrocka. The bioinformatics of genetic origins: how identities become embedded in the tools and practices of bioinformatics. Life Sciences, Society and Policy, 9:7, 09 2013.
  11. Ruha Benjamin. Race After Technology: Abolitionist Tools for the New Jim Code. John Wiley & Sons, 2019.
  12. Sebastian Benthall and Bruce D. Haynes. Racial categories in machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* '19, 2019.
  13. Marianne Bertrand and Sendhil Mullainathan. Are Emily and Greg More Em- ployable Than Lakisha and Jamal? A Field Experiment on Labor Market Dis- crimination. American Economic Review, 94(4):991-1013, September 2004.
  14. Catherine Bliss. Racial taxonomy in genomics. Social Science & Medicine, 73(7):1019 -1027, 2011.
  15. Deborah A Bolnick, Duana Fullwiley, Troy Duster, Richard S Cooper, Joan H Fujimura, Jonathan Kahn, Jay S Kaufman, Jonathan Marks, Ann Morning, Alon- dra Nelson, et al. The science and business of genetic ancestry testing. Science, 318(5849):399-400, 2007.
  16. Eduardo Bonilla-Silva and Tukufu Zuberi. Toward a definition of white logic and white methods. In Tukufu Zuberi and Eduardo Bonilla-Silva, editors, White logic, white methods: racism and methodology, chapter 1, pages 3-27. Rowman & Littlefield Publishers, 2008.
  17. Geoffrey C. Bowker and Susan Leigh Star. Sorting Things Out. MIT Press, 2000.
  18. Jenifer Bratter and Bridget K Gorman. Does multiracial matter? a study of racial disparities in self-rated health. Demography, 48:127-52, 02 2011.
  19. Lundy Braun, Anne Fausto-Sterling, Duana Fullwiley, Evelynn Hammonds, Alondra Nelson, William Quivers, Susan Reverby, and Alexandra E Shields. Racial categories in medical practice: How useful are they? PLoS medicine, 4:e271, 10 2007.
  20. Andre Brock. Life on the wire: Deconstructing race on the internet. Information, Communication & Society, 12(3):344-363, 2009.
  21. Simone Browne. Dark matters: On the surveillance of blackness. Duke Univer- sity Press, 2015.
  22. Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In FAT*, pages 77-91, 2018.
  23. Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5 2:153-163, 2017.
  24. Wendy Hui Kyong Chun. Introduction: Race and/as Technology; or, How to Do Things to Race. Camera Obscura: Feminism, Culture, and Media Studies, 24(1 (70)):7-35, 05 2009.
  25. Patricia Hill Collins. Fighting words: Black women and the search for justice, volume 7. U of Minnesota Press, 1998.
  26. Sam Corbett-Davies and Sharad Goel. The measure and mismeasure of fairness: A critical review of fair machine learning. In arXiv:1808.00023, 2018.
  27. Matthew Desmond and Mustafa Emirbayer. What is racial domination? Du Bois Review: Social Science Research on Race, 6(2):335-355, 2009.
  28. William Dieterich, Christina Mendoza, and Tim Brennan. COMPAS risk scales: Demonstrating accuracy equity and predictive parity. Northpoint Inc, 2016.
  29. Denise J. Drevdahl, Debby A. Philips, and Janette Y. Taylor. Uncontested cat- egories: the use of race and ethnicity variables in nursing research. Nursing Inquiry, 13(1):52-63, 2006.
  30. Troy Duster. Race and reification in science. Science, 307(5712):1050-1051, 2005.
  31. Dave Elder-Vass. Towards a realist social constructionism. Sociologia, problemas e práticas, (70):9-24, 2012.
  32. Diana Elliott, Rob Santos, Steven Martin, and Charmaine Runes. Assessing miscounts in the 2020 census. 2019.
  33. Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '15, pages 259-268, New York, NY, USA, 2015. ACM.
  34. Chandra Ford and Nina Harawa. A new conceptualization of ethnicity for social epidemiologic and health equity research. Social science & Medicine, 71:251-8, 07 2010.
  35. Chandra L. Ford. Public health critical race praxis: An introduction, an inter- vention, and three points for consideration. Wisconsin Law Review, 3:477-491, 01 2016.
  36. Chandra L. Ford and Collins Airhihenbuwa. Critical race theory, race equity, and public health: Toward antiracism praxis. American Journal of Public Health, 100 Suppl 1:S30-5, 02 2010.
  37. Morris W. Foster and Richard R Sharp. Race, ethnicity, and genomics: social classifications as proxies of biological heterogeneity. Genome research, 12 6:844-50, 2002.
  38. Michel Foucault. The order of things. Routledge, 2005.
  39. Marion Fourcade and Kieran Healy. Seeing like a market. Socio-Economic Review, 15(1):9-29, 2016.
  40. Joan H Fujimura, Deborah A Bolnick, Ramya Rajagopalan, Jay S Kaufman, Richard C Lewontin, Troy Duster, Pilar Ossorio, and Jonathan Marks. Clines without classes: How to make sense of human variation. Sociological Theory, 32(3):208-227, 2014.
  41. Mindy Fullilove. Comment: Abandoning "race" as a variable in public health research -an idea whose time has come. American Journal of Public Health, 88:1297-8, 10 1998.
  42. Duana Fullwiley. The molecularization of race: Institutionalizing human differ- ence in pharmacogenetics practice. Science as Culture, 16(1):1-30, 2007.
  43. Duana Fullwiley. The biologistical construction of race. Social studies of science, 38:695-735, 11 2008.
  44. Jennifer Garcìa and Mienah Sharif. Black lives matter: A commentary on racism and public health. American Journal of Public Health, 105:e1-e4, 06 2015.
  45. Gilbert Gee and Devon Payne-Sturges. Environmental health dispari- ties: A framework integrating psychosocial and environmental concepts. Environmental health perspectives, 112:1645-53, 01 2005.
  46. Laura E. Gomez and Nancy Lopez, editors. Mapping Race: Critical Approaches to Health Disparities Research. New Brunswick, NJ, Rutgers University Press, 2013.
  47. Ben Green and Lily Hu. The myth in the methodology: Towards a recontextual- ization of fairness in machine learning. In ICML 2018, 2018.
  48. Ian Hacking. The social construction of what? Harvard university press, 1999.
  49. Nina T Harawa and C. Lawrence Ford. The foundation of modern racial categories and implications for research on black/white disparities in health. Ethnicity & Disease, 19 2:209-17, 2009.
  50. Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in super- vised learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, pages 3323-3331, USA, 2016. Curran Associates Inc.
  51. Alana Helberg-Proctor, Anja Krumeich, Agnes Meershoek, and Klasien Horstman. The multiplicity and situationality of enacting 'ethnicity' in dutch health research articles. BioSocieties, 12 2017.
  52. Michael Herzfeld. The social production of indifference. University of Chicago Press, 1993.
  53. Anna Lauren Hoffmann. Where fairness fails: data, algorithms, and the lim- its of antidiscrimination discourse. Information, Communication & Society, 22(7):900-915, 2019.
  54. Paul W. Holland. Statistics and causal inference. Journal of the American Statistical Association, 81(396):945-960, 1986.
  55. Paul W. Holland. Causation and race. In Tukufu Zuberi and Eduardo Bonilla- Silva, editors, White logic, white methods : racism and methodology, chapter 5, pages 93-109. Rowman & Littlefield Publishers, 2008.
  56. Junia Howell and Michael Emerson. So what "should" we use? evaluating the impact of five racial measures on markers of social inequality. Sociology of Race and Ethnicity, 3:14-30, 5 2017.
  57. Abigail Z. Jacobs and Hanna Wallach. Measurement and fairness. 2019.
  58. Angela James. Making sense of race and racial classification. In Tukufu Zuberi and Eduardo Bonilla-Silva, editors, White logic, white methods : racism and methodology, chapter 2, pages 31-45. Rowman & Littlefield Publishers, 2008.
  59. Jonathan Kahn. Misreading race and genomics after bidil. Nature genetics, 37:655-6, 08 2005.
  60. Judith B. Kaplan and Trude Bennett. Use of Race and Ethnicity in Biomedical Publication. JAMA, 289(20):2709-2716, 2003.
  61. Jay Kaufman and Richard Cooper. Commentary: Considerations for use of racial/ethnic classification in etiologic research. American journal of epidemiology, 154:291-8, 09 2001.
  62. Jay S Kaufman. How inconsistencies in racial classification demystify the race construct in public health statistics. Epidemiology, 10 2:101-3, 1999.
  63. Shannon Kelly and Yashwant Pathak. Race and Ethnicity: Understanding Difference in the Genome Era, pages 71-87. 07 2018.
  64. Ibram X Kendi. Stamped from the beginning: The definitive history of racist ideas in America. Random House, 2017.
  65. Akil Kokayi Khalfani, Tukufu Zuberi, Sulaiman Bah, and Pali J Lehohla. Race and population statistics in south africa. In Tukufu Zuberi and Eduardo Bonilla- Silva, editors, White logic, white methods: racism and methodology, chapter 4, pages 63-92. Rowman & Littlefield Publishers, 2008.
  66. Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade- offs in the fair determination of risk scores. In ITCS, 2017.
  67. Issa Kohler-Hausmann. Eddie Murphy and the Dangers of Counterfactual Causal Thinking About Detecting Racial Discrimination. Northwestern University Law Review, 113(5), 2019.
  68. Gloria Ladson-Billings and William F Tate IV. Toward a critical race theory of education. Teachers College Record, 97:47-68, 1995.
  69. Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. How we analyzed the compas recidivism algorithm. ProPublica, 9, 2016.
  70. Catherine Lee. "race" and "ethnicity" in biomedical research: How do scien- tists construct and explain differences in health? Social Science & Medicine, 68(6):1183 -1190, 2009.
  71. S Lin and J Kelsey. Use of race and ethnicity in epidemiologic research: Concepts, methodological issues, and suggestions for research. Epidemiologic reviews, 22:187-202, 02 2000.
  72. Adam Liptak. Supreme court leaves census question on citizenship in doubt. The New York Times, 2019.
  73. Ian Haney López. White by law: The legal construction of race. NYU Press, 2006.
  74. Mara Loveman, Jeronimo Muniz, and Stanley R. Bailey. Brazil in black and white? race categories, the census, and the study of inequality. Ethnic and Racial Studies, 35, 08 2012.
  75. Neda Maghbouleh. The Limits of Whiteness: Iranian Americans and the Everyday Politics of Race. Stanford University Press, 2017.
  76. Douglas S Massey and Nancy A Denton. American apartheid: Segregation and the making of the underclass. Harvard University Press, 1993.
  77. Vickie Mays, Ninez Ponce, Donna Washington, and Susan Cochran. Classifi- cation of race and ethnicity: Implications for public health. Annual Review of Public Health, 24:83-110, 02 2003.
  78. Amade M'charek. Beyond fact or fiction: On the materiality of race in practice. Cultural Anthropology, 28:420-442, 07 2013.
  79. Katherine McKittrick. Demonic Grounds: Black Women and the Cartographies of Struggle. University of Minnesota Press, ned -new edition edition, 2006.
  80. Charles W Mills. The racial contract. Cornell University Press, 1997.
  81. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasser- man, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In FAT, 2019.
  82. Ellis P Monk Jr. The cost of color: Skin color, discrimination, and health among african-americans. American Journal of Sociology, 121(2):396-444, 2015.
  83. Edward Morris. Researching race: Identifying a social construction through qualitative methods and an interactionist perspective. Symbolic Interaction, 30:409-425, 08 2007.
  84. Karama C. Neal. Use and Misuse of 'Race' in Biomedical Research. Online Journal of Health Ethics, 5(1), 2008.
  85. Safiya Umoja Noble. Algorithms of oppression: How search engines reinforce racism. nyu Press, 2018.
  86. Michael Omi and Howard Winant. Racial formation in the United States. Rout- ledge, 2014.
  87. Michael A Omi. The changing meaning of race. America becoming: Racial trends and their consequences, 1:243-263, 2001.
  88. Peggy Pascoe. Miscegenation law, court cases, and ideologies of "race" in twentieth-century america. The Journal of American History, 83(1):44-69, 1996.
  89. Andrew M Penner and Aliya Saperstein. Disentangling the effects of racial self-identification and classification by others: the case of arrest. Demography, 52(3):1017-1024, 2015.
  90. Elizabeth Phillips, Adebola Odunlami, and Vence Bonham. Mixed race: Un- derstanding difference in the genome era. Social forces; a scientific medium of social study and interpretation, 86:795-820, 01 2008.
  91. Ramya Rajagopalan and Joan H Fujimura. Making history via dna, making dna from history. Genetics and the unsettled past: The collision of DNA, race, and history, page 143, 2012.
  92. Inioluwa Deborah Raji and Joy Buolamwini. Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. In AIES, 2019.
  93. Victor Ray. A theory of racialized organizations. American Sociological Review, 84(1):26-53, 2019.
  94. Rashida Richardson, Jason Schultz, and Kate Crawford. Dirty data, bad predic- tions: How civil rights violations impact police data, predictive policing systems, and justice. New York University Law Review, 2019.
  95. Wendy D Roth. The multiple dimensions of race. Ethnic and Racial Studies, 39(8):1310-1338, 2016.
  96. Wendy D. Roth. Methodological pitfalls of measuring race: international com- parisons and repurposing statistical categories. Ethnic and Racial Studies, 40(13):2347-2353, 2017.
  97. Wendy D Roth. Unsettled identities amid settled classifications? toward sociology of racial appraisals. Ethnic and Racial Studies, 41(6):1093-1112, 2018.
  98. Wendy D. Roth and Biorn Ivemark. Genetic options: The impact of genetic ancestry testing on consumers' racial and ethnic identities. American Journal of Sociology, 124(1):150-184, 2018.
  99. Jacob S Rugh and Douglas S Massey. Racial segregation and the american foreclosure crisis. American sociological review, 75(5):629-651, 2010.
  100. Phia Salter and Glenn Adams. Toward a critical race psychology. Social and Personality Psychology Compass, 7, 11 2013.
  101. Gary D. Sandefur, Mary E. Campbell, and Jennifer Eggerling-Boeck. Racial and ethnic disparities in health and mortality among the u.s. elderly population. In Anderson NB, Bulatao RA, and Cohen B, editors, Critical Perspectives on Racial and Ethnic Differences in Health in Late Life, pages 53-94. Washington, DC: The National Academies Press, 2004.
  102. Aliya Saperstein. Double-checking the race box: Examining inconsistency between survey measures of observed and self-reported race. Social Forces, 85(1):57-74, 2006.
  103. Aliya Saperstein. Capturing complexity in the united states: which aspects of race matter and when? Ethnic and Racial Studies, 35(8):1484-1502, 2012.
  104. Aliya Saperstein, Jessica M Kizer, and Andrew M Penner. Making the most of multiple measures: Disentangling the effects of different dimensions of race in survey research. American Behavioral Scientist, 60(4):519-537, 2016.
  105. Aliya Saperstein and Andrew M Penner. Racial fluidity and inequality in the united states. American Journal of Sociology, 118(3):676-727, 2012.
  106. James C Scott. Seeing like a state: How certain schemes to improve the human condition have failed. Yale University Press, 1998.
  107. James C Scott. Against the grain: a deep history of the earliest states. Yale University Press, 2017.
  108. Andrew D. Selbst, danah boyd, Sorelle A. Friedler, Suresh Venkatasubrama- nian, and Janet Vertesi. Fairness and abstraction in sociotechnical systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* '19, pages 59-68, New York, NY, USA, 2019. ACM.
  109. Maya Sen and Omar Wasow. Race as a bundle of sticks: Designs that estimate effects of seemingly immutable characteristics. Annual Review of Political Science, 19:499-522, 05 2016.
  110. Abigail A. Sewell. The racism-race reification process: A mesolevel political economic framework for understanding racial health disparities. Sociology of Race and Ethnicity, 2(4):402-432, 2016.
  111. Andrew Smart, Richard Tutton, Paul Martin, George T.H. Ellison, and Richard Ashcroft. The standardization of race and ethnicity in biomedical science editorials and uk biobanks. Social Studies of Science, 38(3):407-423, 2008.
  112. C Matthew Snipp. Racial measurement in the american census: Past practices and implications for the future. Annual Review of Sociology, 29(1):563-588, 2003.
  113. Kim TallBear. Native American DNA: Tribal belonging and the false promise of genetic science. U of Minnesota Press, 2013.
  114. Edward Telles, René D. Flores, and Fernando Urrea-Giraldo. Pigmentocracies: Educational inequality, skin color and census ethnoracial identification in eight latin american countries. Research in Social Stratification and Mobility, 40:39 - 58, 2015.
  115. Edward E. Telles and Nelson Lim. Does it matter who answers the race question? racial classification and income inequality in brazil. Demography, 35(4):465-474, 1998.
  116. Charles Tilly. Coercion, Capital, and European States, AD 990-1992. B. Black- well, 1990.
  117. David Williams and Pamela Braboy Jackson. Social sources of racial disparities in health. Health affairs (Project Hope), 24:325-34, 03 2005.
  118. David Williams and Chiquita Collins. Racial residential segregation: A funda- mental cause of racial disparities in health. Public Health Reports, 116:404-416, 09 2001.
  119. David R. Williams. The concept of race in health services research: 1966 to 1990. Health Services Research, 29(3):261-274, 1994.
  120. David R. Williams and Michelle Sternthal. Understanding racial-ethnic dispari- ties in health: sociological contributions. Journal of Health and Social Behavior, 51 Suppl(Suppl):S15-S27, 2010.
  121. Blaise Agüera y Arcas, Margaret Mitchell, and Alexander Todorov. Physiog- nomy's new clothes. Medium, 2017.
  122. Michael Yudell, Dorothy Roberts, Rob DeSalle, and Sarah Tishkoff. Taking race out of human genetics. Science, 351(6273):564-565, 2016.
  123. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. Fairness beyond disparate treatment & disparate impact: Learn- ing classification without disparate mistreatment. In Proceedings of the 26th International Conference on World Wide Web, WWW '17, pages 1171-1180, Republic and Canton of Geneva, Switzerland, 2017. International World Wide Web Conferences Steering Committee.
  124. Tukufu Zuberi. Deracializing social statistics: Problems in the quantification of race. The Annals of the American Academy of Political and Social Science, 568:172-185, 2000.
  125. Tukufu Zuberi. Thicker Than Blood: How Racial Statistics Lie. U of Minnesota Press, 2001.

Related papers

An Empirical Analysis of Racial Categories in the Algorithmic Fairness Literature
Amina Abdu

2023 ACM Conference on Fairness, Accountability, and Transparency

Recent work in algorithmic fairness has highlighted the challenge of defining racial categories for the purposes of anti-discrimination. These challenges are not new but have previously fallen to the state, which enacts race through government statistics, policies, and evidentiary standards in anti-discrimination law. Drawing on the history of state race-making, we examine how longstanding questions about the nature of race and discrimination appear within the algorithmic fairness literature. Through a content analysis of 60 papers published at FAccT between 2018 and 2020, we analyze how race is conceptualized and formalized in algorithmic fairness frameworks. We note that differing notions of race are adopted inconsistently, at times even within a single analysis. We also explore the institutional influences and values associated with these choices. While we find that categories used in algorithmic fairness work often echo legal frameworks, we demonstrate that values from academic computer science play an equally important role in the construction of racial categories. Finally, we examine the reasoning behind different operationalizations of race, finding that few papers explicitly describe their choices and even fewer justify them. We argue that the construction of racial categories is a value-laden process with significant social and political consequences for the project of algorithmic fairness. The widespread lack of justification around the operationalization of race reflects institutional norms that allow these political decisions to remain obscured within the backstage of knowledge production.

View PDFchevron_right
Imagining the diversity algorithm: Approaches to computational injustice and their challenges
Payal Arora

Amsterdam Privacy Conference , 2018

The technology industry is under pressure to diversify their processes and outcomes at a time where decisions are increasingly being made by computing systems in almost all aspects of our lives. Algorithms are of specific interest, given they adjudicate and automate a large array of decisions. In recent years, a number of scholars have pushed for a conceptual shift from platforms as 'neutral' intermediaries to those that are driven by specific ideologies. A growing body of research has embarked on imagining, and suggesting ways in which we can embed socially progressive values into technological design as a corrective for discriminatory practices. However, as the field stands, these ideologies are dispersed and often do not make explicit their operationalization, their value judgments and trade-offs that may ensue if instrumentalized. Therefore, this paper identifies and evaluates three interventionist strategies to tackle the issue of diversity in the context of algorithms, namely: Universalism, Social Justice and Utilitarian driven algorithms. It argues that the notion of diversity can serve as a unifying ontology to unpack these approaches. Bringing together two discourses and fields of study that have not sufficiently intersected-sociology of diversity and internet studies-to arrive at new understandings of the challenges we face in the embedding of diversity as a value in the design of socio-technical systems. As we play these specific diversity imaginaries out, we present possible outcomes, begging the question of the kind of digital future that awaits with each of these ideological turns. Word count:

View PDFchevron_right
Demographic-Reliant Algorithmic Fairness: Characterizing the Risks of Demographic Data Collection in the Pursuit of Fairness
McKane Andrus

2022 ACM Conference on Fairness, Accountability, and Transparency

Most proposed algorithmic fairness techniques require access to demographic data in order to make performance comparisons and standardizations across groups, however this data is largely unavailable in practice, hindering the widespread adoption of algorithmic fairness. Through this paper, we consider calls to collect more data on demographics to enable algorithmic fairness and challenge the notion that discrimination can be overcome with smart enough technical methods and sufficient data. We show how these techniques largely ignore broader questions of data governance and systemic oppression when categorizing individuals for the purpose of fairer algorithmic processing. In this work, we explore under what conditions demographic data should be collected and used to enable algorithmic fairness methods by characterizing a range of social risks to individuals and communities. For the risks to individuals we consider the unique privacy risks of sensitive attributes, the possible harms of miscategorization and misrepresentation, and the use of sensitive data beyond data subjects' expectations. Looking more broadly, the risks to entire groups and communities include the expansion of surveillance infrastructure in the name of fairness, misrepresenting and mischaracterizing what it means to be part of a demographic group, and ceding the ability to define what constitutes biased or unfair treatment. We argue that, by confronting these questions before and during the collection of demographic data, algorithmic fairness methods are more likely to actually mitigate harmful treatment disparities without reinforcing systems of oppression. Towards this end, we assess privacy-focused methods of data collection and use and participatory data governance structures as proposals for more responsibly collecting demographic data.

View PDFchevron_right
Fairness and Abstraction in Sociotechnical Systems
danah boyd

Proceedings of the Conference on Fairness, Accountability, and Transparency

A key goal of the fair-ML community is to develop machine-learning based systems that, once introduced into a social context, can achieve social and legal outcomes such as fairness, justice, and due process. Bedrock concepts in computer science-such as abstraction and modular design-are used to define notions of fairness and discrimination, to produce fairness-aware learning algorithms, and to intervene at different stages of a decision-making pipeline to produce "fair" outcomes. In this paper, however, we contend that these concepts render technical interventions ineffective, inaccurate, and sometimes dangerously misguided when they enter the societal context that surrounds decision-making systems. We outline this mismatch with five "traps" that fair-ML work can fall into even as it attempts to be more context-aware in comparison to traditional data science. We draw on studies of sociotechnical systems in Science and Technology Studies to explain why such traps occur and how to avoid them. Finally, we suggest ways in which technical designers can mitigate the traps through a refocusing of design in terms of process rather than solutions, and by drawing abstraction boundaries to include social actors rather than purely technical ones. CCS CONCEPTS • Applied computing → Law, social and behavioral sciences; • Computing methodologies → Machine learning;

View PDFchevron_right
Tackling Racial Bias in AI Systems: Applying the Bioethical Principle of Justice and Insights from Joy Buolamwini's "Coded Bias" and the "Algorithmic Justice League"
Etaoghene P A U L Polo

Bangladesh Journal of Bioethics, 2025

This paper examines the issue of racial bias in artificial intelligence (AI) through the lens of the bioethical principle of justice, with a focus on Joy Buolamwini's Coded Bias and the work of the Algorithmic Justice League. AI technologies, particularly facial recognition systems, have been shown to disproportionately misidentify individuals from marginalised racial groups, raising profound ethical concerns about fairness and equity. The bioethical principle of justice stresses the importance of equal treatment and protecting vulnerable populations. Through qualitative research, including content analysis of Buolamwini's works and case studies of AI bias, this paper assesses the efforts of the Algorithmic Justice League to combat racial bias in AI. It emphasises their advocacy for developing fair, equitable algorithms and calls for systemic reform in AI development to ensure justice for marginalised communities.

View PDFchevron_right

Related topics

  • Sociology
  • Computer Science

    • Cited by

    Recommenders with a Mission
    Natali Helberger

    Proceedings of the 2021 Conference on Human Information Interaction and Retrieval

    News recommenders help users to find relevant online content and have the potential to fulfill a crucial role in a democratic society, directing the scarce attention of citizens towards the information that is most important to them. Simultaneously, recent concerns about so-called filter bubbles, misinformation and selective exposure are symptomatic of the disruptive potential of these digital news recommenders. Recommender systems can make or break filter bubbles, and as such can be instrumental in creating either a more closed or a more open internet. Current approaches to evaluating recommender systems are often focused on measuring an increase in user clicks and short-term engagement, rather than measuring the user's longer term interest in diverse and important information. This paper aims to bridge the gap between normative notions of diversity, rooted in democratic theory, and quantitative metrics necessary for evaluating the recommender system. We propose a set of metrics grounded in social science interpretations of diversity and suggest ways for practical implementations.

    View PDFchevron_right
    An Agent-based Model to Evaluate Interventions on Online Dating Platforms to Decrease Racial Homogamy
    Stefania Ionescu

    2021

    Perhaps the most controversial questions in the study of online platforms today surround the extent to which platforms can intervene to reduce the societal ills perpetrated on them. Up for debate is whether there exist any effective and lasting interventions a platform can adopt to address, e.g., online bullying, or if other, more far-reaching change is necessary to address such problems. Empirical work is critical to addressing such questions. But it is also challenging, because it is time-consuming, expensive, and sometimes limited to the questions companies are willing to ask. To help focus and inform this empirical work, we here propose an agent-based modeling (ABM) approach. As an application, we analyze the impact of a set of interventions on a simulated online dating platform on the lack of long-term interracial relationships in an artificial society. In the real world, a lack of interracial relationships are a critical vehicle through which inequality is maintained. Our work...

    View PDFchevron_right
    Algorithmic reparation
    Jenny L Davis

    Big Data & Society

    Machine learning algorithms pervade contemporary society. They are integral to social institutions, inform processes of governance, and animate the mundane technologies of daily life. Consistently, the outcomes of machine learning reflect, reproduce, and amplify structural inequalities. The field of fair machine learning has emerged in response, developing mathematical techniques that increase fairness based on anti-classification, classification parity, and calibration standards. In practice, these computational correctives invariably fall short, operating from an algorithmic idealism that does not, and cannot, address systemic, Intersectional stratifications. Taking present fair machine learning methods as our point of departure, we suggest instead the notion and practice of algorithmic reparation. Rooted in theories of Intersectionality, reparative algorithms name, unmask, and undo allocative and representational harms as they materialize in sociotechnical form. We propose algori...

    View PDFchevron_right
    “Garbage in, garbage out” revisited: What do machine learning application papers report about human-labeled training data?
    aayush shah

    Quantitative Science Studies, 2021

    Supervised machine learning, in which models are automatically derived from labeled training data, is only as good as the quality of that data. This study builds on prior work that investigated to what extent “best practices” around labeling training data were followed in applied ML publications within a single domain (social media platforms). In this paper, we expand by studying publications that apply supervised ML in a far broader spectrum of disciplines, focusing on human-labeled data. We report to what extent a random sample of ML application papers across disciplines give specific details about whether best practices were followed, while acknowledging that a greater range of application fields necessarily produces greater diversity of labeling and annotation methods. Because much of machine learning research and education only focuses on what is done once a “ground truth” or “gold standard” of training data is available, it is especially relevant to discuss issues around the e...

    View PDFchevron_right
    The Use and Misuse of Counterfactuals in Ethical Machine Learning
    Andrew Smart

    Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021

    The use of counterfactuals for considerations of algorithmic fairness and explainability is gaining prominence within the machine learning community and industry. This paper argues for more caution with the use of counterfactuals when the facts to be considered are social categories such as race or gender. We review a broad body of papers from philosophy and social sciences on social ontology and the semantics of counterfactuals, and we conclude that the counterfactual approach in machine learning fairness and social explainability can require an incoherent theory of what social categories are. Our findings suggest that most often the social categories may not admit counterfactual manipulation, and hence may not appropriately satisfy the demands for evaluating the truth or falsity of counterfactuals. This is important because the widespread use of counterfactuals in machine learning can lead to misleading results when applied in high-stakes domains. Accordingly, we argue that even though counterfactuals play an essential part in some causal inferences, their use for questions of algorithmic fairness and social explanations can create more problems than they resolve. Our positive result is a set of tenets about using counterfactuals for fairness and explanations in machine learning. CCS CONCEPTS • Computing methodologies → Philosophical/theoretical foundations of artificial intelligence; Machine learning; • Social and professional topics → Socio-technical systems; Race and ethnicity.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved