Academia.eduAcademia.edu

Outline

Title
Abstract
Key Takeaways
Figures
Introduction
Conclusion
References
All Topics
Philosophy
Political Philosophy

Artificial Intelligence in a Structurally Unjust Society

Profile image of Ting-An LinTing-An Lin

2022, Feminist Philosophy Quarterly

visibility

…

description

32 pages

link

1 file

Abstract

Increasing concerns have been raised regarding artificial intelligence (AI) bias, and in response, efforts have been made to pursue AI fairness. In this paper, we argue that the idea of structural injustice serves as a helpful framework for clarifying the ethical concerns surrounding AI bias-including the nature of its moral problem and the responsibility for addressing it-and reconceptualizing the approach to pursuing AI fairness. Using AI in health care as a case study, we argue that AI bias is a form of structural injustice that exists when AI systems interact with other social factors to exacerbate existing social inequalities, making some groups of people more vulnerable to undeserved burdens while conferring unearned benefits to others. The goal of AI fairness, understood this way, is to pursue a more just social structure with the development and use of AI systems when appropriate. We further argue that all participating agents in the unjust social structure associated with AI bias bear a shared responsibility to join collective action with the goal of reforming the social structure, and we provide a list of practical recommendations for agents in various social positions to contribute to this collective action.

Key takeaways
sparkles

AI

  1. AI bias exemplifies structural injustice, heightening social inequalities in healthcare and other sectors.
  2. AI fairness should aim to create a just social structure, not merely ensure statistical parity among groups.
  3. A diverse range of societal agents shares responsibility for addressing and reforming AI bias.
  4. The paper provides actionable recommendations for AI fairness throughout the AI development process.
  5. The development of AI systems requires careful consideration of social context to mitigate existing disparities.
Figures (1)
For the first stage—problem selection—we provide three recommendations that emphasize paying attention to decisions that are rarely made with transparency. First, we suggest that in deciding whether to develop an Al system for a certain use, efforts should be made to assess relevant social contexts and existing social inequalities in order to identify the potential risks and benefits that such a system may bring about. Empirical work in the social sciences and other areas could play a crucial role in these efforts (Joyce et al. 2021), for example, by noting the unexplained
For the first stage—problem selection—we provide three recommendations that emphasize paying attention to decisions that are rarely made with transparency. First, we suggest that in deciding whether to develop an Al system for a certain use, efforts should be made to assess relevant social contexts and existing social inequalities in order to identify the potential risks and benefits that such a system may bring about. Empirical work in the social sciences and other areas could play a crucial role in these efforts (Joyce et al. 2021), for example, by noting the unexplained

Related papers

From Bias to Fairness A Review of Ethical Considerations and Mitigation Strategies in Artificial Intelligence
IJRASET Publication

International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2023

Artificial intelligence (AI) has become increasingly popular in recent years and has been used in a range of industries to improve outcomes, streamline processes, and improve decision-making. But there are also moral questions raised by the employment of AI, particularly in light of potential bias and discrimination. In order to promote justice and reduce bias, this paper offers a thorough discussion of ethical issues and mitigation techniques in AI.

View PDFchevron_right
Epistemic Injustice as a Philosophical Conception for Considering Fairness and Diversity in Human-centered AI Principles
Mariko NIHEI

Interdisciplinary Information Sciences

The sheer quantity of information in the modern world has increased significantly, which exceeds the volume that can be managed using human power. Although information is necessary for decision making, excessive information is not beneficial for proper decision making. Therefore, data mining conducted using machine learning and artificial intelligence (AI)-assisted decision-making systems are increasingly being used in our society. However, problems, such as discriminatory decisions and the promulgation of injustice by AI, have been exposed recently. In response to this, numerous countries and organizations have recently announced a set of AI principles based on the concept of human-centered AI that fosters human values. The principles call for understanding diversity, ensuring fairness, and eliminating discrimination in the use of AI. To implement these values in AI systems, having a philosophical understanding of the structure of injustice in human knowledge production is essential. The problems of injustice and discrimination in knowledge production have recently been categorized as ''epistemic injustice'' in philosophy and epistemology, and the theories explaining these phenomena are becoming more sophisticated. This paper aims to contribute to the understanding of ''human-centered'' AI by connecting the philosophical concept of ''epistemic injustice'' to the discussion of AI ethical principles. It further points out that the issue of injustice and unfairness in AI use is not only a social-ethical as well as an epistemic concern.

View PDFchevron_right
Artificial Intelligence and Bias: Challenges, Implications, and Remedies
Alfonso Min, PhD

2023

This paper investigates the multifaceted issue of algorithmic bias in artificial intelligence (AI) systems and explores its ethical and human rights implications. The study encompasses a comprehensive analysis of AI bias, its causes, and potential remedies, with a particular focus on its impact on individuals and marginalized communities.

View PDFchevron_right
“Democratizing AI” and the Concern of Algorithmic Injustice
Ting-An Lin

Philosophy & Technology, 2024

The call to make artificial intelligence (AI) more democratic, or to “democratize AI,” is sometimes framed as a promising response for mitigating algorithmic injustice or making AI more aligned with social justice. However, the notion of “democratizing AI” is elusive, as the phrase has been associated with multiple meanings and practices, and the extent to which it may help mitigate algorithmic injustice is still underexplored. In this paper, based on a socio-technical understanding of algorithmic injustice, I examine three notable notions of democratizing AI and their associated measures—democratizing AI use, democratizing AI development, and democratizing AI governance—regarding their respective prospects and limits in response to algorithmic injustice. My examinations reveal that while some versions of democratizing AI bear the prospect of mitigating the concern of algorithmic injustice, others are somewhat limited and might even function to perpetuate unjust power hierarchies. This analysis thus urges a more fine-grained discussion on how to democratize AI and suggests that closer scrutiny of the power dynamics embedded in the socio-technical structure can help guide such explorations.

View PDFchevron_right
Why AI Cannot Be an Agent of Equality: A Critique of Technological Utopianism and the Category Error of Algorithmic Justice
Amitabh V I K R A M Dwivedi

This paper presents a critique of the argument that artificial intelligence (AI) has the potential to overcome social hierarchies and achieve equality. It contends that this position is misguided insofar as it commits a category error by confusing epistemic automatism with moral agency. This study challenges the separation of instrumental and moral rationality, as well as formal and substantive equality, to demonstrate how, despite popular imagination and representations of neutrality, AI functions to reify existing hierarchical systems while obscuring the range of human choices made about its design and deployment. The idea of AI as a leveler is a secular feat of metaphysical illusion in the tradition that Nietzsche criticized: projecting justice onto a bodiless future of technology. The paper argues that algorithmic optimization cannot replace normative judgment, ethical deliberation, and political action, which are necessary for true equality.

View PDFchevron_right
ARTIFICIAL INTELLIGENCE AND THE SOCIETY: NAVIGATING THE IMPACT OF AI ON JUSTICE, FREEDOM AND HUMAN RIGHTS
Uchechukwu K I Z I T O Ogu

Nnadiebube Journal of Philosophy (NJP) , 2024

In an era defined by rapid technological advancement, the transformative potential of Artificial Intelligence (AI) stands both as a beacon of hope, promise and a challenge. It has become increasingly evident that AI is not just a tool for innovation but, when deployed responsibly and ethically, can be a force for social innovation and change-improving equity and human rights, shaping societies and redefining the global landscape. The potential of AI to impact society has been noted by scholars, while it has been touted as providing solutions to many social problems, it also has the potential to perpetuate existing inequalities and injustices. It has been shown to replicate, amplify and even exacerbate biases related to race, gender and other social factors. It is, therefore, essential to consider the implication of AI development on issues like justice, human dignity and human rights in order to ensure that AI benefits equitably across populations and demographics without amplifying the already existing inequalities. This paper, seeks therefore, to examine the impact of AI on justice, human freedom and human rights especially those rights that are most impacted or most likely to be impacted by AI. The paper posits that AI can impact social justice through biased decision making, job displacement and so on. It therefore, recommends increased transparency and accountability in AI decision-making so that its benefits can be shared fairly and equitably across society.

View PDFchevron_right
Ethical Redress of Racial Inequities in AI: Lessons from Decoupling Machine Learning from Optimization in Medical Appointment Scheduling
Robert Shanklin

Philosophy & Technology

An Artificial Intelligence algorithm trained on data that reflect racial biases may yield racially biased outputs, even if the algorithm on its own is unbiased. For example, algorithms used to schedule medical appointments in the USA predict that Black patients are at a higher risk of no-show than non-Black patients, though technically accurate given existing data that prediction results in Black patients being overwhelmingly scheduled in appointment slots that cause longer wait times than non-Black patients. This perpetuates racial inequity, in this case lesser access to medical care. This gives rise to one type of Accuracy-Fairness trade-off: preserve the efficiency offered by using AI to schedule appointments or discard that efficiency in order to avoid perpetuating ethno-racial disparities. Similar trade-offs arise in a range of AI applications including others in medicine, as well as in education, judicial systems, and public security, among others. This article presents a fram...

View PDFchevron_right
Bridging Sociology and Technology for Fair Economic Outcomes in AI
Constantinos Challoumis

If AI has the potential to be quite lucrative. AI has now been deployed for automated legal assistance, nursing bot services, and predictive policing. Dozens of tech companies are developing AI for prison and probation systems, jails, and immigration enforcement. While bringing efficiency and technological innovation to economic sectors, there is concern that the application of IFAIs will exacerbate socioeconomic disparities and encroach on labor rights. Socioeconomic disparities ensured that Black and Hispanic communities of hospital employees bore the largest burden of COVID-19 infections in the earliest days of the pandemic, even as they were praised as heroes.

View PDFchevron_right
Artificial intelligence and algorithmic bias: implications for health systems
Heather Mattie

Journal of Global Health, 2019

View PDFchevron_right
AI as moral cover: How algorithmic bias exploits psychological mechanisms to perpetuate social inequality
Islam Borinca

Analyses of social issues and public policy, 2025

Algorithmic decision-making systems are increasingly shaping critical social outcomes (e.g., hiring, lending, criminal justice, healthcare), yet technical approaches to bias mitigation ignore crucial psychological mechanisms that enable discriminatory use. To address this gap, I integrate motivated reasoning, system justification, and moral disengagement theories to argue that AI systems may function as "moral cover," allowing users to perpetuate inequality while maintaining beliefs in their own objectivity. Users often demonstrate "selective adherence," following algorithmic advice when it confirms stereotypes while dismissing counter-stereotypical outputs. System justification motives lead people to defend discriminatory algorithmic outcomes as legitimate, "data-driven" decisions. Moral disengagement mechanisms (including responsibility displacement, euphemistic labeling, and advantageous comparison) can enable discrimination while preserving moral self-regard. Finally, I argue that understanding AI bias as fundamentally psychological rather than merely technical demands interventions addressing these underlying psychological processes alongside algorithmic improvements.

View PDFchevron_right

References (68)

  1. Abràmoff, Michael D., Philip T. Lavin, Michele Birch, Nilay Shah, and James C. Folk. 2018. "Pivotal Trial of an Autonomous AI-Based Diagnostic System for Detection of Diabetic Retinopathy in Primary Care Offices." npj Digital Medicine 1 (August): 39. https://doi.org/10.1038/s41746-018-0040-6.
  2. Adamson, Adewole S., and Avery Smith. 2018. "Machine Learning and Health Care Disparities in Dermatology." JAMA Dermatology 154, no. 11 (November): 1247-48. https://doi.org/10.1001/jamadermatol.2018.2348.
  3. Allen, K. D., C. G. Helmick, T. A. Schwartz, R. F. DeVellis, J. B. Renner, and J. M. Jordan. 2009. "Racial Differences in Self-Reported Pain and Function among Individuals with Radiographic Hip and Knee Osteoarthritis: The Johnston County Osteoarthritis Project." Osteoarthritis and Cartilage 17, no. 9 (September): 1132-36. https://doi.org/10.1016/j.joca.2009.03.003.
  4. Anderson, Elizabeth. 1995. "Knowledge, Human Interests, and Objectivity in Feminist Epistemology." Philosophical Topics 23, no. 2 (Fall): 27-58.
  5. Angwin, Julia, Jeff Larson, Lauren Kirchner, and Surya Mattu. 2016. "Machine Bias." ProPublica, May 23, 2016. https://www.propublica.org/article/machine-bias- risk-assessments-in-criminal-sentencing.
  6. Aragon, Corwin, and Alison M. Jaggar. 2018. "Agency, Complicity, and the Responsibility to Resist Structural Injustice." Journal of Social Philosophy 49, no. 3 (Fall): 439-60. https://doi.org/10.1111/josp.12251.
  7. Barocas, Solon, and Andrew D. Selbst. 2016. "Big Data's Disparate Impact." California Law Review 104, no. 3 (June): 671-732. https://doi.org/10.15779/Z38BG31.
  8. Chen, Irene Y., Shalmali Joshi, and Marzyeh Ghassemi. 2020. "Treating Health Disparities with Artificial Intelligence." Nature Medicine 26, no. 1 (January): 16-17. https://doi.org/10.1038/s41591-019-0649-2.
  9. Chen, Irene Y., Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. 2021. "Ethical Machine Learning in Healthcare." Annual Review of Biomedical Data Science 4:123-44. https://doi.org/10.1146/annu rev-biodatasci-092820-114757.
  10. Chen, Po-Hsuan Cameron, Yun Liu, and Lily Peng. 2019. "How to Develop Machine Learning Models for Healthcare." Nature Materials 18, no. 5 (May): 410-14. https://doi.org/10.1038/s41563-019-0345-0.
  11. Coeckelbergh, Mark. 2020. "Artificial Intelligence, Responsibility Attribution, and a Relational Justification of Explainability." Science and Engineering Ethics 26, no. 4 (August): 2051-68. https://doi.org/10.1007/s11948-019-00146-8.
  12. Collins, Patricia Hill. 1990. Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment. New York: Routledge.
  13. Dahl, Robert A. 1989. Democracy and Its Critics. New Haven, CT: Yale University Press.
  14. Dastin, Jeffrey. 2018. "Amazon Scraps Secret AI Recruiting Tool That Showed Bias against Women." Reuters, October 10, 2018. https://www.reuters.com/arti cle/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G.
  15. Davis, Jenny L., Apryl Williams, and Michael W. Yang. 2021. "Algorithmic Reparation." Big Data and Society 8 (2). https://doi.org/10.1177/20539517211044808.
  16. Eberly, Lauren, Dustin Richter, George Comerci, Justin Ocksrider, Deana Mercer, Gary Mlady, Daniel Wascher, and Robert Schenck. 2018. "Psychosocial and Demographic Factors Influencing Pain Scores of Patients with Knee Osteoarthritis." PloS One 13 (4): e0195075. https://doi.org/10.1371/journal .pone.0195075, Escobar, Oliver, and Stephen Elstub. 2017. "Forms of Mini-Publics: An Introduction to Deliberative Innovations in Democratic Practice." newDemocracy, Research and Development Note 4, May 8, 2017. https://www.newdemocracy.com.au /2017/05/08/forms-of-mini-publics/.
  17. Eveleth, Rose. 2014. "How Self-Tracking Apps Exclude Women." Atlantic, December 15, 2014. https://www.theatlantic.com/technology/archive/2014/12/how-se lf-tracking-apps-exclude-women/383673/.
  18. Fallin-Bennett, Keisa. 2015. "Implicit Bias against Sexual Minorities in Medicine: Cycles of Professional Influence and the Role of the Hidden Curriculum." Academic Medicine: Journal of the Association of American Medical Colleges 90, no. 5 (May): 549-52. https://doi.org/10.1097/ACM.0000000000000662.
  19. Farooq, Faheem, and John J. Strouse. 2018. "Disparities in Foundation and Federal Support and Development of New Therapeutics for Sickle Cell Disease and Cystic Fibrosis." Blood 132, Supplement 1 (November 29): 4687. https://doi .org/10.1182/blood-2018-99-115609.
  20. Fazelpour, Sina, and Zachary C. Lipton. 2020. "Algorithmic Fairness from a Non-ideal Perspective." In AIES '20: Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 57-63. New York: Association for Computing Machinery. https://doi.org/10.1145/3375627.3375828.
  21. FDA (Food and Drug Administration). 2018. "FDA Permits Marketing of Artificial Intelligence-Based Device to Detect Certain Diabetes-Related Eye Problems." FDA new release, April 11, 2018. https://www.fda.gov/news-events/press- announcements/fda-permits-marketing-artificial-intelligence-based-device- detect-certain-diabetes-related-eye.
  22. Fisk, N. M., and R. Atun. 2009. "Systematic Analysis of Research Underfunding in Maternal and Perinatal Health." BJOG: An International Journal of Obstetrics and Gynaecology 116, no. 3 (February): 347-56. https://doi.org/10.1111/j .1471-0528.2008.02027.x, Fraser, Nancy, and Axel Honneth. 2003. Redistribution or Recognition? A Political- Philosophical London: Verso.
  23. Friedler, Sorelle A., Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. "On the (Im)Possibility of Fairness." arXiv preprint, arXiv:1609.07236 [cs.CY]. https://doi.org/10.48550/arXiv.1609.07236.
  24. Friedler, Sorelle A., Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P. Hamilton, and Derek Roth. 2019. "A Comparative Study of Fairness-Enhancing Interventions in Machine Learning." FAT* '19: Proceedings of the Conference on Fairness, Accountability, and Transparency, 329-38. New York: Association for Computing Machinery. https://doi.org/10.1145/328756
  25. Geiger, H. Jack. 2003. "Racial and Ethnic Disparities in Diagnosis and Treatment: A Review of the Evidence and a Consideration of Causes." In Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care, edited by Brian D. Smedley, Adrienne Y. Stith, and Alan R. Nelson, 417-54. Washington, DC: National Academies Press.
  26. Good, Mary-Jo Delvecchio, Cara James, Byron J. Good, and Anne E. Becker. 2005. "The Culture of Medicine and Racial, Ethnic, and Class Disparities in Healthcare." In The Blackwell Companion to Social Inequalities, edited by Mary Romero and Eric Margolis, 396-423. Malden, MA: Blackwell.
  27. Gulshan, Varun, Lily Peng, Marc Coram, Martin C. Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, et al. 2016. "Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs." JAMA: The Journal of the American Medical Association 316, no. 22 (December 13): 2402-10. https://doi.org/10.1001/jam a.2016.17216.
  28. Gunnemyr, Mattias. 2020. "Why the Social Connection Model Fails: Participation Is Neither Necessary nor Sufficient for Political Responsibility." Hypatia 35, no. 4 (Fall): 567-86. https://doi.org/10.1017/hyp.2020.40.
  29. Hajian, Sara, and Josep Domingo-Ferrer. 2013. "A Methodology for Direct and Indirect Discrimination Prevention in Data Mining." IEEE Transactions on Knowledge Kaushal, Amit, Russ Altman, and Curt Langlotz. 2020. "Geographic Distribution of US Cohorts Used to Train Deep Learning Algorithms." JAMA: The Journal of the American Medical Association 324, no. 12 (September 22/29): 1212-13. https://doi.org/10.1001/jama.2020.12067.
  30. Kelly, Christopher J., Alan Karthikesalingam, Mustafa Suleyman, Greg Corrado, and Dominic King. 2019. "Key Challenges for Delivering Clinical Impact with Artificial Intelligence." BMC Medicine 17 (October 29): 195. https://doi.org/10 .1186/s12916-019-1426-2.
  31. Köhler, Sebastian, Neil Roughley, and Hanno Sauer. 2017. "Technologically Blurred Accountability? Technology, Responsibility Gaps and the Robustness of Our Everyday Conceptual Scheme." In Moral Agency and the Politics of Responsibility, edited by Cornelia Ulbert, Peter Finkenbusch, Elena Sondermann, and Tobias Debiel, 51-68. New York: Routledge.
  32. Le Bui, Matthew, and Safiya Umoja Noble. 2020. "We're Missing a Moral Framework of Justice in Artificial Intelligence: On the Limits, Failings, and Ethics of Fairness." The Oxford Handbook of Ethics of AI, edited by Markus D. Dubber, Frank Pasquale, and Sunit Das, 162-79. Oxford: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190067397.013.9.
  33. LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. "Deep Learning." Nature 521, no. 7553 (May 28): 436-44. https://doi.org/10.1038/nature14539.
  34. Lepri, Bruno, Nuria Oliver, Emmanuel Letouzé, Alex Pentland, and Patrick Vinck. 2018. "Fair, Transparent, and Accountable Algorithmic Decision-Making Processes." Philosophy & Technology 31, no. 4 (December): 611-27. https://doi.org/10.10 07/s13347-017-0279-x.
  35. Liao, Shen-Yi, and Vanessa Carbonell. 2022. "Materialized Oppression in Medical Tools and Technologies Materialized Oppression in Medical Tools and Technologies." American Journal of Bioethics. Published online ahead of print, March 9, 2022. https://doi.org/10.1080/15265161.2022.2044543.
  36. Lin, Ting-An. Forthcoming. "Sexual Violence and Two Types of Moral Wrongs." Hypatia.
  37. Longino, Helen E. 1990. Science as Social Knowledge: Values and Objectivity in Scientific Inquiry. Princeton, NJ: Princeton University Press.
  38. Lu, Catherine. 2017. Justice and Reconciliation in World Politics. Cambridge: Cambridge University Press.
  39. Manuel, Jennifer I. 2018. "Racial/Ethnic and Gender Disparities in Health Care Use and Access." Health Services Research 53, no. 3 (June): 1407-29. https://doi.org /10.1111/1475-6773.12705.
  40. Matthias, Andreas. 2004. "The Responsibility Gap: Ascribing Responsibility for the Actions of Learning Automata." Ethics and Information Technology 6, no. 3 (September): 175-83. https://doi.org/10.1007/s10676-004-3422-1.
  41. McKeown, Maeve. 2016. "Global Structural Exploitation: Towards an Intersectional Definition." Global Justice: Theory Practice Rhetoric 9 (2): 155-77. https://doi.org/10.21248/gjn.9.2.116.
  42. ---. 2018. "Iris Marion Young's 'Social Connection Model' of Responsibility: Clarifying the Meaning of Connection." Journal of Social Philosophy 49, no. 3 (Fall): 484-502. https://doi.org/10.1111/josp.12253.
  43. Miconi, Thomas. 2017. "The Impossibility of 'Fairness': A Generalized Impossibility Result for Decisions." arXiv preprint, arXiv:1707.01195 [stat.AP]. https://doi .org/10.48550/arXiv.1707.01195.
  44. Nelson, Alan. 2002. "Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care." Journal of the National Medical Association 94, no. 8 (August): 666-68.
  45. Noble, Safiya Umoja. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. New York: New York University Press.
  46. Nussbaum, Martha C. 2009. "Iris Young's Last Thoughts on Responsibility for Global Justice." In Dancing with Iris: The Philosophy of Iris Marion Young, edited by Ann Ferguson and Mechthild Nagle, 133-45. Oxford: Oxford University Press.
  47. Obermeyer, Ziad, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. "Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations." Science 366, no. 6464 (October 25): 447-53. https://doi.org/10 .1126/science.aax2342.
  48. Pierson, Emma, David M. Cutler, Jure Leskovec, Sendhil Mullainathan, and Ziad Obermeyer. 2021. "An Algorithmic Approach to Reducing Unexplained Pain Disparities in Underserved Populations." Nature Medicine 27, no. 1 (January): 136-40. https://doi.org/10.1038/s41591-020-01192-7.
  49. Poleshuck, Ellen L., and Carmen R. Green. 2008. "Socioeconomic Disadvantage and Pain." Pain 136, no. 3 (June): 235-38. https://doi.org/10.1016/j.pain.2008.04 .003.
  50. Rajkomar, Alvin, Michaela Hardt, Michael D. Howell, Greg Corrado, and Marshall H. Chin. 2018. "Ensuring Fairness in Machine Learning to Advance Health Equity." Annals of Internal Medicine 169, no. 12 (December 18): 866-72. https://doi.org/10.7326/M18-1990.
  51. Rodriguez, Rudolph A., Saunak Sen, Kala Mehta, Sandra Moody-Ayers, Peter Bacchetti, and Ann M. O'Hare. 2007. "Geography Matters: Relationships among Urban Residential Segregation, Dialysis Facilities, and Patient Outcomes." Annals of Internal Medicine 146, no. 7 (April 3): 493-501. https://doi.org/10.7326/0003- 4819-146-7-200704030-00005.
  52. Rudy-Hiller, Fernando. 2018. "The Epistemic Condition for Moral Responsibility." In The Stanford Encyclopedia of Philosophy, edited by Edward N. Zalta, Fall 2018 edition. Metaphysics Research Lab, Stanford University. https://plato.stanford .edu/archives/fall2018/entries/moral-responsibility-epistemic/.
  53. Schwenkenbecher, Anne. 2021. "Structural Injustice and Massively Shared Obligations." Journal of Applied Philosophy 38, no. 1 (February): 23-39. https://doi.org/10.1111/japp.12431.
  54. Shankar, Shreya, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D. Sculley. 2017. "No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World." arXiv preprint, arXiv:1711.08536 [stat.ML]. https://doi.org/10.48550/arXiv.1711.08536.
  55. Smith, Aaron. 2018. "Public Attitudes toward Computer Algorithms." Pew Research Center, November 16, 2018. https://www.pewresearch.org/internet/2018/11 /16/public-attitudes-toward-computer-algorithms/.
  56. Tigard, Daniel W. 2021. "There Is No Techno-Responsibility Gap." Philosophy & Technology 34, no. 3 (September): 589-607. https://doi.org/10.1007/s13347- 020-00414-7.
  57. van de Poel, Ibo, Jessica Nihlén Fahlquist, Neelke Doorn, Sjoerd Zwart, and Lambèr Royakkers. 2012. "The Problem of Many Hands: Climate Change as an Example." Science and Engineering Ethics 18, no. 1 (March): 49-67. https://doi .org/10.1007/s11948-011-9276-0.
  58. van der Veer, Sabine N., Lisa Riste, Sudeh Cheraghi-Sohi, Denham L. Phipps, Mary P. Tully, Kyle Bozentko, Sarah Atwood, et al. 2021. "Trading off Accuracy and Explainability in AI Decision-Making: Findings from 2 Citizens' Juries." Journal of the American Medical Informatics Association: JAMIA 28, no. 10 (October): 2128-38. https://doi.org/10.1093/jamia/ocab127.
  59. Vidyasagar, D. 2006. "Global Notes: The 10/90 Gap Disparities in Global Health Research." Journal of Perinatology 26, no. 1 (January): 55-56. https://doi.org /10.1038/sj.jp.7211402.
  60. Vyas, Darshali A., Leo G. Eisenstein, and David S. Jones. 2020. "Hidden in Plain Sight- Reconsidering the Use of Race Correction in Clinical Algorithms." New England Journal of Medicine 383, no. 9 (August 27): 874-82. https://doi.org/10.1056 /nejmms2004740.
  61. Wawira Gichoya, Judy, Liam G. McCoy, Leo Anthony Celi, Marzyeh Ghassemi. 2021. "Equity in Essence: A Call for Operationalising Fairness in Machine Learning for Healthcare." BMJ Health & Care Informatics 28 (1): e100289. https://doi.org /10.1136/bmjhci-2020-100289.
  62. Wong, Pak-Hang, and Judith Simon. 2020. "Thinking About 'Ethics' in the Ethics of AI." Idees, no. 48 ("Artificial Intelligence"). https://revistaidees.cat/en/thinking- about-ethics-in-the-ethics-of-ai/.
  63. Wylie, Alison, and Lynn Hankinson Nelson. 2007. "Coming to Terms with the Values of Science: Insights From Feminist Science Studies Scholarship." In Value-Free Science? Ideals and Illusions, edited by Harold Kincaid, John Dupré, and Alison Wylie, 58-86. New York: Oxford University Press.
  64. Young, Iris Marion. 2006. "Responsibility and Global Justice: A Social Connection Model." Social Philosophy & Policy 23, no. 1 (January). Cambridge University Press: 102-30. https://doi.org/10.1017/S0265052506060043.
  65. ---. 2011. Responsibility for Justice. Oxford: Oxford University Press.
  66. Zhao, Qingyu, Ehsan Adeli, and Kilian M. Pohl. 2020. "Training Confounder-Free Deep Learning Models for Medical Applications." Nature Communications 11:6010. https://doi.org/10.1038/s41467-020-19784-9.
  67. Zheng, Robin. 2018. "What Is My Role in Changing the System? A New Model of Responsibility for Structural Injustice." Ethical Theory and Moral Practice 21, no. 4 (August): 869-85. https://doi.org/10.1007/s10677-018-9892-8.
  68. Zimmermann, Annette, Elena Di Rosa, and Hochan Kim. 2020. "Technology Can't Fix Algorithmic Injustice." Boston Review, January 9, 2020. https://bostonreview .net/articles/annette-zimmermann-algorithmic-political/.

Related papers

Fair and equitable AI in biomedical research and healthcare: social science perspectives
Payal Arora, Renate Baumgartner

Artificial Intelligence in Medicine, 2023

Artificial intelligence (AI) offers opportunities but also challenges for biomedical research and healthcare. This position paper shares the results of the international conference “Fair medicine and AI” (online 3-5 March 2021). Scholars from science and technology studies (STS), gender studies, and ethics of science and technology formulated opportunities, challenges, and research and development desiderata for AI in healthcare. AI systems and solutions, which are being rapidly developed and applied, may have undesirable and unintended consequences including the risk of perpetuating health inequalities for marginalized groups. Socially robust development and implications of AI in healthcare require urgent investigation. There is a particular dearth of studies in human-AI interaction and how this may best be configured to dependably deliver safe, effective and equitable healthcare. To address these challenges, we need to establish diverse and interdisciplinary teams equipped to develop and apply medical AI in a fair, accountable and transparent manner. We formulate the importance of including social science perspectives in the development of intersectionally beneficent and equitable AI for biomedical research and healthcare, in part by strengthening AI health evaluation.

View PDFchevron_right
Beyond bias and discrimination: redefining the AI ethics principle of fairness in healthcare machine-learning algorithms
Benedetta Giovanola

AI & SOCIETY

The increasing implementation of and reliance on machine-learning (ML) algorithms to perform tasks, deliver services and make decisions in health and healthcare have made the need for fairness in ML, and more specifically in healthcare ML algorithms (HMLA), a very important and urgent task. However, while the debate on fairness in the ethics of artificial intelligence (AI) and in HMLA has grown significantly over the last decade, the very concept of fairness as an ethical value has not yet been sufficiently explored. Our paper aims to fill this gap and address the AI ethics principle of fairness from a conceptual standpoint, drawing insights from accounts of fairness elaborated in moral philosophy and using them to conceptualise fairness as an ethical value and to redefine fairness in HMLA accordingly. To achieve our goal, following a first section aimed at clarifying the background, methodology and structure of the paper, in the second section, we provide an overview of the discussion of the AI ethics principle of fairness in HMLA and show that the concept of fairness underlying this debate is framed in purely distributive terms and overlaps with non-discrimination, which is defined in turn as the absence of biases. After showing that this framing is inadequate, in the third section, we pursue an ethical inquiry into the concept of fairness and argue that fairness ought to be conceived of as an ethical value. Following a clarification of the relationship between fairness and non-discrimination, we show that the two do not overlap and that fairness requires much more than just non-discrimination. Moreover, we highlight that fairness not only has a distributive but also a socio-relational dimension. Finally, we pinpoint the constitutive components of fairness. In doing so, we base our arguments on a renewed reflection on the concept of respect, which goes beyond the idea of equal respect to include respect for individual persons. In the fourth section, we analyse the implications of our conceptual redefinition of fairness as an ethical value in the discussion of fairness in HMLA. Here, we claim that fairness requires more than non-discrimination and the absence of biases as well as more than just distribution; it needs to ensure that HMLA respects persons both as persons and as particular individuals. Finally, in the fifth section, we sketch some broader implications and show how our inquiry can contribute to making HMLA and, more generally, AI promote the social good and a fairer society.

View PDFchevron_right
A Proposed Framework on Integrating Health Equity and Racial Justice into the Artificial Intelligence Development Lifecycle
Yuri Quintana

Journal of Health Care for the Poor and Underserved, 2021

The COVID-19 pandemic has created multiple opportunities to deploy artificial intelligence (AI)-driven tools and applied interventions to understand, mitigate, and manage the pandemic and its consequences. The disproportionate impact of COVID-19 on racial/ ethnic minority and socially disadvantaged populations underscores the need to anticipate and address social inequalities and health disparities in AI development and application. Before the pandemic, there was growing optimism about AI's role in addressing inequities and enhancing personalized care. Unfortunately, ethical and social issues that are encountered in scaling, developing, and applying advanced technologies in health care settings have intensified during the rapidly evolving public health crisis. Critical voices concerned with the disruptive potentials and risk for engineered inequities have called for reexamining ethical guidelines in the development and application of AI. This paper proposes a framework to incorporate ethical AI principles into the development process in ways that intentionally promote racial health equity and social justice. Without centering on equity, justice, and ethical AI, these tools may exacerbate structural inequities that can lead to disparate health outcomes.

View PDFchevron_right
Who does the fairness in health AI community represent?
Amelia Fiske

medRxiv (Cold Spring Harbor Laboratory), 2023

OBJECTIVE: Artificial intelligence (AI) and machine learning are central components of today's medical environment. The fairness of AI, i.e. the ability of AI to be free from bias, has repeatedly come into question. This study investigates the diversity of the members of academia whose scholarship poses questions about the fairness of AI. METHODS: The articles that combine the topics of fairness, artificial intelligence, and medicine were selected from Pubmed, Google Scholar, and Embase using keywords. Eligibility and data extraction from the articles were done manually and cross-checked by another author for accuracy. 375 articles were selected for further analysis, cleaned, and organized in Microsoft Excel; spatial diagrams were generated .

View PDFchevron_right
An Expert Panel DiscussionEmbedding Ethics and Equity in Artificial Intelligence and Machine Learning Infrastructure
Malaika Simmons

Big Data

Alliance against Disparities in Patient Health (NADPH), a data-driven nonprofit health research organization. She uses her background in research, psychology, and design thinking to promote empathy-based leadership with over 20 years of experience in corporate policy, training, and program management, operating at the convergence of health research, IT, and consumer issues. Malaika has a strong commitment to improving the health of underserved populations and more broadly supports these efforts through her organization Momentology Media. Through Momentology, Malaika fulfills her passion for the elimination of economic and health disparities in minority communities by using her proprietary framework to empower women to own businesses, champion health causes, and enter the corporate and political landscapes to affect change in their corner of the world. The NADPH is a nonprofit health research organization applying scientific research and technology, community education and outreach, and advocacy to reduce health disparities and improve public health outcomes for our nation. NADPH operates through a national partnership ecosystem that spans government, academic, nonprofit, and industry stakeholders. One of NADPH's major objectives is to bridge the gap in access to advanced precision-based health care in underserved and under-resourced communities, NADPH seeks to do this by providing access to information, subject matter experts, and technology that help community members: (1) raise their awareness and education about the array of factors impacting their quality of health and health care; (2) capture, manage, and understand their health information across its various domains; and (3) use this increased knowledge and access more actionably to make better decisions about their health and health care.

View PDFchevron_right

Related topics

  • structural injustice
  • Ethics of AI
  • AI and BIAS
  • AI in Healthcare
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved