Algorithmic injustice and human rights

JIPITEC, 2018

Nowadays algorithms can decide if one can get a loan, is allowed to cross a border, or must go to prison. Artificial intelligence techniques (natural language processing and machine learning in the first place) enable private and public decision-makers to analyse big data in order to build profiles, which are used to make decisions in an automated way. This work presents ten arguments against algorithmic decision-making. These revolve around the concepts of ubiquitous discretionary interpretation, holistic intuition, algorithmic bias, the three black boxes, psychology of conformity, power of sanctions, civilising force of hypocrisy, pluralism, empathy, and technocracy. The lack of transparency of the algorithmic decision-making process does not stem merely from the characteristics of the relevant techniques used, which can make it impossible to access the rationale of the decision. It depends also on the abuse of and overlap between intellectual property rights (the “legal black box”). In the US, nearly half a million patented inventions concern algorithms; more than 67% of the algorithm-related patents were issued over the last ten years and the trend is increasing. To counter the increased monopolisation of algorithms by means of intellectual property rights (with trade secrets leading the way), this paper presents three legal routes that enable citizens to ‘open’ the algorithms. First, copyright and patent exceptions, as well as trade secrets are discussed. Second, the GDPR is critically assessed. In principle, data controllers are not allowed to use algorithms to take decisions that have legal effects on the data subject’s life or similarly significantly affect them. However, when they are allowed to do so, the data subject still has the right to obtain human intervention, to express their point of view, as well as to contest the decision. Additionally, the data controller shall provide meaningful information about the logic involved in the algorithmic decision. Third, this paper critically analyses the first known case of a court using the access right under the freedom of information regime to grant an injunction to release the source code of the computer program that implements an algorithm. Only an integrated approach – which takes into account intellectual property, data protection, and freedom of information – may provide the citizen affected by an algorithmic decision of an effective remedy as required by the Charter of Fundamental Rights of the EU and the European Convention on Human Rights. Recommended citation: Guido Noto La Diega, Against the Dehumanisation of Decision-Making – Algorithmic Decisions at the Crossroads of Intellectual Property, Data Protection, and Freedom of Information, 9 (2018) JIPITEC 3 para 1

Philosophy & Technology, 2024

The call to make artificial intelligence (AI) more democratic, or to “democratize AI,” is sometimes framed as a promising response for mitigating algorithmic injustice or making AI more aligned with social justice. However, the notion of “democratizing AI” is elusive, as the phrase has been associated with multiple meanings and practices, and the extent to which it may help mitigate algorithmic injustice is still underexplored. In this paper, based on a socio-technical understanding of algorithmic injustice, I examine three notable notions of democratizing AI and their associated measures—democratizing AI use, democratizing AI development, and democratizing AI governance—regarding their respective prospects and limits in response to algorithmic injustice. My examinations reveal that while some versions of democratizing AI bear the prospect of mitigating the concern of algorithmic injustice, others are somewhat limited and might even function to perpetuate unjust power hierarchies. This analysis thus urges a more fine-grained discussion on how to democratize AI and suggests that closer scrutiny of the power dynamics embedded in the socio-technical structure can help guide such explorations.

Boston Review, 2019

There may be some machine learning systems that should not be deployed in the first place, no matter how much we can optimize them. Developers cannot just ask, “What do I need to do to fix my algorithm?” They must rather ask: “How does my algorithm interact with society at large, and as it currently is, including its structural inequalities?” Ultimately, we must resist the apocalypse-saturated discourse on AI that encourages a mentality of learned helplessness—and citizens must come to view issues surrounding AI as a collective problem for all of us rather than a technical problem just for them.

Brazilian Journal of Law Technology and Innovation, 2023

This article scrutinizes the escalating apprehensions surrounding algorithmic transparency, positing it as a pivotal facet for ethics and accountability in the development and deployment of artificial intelligence (AI) systems. By delving into legislative and regulatory initiatives across various jurisdictions, the article discerns how different countries and regions endeavor to institute guidelines fostering ethical and responsible AI systems. Within the United States, both the US Algorithmic Accountability Act of 2022 and The European Artificial Intelligence Act share a common objective of establishing governance frameworks to hold errant entities accountable, ensuring the ethical, legal, and secure implementation of AI systems. A key emphasis in both legislations is placed on algorithmic transparency and elucidation of system functionalities, with the overarching goal of instilling accountability in AI operations. This examination extends to Brazil, where legislative proposals such as PL 2.338/2023 grapple with the intricacies of AI deployment and algorithmic transparency. Furthermore, PEC 29/2023 endeavors to enshrine algorithmic transparency as a fundamental right, recognizing its pivotal role in safeguarding users' mental integrity in the face of advancing neurotechnology and algorithmic utilization. To ascertain the approaches adopted by Europe, the United States, and Brazil in realizing the concept of Algorithmic Transparency in AI systems employed for decision-making, a comparative and deductive methodology is employed. This methodology aligns with bibliographical analysis, incorporating legal doctrines, legislative texts, and jurisprudential considerations from the respective legal systems. The analysis encompasses Algorithmic Transparency, Digital Due Process, and Accountability as inherent legal constructs, offering a comprehensive comparative perspective. However, the mere accessibility of source codes is deemed insufficient to guarantee effective comprehension and scrutiny by end-users. Recognizing this, the imperative of explainability in elucidating how AI systems function becomes evident, enabling citizens to comprehend the rationale behind decisions made by these systems. Legislative initiatives, exemplified by Resolution No. 332/2020 of the National Council of Justice (CNJ), underscore the acknowledgment of the imperative for transparency and accountability in AI systems utilized within the Judiciary.

In October 2017, Palestinian worker Halawin Halawi was arrested for posting a photo of himself next to a bulldozer with a caption that read "attack them" in Arabic. Only after questioning him for several hours, the police realised Facebook's AI powered translation service had made a mistake: the Arabic for "attack them" and "good morning" are actually very similar-and the algorithm had chose the wrong one. Also last year, Chinese messaging app WeChat apologised for translating "hei laowai", a neutral phrase that means "black foreigner" to the derogatory term "nigger". Google translate, that has had an undeniable importance in helping society overcome the language divide and understanding other cultures other than our own, has also shown it carries a fair share of bias-when translating Turkish (a gender neutral language) to English, it would assume the pronoun "o" to be he or she, according to the profession that followed. For example, "o bir doktor" becomes "he is a doctor" but "o bir aşçı" translates to "she is a cook". While "O bekar" goes for "he is single", "o evli" means "she is married". The same for "o çalışkan"-"he is hardworking" and "o tembel" that translates to "she is lazy". The role these algorithms play in perpetuating our cultural biases are deeply problematic. In an ever mediated society where technology plays a bigger and bigger part in how we relate to each other and the outside world-the prejudice hiding behind the code means that important decisions go unchecked and lack a great deal of ownership. The fact that Halawi was arrested before any Arabic speaking officer read the original post poses many important questions as these algorithms that conceal hidden biases are already making vital decisions in our everyday lives. Proprietary algorithms can decide who gets granted parole, who gets interviewed for a job and who gets a loan. They also decide what we buy, what content we watch, what books we read and, to an extent, who we date and are friends with.

Daedelus, 2022

This paper explores the relationship between artificial intelligence and principles of distributive justice. Drawing upon the political philosophy of John Rawls, it holds that the basic structure of society should be understood as a composite of socio-technical systems, and that the operation of these systems is increasingly shaped and influenced by AI. As a consequence, egalitarian norms of justice apply to the technology when it is deployed in these contexts. These norms entail that the relevant AI systems must meet a certain standard of public justification, support citizens rights, and promote substantively fair outcomes – something that requires specific attention be paid to the impact they have on the worst-off members of society.

Law and Contemporary Problems, 84(3), 19-33, 2021

Res Publica

Algorithmic discrimination has rapidly become a topic of intense public and academic interest. This article explores three issues raised by algorithmic discrimination: 1) the distinction between direct and indirect discrimination, 2) the notion of disadvantageous treatment, and 3) the moral badness of discriminatory automated decision-making. It argues that some conventional distinctions between direct and indirect discrimination appear not to apply to algorithmic discrimination, that algorithmic discrimination may often be discrimination between groups, as opposed to against groups, and that it is not necessarily the case that morally bad algorithmic discrimination gives us reason to not use automated decision-making. For each of the three issues, the article explores implications for algorithmic discrimination, suggests some alternative answers, and clarifies how we may want to think of discrimination more broadly in light of lessons drawn from the context of algorithmic discrimination.

Equity and Access in Algorithms, Mechanisms, and Optimization

In this position paper, we introduce a new epistemic lens for analyzing algorithmic harm. We argue that the epistemic lens we propose herein has two key contributions to help reframe and address some of the assumptions underlying inquiries into algorithmic fairness. First, we argue that using the framework of epistemic injustice helps to identify the root causes of harms currently framed as instances of representational harm. We suggest that the epistemic lens offers a theoretical foundation for expanding approaches to algorithmic fairness in order to address a wider range of harms not recognized by existing technical or legal definitions. Second, we argue that the epistemic lens helps to identify the epistemic goals of inquiries into algorithmic fairness. There are two distinct contexts within which we examine algorithmic harm: at times, we seek to understand and describe the world as it is, and, at other times, we seek to build a more just future. The epistemic lens can serve to direct our attention to the epistemic frameworks that shape our interpretations of the world as it is and the ways we envision possible futures. Clarity with respect to which epistemic context is relevant in a given inquiry can further help inform choices among the different ways of measuring and addressing algorithmic harms. We introduce this framework with the goal of initiating new research directions bridging philosophical, legal, and technical approaches to understanding and mitigating algorithmic harms. CCS CONCEPTS • Computing methodologies → Philosophical/theoretical foundations of artificial intelligence.

Res Publica, 2023

Algorithmic discrimination has rapidly become a topic of intense public and academic interest. This article explores three issues raised by algorithmic discrimination: 1) the distinction between direct and indirect discrimination, 2) the notion of disadvantageous treatment, and 3) the moral badness of discriminatory automated decision-making. It argues that some conventional distinctions between direct and indirect discrimination appear not to apply to algorithmic discrimination, that algorithmic discrimination may often be discrimination between groups, as opposed to against groups, and that it is not necessarily the case that morally bad algorithmic discrimination gives us reason to not use automated decisionmaking. For each of the three issues, the article explores implications for algorithmic discrimination, suggests some alternative answers, and clarifies how we may want to think of discrimination more broadly in light of lessons drawn from the context of algorithmic discrimination.