Artificial intelligence, transparency, and public decision-making

Technology in Society, 2020

Artificial intelligence (AI) has proven to be superior to human decision-making in certain areas. This is particularly the case whenever there is a need for advanced strategic reasoning and analysis of vast amounts of data in order to solve complex problems. Few human activities fit this description better than politics. In politics we deal with some of the most complex issues humans face, short-term and long-term consequences have to be balanced, and we make decisions knowing that we do not fully understand their consequences. I examine an extreme case of the application of AI in the domain of government, and use this case to examine a subset of the potential harms associated with algorithmic governance. I focus on five objections based on political theoretical considerations and the potential political harms of an AI technocracy. These are objections based on the ideas of 'political man' and participation as a prerequisite for legitimacy, the non-morality of machines and the value of transparency and accountability. I conclude that these objections do not successfully derail AI technocracy, if we make sure that mechanisms for control and backup are in place, and if we design a system in which humans have control over the direction and fundamental goals of society. Such a technocracy, if the AI capabilities of policy formation here assumed becomes reality, may, in theory, provide us with better means of participation, legitimacy, and more efficient government.

Media Theory , 2023

So-called artificial intelligence (AI) is infiltrating our public and communication structures. The Dutch childcare benefit scandal, revealed in 2019, demonstrates how disadvantageous the opacity of AI can be for already vulnerable groups. In its aftermath, many scholars urged for the need for more explainable AI so that decision-makers can intervene in discriminatory systems. Fostering the explainability of AI (XAI) is a good start to address the issue, but not enough to empower vulnerable groups to fully deal with its repercussions. As a canon in data and computer sciences, XAI aims to illustrate and explain complex AI via simpler models making it more accessible and ethical. The issue being that, in doing so, XAI depoliticises transparency into a remedy for algorithmic opacity, treating transparency as artificially stripped of its ideological meanings. Transparency is presented as an antidote to ideology, though I will show how this is an ideological move with consequences. For instance, it makes us focus too much on algorithmic opacity, rather than explaining the wider power of AI. Second, it hinders us from having debates on who holds the power around AI's explanations, application or critique. The problem is that those affected by or discriminated against by AI, as in the Dutch case, have little tools to deal with the opacity of AI as a system, while those who focus on data opacity are shaping the literacy discussion. To address these concerns, I suggest moving beyond the focus on algorithmic transparency and towards a post-critical AI literacy to strengthen debates on access, empowerment, and resistance, while not dismissing XAI as a field, nor algorithmic transparency as an intention. What I challenge here is the hegemony of treating transparency as a depoliticised and algorithmic issue and viewing the explainability of AI as the sufficient path to citizen empowerment.

Science and Engineering Ethics, 2020

In this article, we develop the concept of Transparency by Design that serves as practical guidance in helping promote the beneficial functions of transparency while mitigating its challenges in automated-decision making (ADM) environments. With the rise of artificial intelligence (AI) and the ability of AI systems to make automated and self-learned decisions, a call for transparency of how such systems reach decisions has echoed within academic and policy circles. The term transparency, however, relates to multiple concepts, fulfills many functions, and holds different promises that struggle to be realized in concrete applications. Indeed, the complexity of transparency for ADM shows tension between transparency as a normative ideal and its translation to practical application. To address this tension, we first conduct a review of transparency, analyzing its challenges and limitations concerning automated decision-making practices. We then look at the lessons learned from the development of Privacy by Design, as a basis for developing the Transparency by Design principles. Finally, we propose a set of nine principles to cover relevant con-textual, technical, informational, and stakeholder-sensitive considerations. Transparency by Design is a model that helps organizations design transparent AI systems, by integrating these principles in a step-by-step manner and as an ex-ante value, not as an afterthought.

Requirements Engineering: Foundation for Software Quality, 2021

Artificial intelligence (AI) creates many opportunities for public institutions, but the unethical use of AI in public services can reduce citizens' trust. [Question] The aim of this study was to identify what kind of requirements citizens have for trustworthy AI services in the public sector. The study included 21 interviews and a design workshop of four public AI services. [Results] The main finding was that all the participants wanted public AI services to be transparent. This transparency requirement covers a number of questions that trustworthy AI services must answer, such as about their purposes. The participants also asked about the data used in AI services and from what sources the data were collected. They pointed out that AI must provide easy-to-understand explanations. We also distinguished two other important requirements: controlling personal data usage and involving humans in AI services. [Contribution] For practitioners, the paper provides a list of questions that trustworthy public AI services should answer. For the research community, it illuminates the transparency requirement of AI systems from the perspective of citizens.

AI governance: Compromising democracy or democratising AI?, 2024

The increasing integration of artificial intelligence (AI) into society raises critical questions about its impact on democracy. The development of AI governance frameworks presents a convenient and crucial ground to strengthen democracy, particularly through the lens of participatory and deliberative theories. From this viewpoint, this article seeks to explore the extent to which emerging AI governance frameworks uphold participatory and deliberative democracy. Through a comparative analysis of proposed or existing legislation in the European Union (EU), Brazil and Canada, this study investigates the specific tools and mechanisms each framework uses to involve the citizens in AI governance. The analysis reveals that while all three jurisdictions emphasise ethical governance and assessment of AI along with regulatory dialogue and multi-stakeholder collaboration, this does not effectively extend to creation of robust and specific mechanisms to facilitate citizens' participation and deliberation. Exhibiting a stronger commitment to participatory and deliberative democracy, the Brazilian framework demonstrates a firmer position, incorporating a wider range of individual rights and direct avenues for citizen input and engagement, e.g. mandatory public consultation in advance of algorithmic impact assessment. Conversely, the Canadian and EU approaches largely rely on existing institutions and processes, overlooking the unique challenges, i.e. knowledge barriers, economic and social injustices, expert rule, that cannot be fully resolved by toolset of ethics governance or through alternative venues such as citizens' assemblies. Overall, this study concludes by advocating for the integration of novel mechanisms that can facilitate citizen participation and deliberation within AI governance frameworks, including specifically designed and institutionalised deliberative venues.

Journal of the American Medical Informatics Association

Objective To investigate how the general public trades off explainability versus accuracy of artificial intelligence (AI) systems and whether this differs between healthcare and non-healthcare scenarios. Materials and Methods Citizens’ juries are a form of deliberative democracy eliciting informed judgment from a representative sample of the general public around policy questions. We organized two 5-day citizens’ juries in the UK with 18 jurors each. Jurors considered 3 AI systems with different levels of accuracy and explainability in 2 healthcare and 2 non-healthcare scenarios. Per scenario, jurors voted for their preferred system; votes were analyzed descriptively. Qualitative data on considerations behind their preferences included transcribed audio-recordings of plenary sessions, observational field notes, outputs from small group work and free-text comments accompanying jurors’ votes; qualitative data were analyzed thematically by scenario, per and across AI systems. Results I...

Moral Philosophy and Politics, 2019

Policymakers are responsible for key decisions about political governance. Usually, they are selected or elected based on experience and then supported in their decision-making by the additional counsel of subject experts. Those satisfied with this system believe these individuals – generally speaking – will have the right intuitions about the best types of action. This is important because political decisions have ethical implications; they affect how we all live in society. Nevertheless, there is a wealth of research that cautions against trusting human judgment as it can be severely flawed. This paper will look at the root causes of the most common errors of human judgment before arguing –contrathe instincts of many – that future AI systems could take a range of political decisions more reliably. I will argue that, if/when engineers establish ethically robust systems, governments will have amoral obligationto refer to them as a part of decision-making.

Data & Policy, 2022

A number of governmental and nongovernmental organizations have made significant efforts to encourage the development of artificial intelligence in line with a series of aspirational concepts such as transparency, interpretability, explainability, and accountability. The difficulty at present, however, is that these concepts exist at a fairly abstract level, whereas in order for them to have the tangible effects desired they need to become more concrete and specific. This article undertakes precisely this process of concretisation, mapping how the different concepts interrelate and what in particular they each require in order to move from being high-level aspirations to detailed and enforceable requirements. We argue that the key concept in this process is accountability, since unless an entity can be held accountable for compliance with the other concepts, and indeed more generally, those concepts cannot do the work required of them. There is a variety of taxonomies of accountabil...

Computer, 2020

AI decision-making can cause discriminatory harm to many vulnerable groups. Redress is often suggested through increased transparency of these systems. But who are we implementing it for? This article seeks to identify what transparency means for technical, legislative and public realities and stakeholders.

Review of Politics, 2024

Artificial intelligence cannot bear legal or moral responsibility for the consequences of its use. Responsibility lies with those human beings who design, develop, and deploy it, from lead engineers and project managers to corporate leadership, and from government regulators to elected representatives. Either these persons assign responsibility to themselves (or their colleagues, or associates, or employees) or responsibility is assigned to them by nonelite outsiders and laypersons. Josh Simons envisions as much: the citizens of a political community assigning responsibility via democratic participation in public-sphere decision-making, about some features of corporate design of AI. Citizens would assign responsibility for AI-related, inequality-based social injustice that follows from ranking systems, which are predictive tools employed in machine learning. The design of ranking systems-with unforeseen political ramifications of uncertain because obscured provenance-imposes an "artificial kind of scarcity on vast quantities of content and websites" and restricts "who is seen and heard by whom" (195). With "top-line metrics, values, and concepts built into [their] design" (195), ranking systems "bake in political choices, naturalizing the web it ceaselessly shapes" (127). They "direct citizens' attention" and "shape the exercise of self-governance" (181) in ways that "corrupt the public sphere" by producing "filter bubbles and social division" (135). Simons focuses on two corporations, Facebook and Google.