Academia.eduAcademia.edu

Outline

Title
Abstract
Figures
Introduction
Responsible Design for Ai
Method of Design Framework Construction
Data Collection
Analysis & Synthesis
Human Review and Intervention Requests
Tools for Scrutiny by Subjects or Third Parties
Contestable Ai by Design: Towards a Framework
Discussion
Appendix 1: Summary of Reviewed Literature
References
All Topics
Philosophy
Philosophy of Religion

Contestable AI by Design: Towards a Framework

Profile image of Kars AlfrinkKars Alfrink

Minds and Machines

https://doi.org/10.1007/S11023-022-09611-Z
visibility

…

description

27 pages

link

1 file

Abstract

As the use of AI systems continues to increase, so do concerns over their lack of fairness, legitimacy and accountability. Such harmful automated decision-making can be guarded against by ensuring AI systems are contestable by design: responsive to human intervention throughout the system lifecycle. Contestable AI by design is a small but growing field of research. However, most available knowledge requires a significant amount of translation to be applicable in practice. A proven way of conveying intermediate-level, generative design knowledge is in the form of frameworks. In this article we use qualitative-interpretative methods and visual mapping techniques to extract from the literature sociotechnical features and practices that contribute to contestable AI, and synthesize these into a design framework.

Figures (6)
generalized knowledge (theory). The design knowledge we seek to create describes particular sociotechnical system properties operationalizing the principle of con- testability. We ground our framework in current knowledge on contestable AI. The purpose of the framework is to aid in the creation of designed artifacts. Following Stolterman and Wiberg (2010), we understand such design artifacts to be either in the service of improving a use situation, or in service of embodying new ideas (con- cepts) and theories. Our definition of “design framework” is aligned with Obrenovi¢ (2011). It should describe “the characteristics that a design solution should have to achieve a particular set of goals in a particular context”, where our goal is contest- able AI in the context of automated decision-making.
generalized knowledge (theory). The design knowledge we seek to create describes particular sociotechnical system properties operationalizing the principle of con- testability. We ground our framework in current knowledge on contestable AI. The purpose of the framework is to aid in the creation of designed artifacts. Following Stolterman and Wiberg (2010), we understand such design artifacts to be either in the service of improving a use situation, or in service of embodying new ideas (con- cepts) and theories. Our definition of “design framework” is aligned with Obrenovi¢ (2011). It should describe “the characteristics that a design solution should have to achieve a particular set of goals in a particular context”, where our goal is contest- able AI in the context of automated decision-making.
Fig. 1 Flow of information through the different phases of the systematic review
Fig. 1 Flow of information through the different phases of the systematic review
business and use-case development, ex-ante safeguards are put in place to protect against potential harms. During design and procurement of training and test data, agonistic development approaches enable stakeholder participation, making room for and leveraging conflict towards continuous improvement. During building and testing, quality assurance measures are used to ensure stakeholder interests are cen- tered and progress towards shared goals is tracked. During deployment and monitor- ing, further quality assurance measures ensure system performance is tracked on an ongoing basis, and the feedback loop with future system development is closed. Finally, throughout, risk mitigation intervenes in the system context to reduce the odds of failure, and third party oversight strengthens the role of external reviewers to enable ongoing outside scrutiny.
business and use-case development, ex-ante safeguards are put in place to protect against potential harms. During design and procurement of training and test data, agonistic development approaches enable stakeholder participation, making room for and leveraging conflict towards continuous improvement. During building and testing, quality assurance measures are used to ensure stakeholder interests are cen- tered and progress towards shared goals is tracked. During deployment and monitor- ing, further quality assurance measures ensure system performance is tracked on an ongoing basis, and the feedback loop with future system development is closed. Finally, throughout, risk mitigation intervenes in the system context to reduce the odds of failure, and third party oversight strengthens the role of external reviewers to enable ongoing outside scrutiny.
Table 4 Practices contributing to contestable AI Acknowledgements The authors would like to thank the reviewers for their constructive comments. Author Contributions All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by KA. The first draft of the manuscript was written by KA and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Table 4 Practices contributing to contestable AI Acknowledgements The authors would like to thank the reviewers for their constructive comments. Author Contributions All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by KA. The first draft of the manuscript was written by KA and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Related papers

Towards a Design Framework for Humanized AI
Luca Iandoli

2021

Negative impacts and unintended dysfunctions of AI are increasingly under scrutiny, and AI designers are often accused of not being considerate enough of the depth and consequences of the innovation they create. In this paper, we present a framework to assess the impact of AI solutions on their users, society, and environment to help designers to anticipate potential issues and create solutions that are not harmful and support individuals and society's betterment. Humanistic approaches to digital design inspire the proposed framework, including Positive and Inclusive design. The originality of the proposed approach derives from assessing the consequences of using computing solutions as they propagate across individual, social and environmental layers centered around the task the solution is supposed to perform. This paper presents an initial version of the framework and outlines the next steps for its full development.

View PDFchevron_right
Rules for Radical AI: A Counter-Framework for Algorithmic Contestation
Victor Frimpong

Annals of Social Sciences & Management Studies, 2025

AI ethics frameworks—grounded in principles of transparency, fairness, and accountability—have become the dominant mechanism for governing algorithmic systems. However, by treating AI as a neutral technology amenable to technical fixes, they obscure its status as a political project embedded in power, inequality, and historical injustice. In this paper, we introduce the concept of algorithmic contestation—the use of collective, tactical interventions to challenge and reshape the socio-political forces that drive the design and deployment of AI. Drawing on Saul Alinsky’s Rules for Radicals, we derive ten “Rules for Radical AI” that activists and researchers can deploy, including “Reveal the Code,” “Organize a Data Strike,” and “Seize Policy Windows.” We first map mainstream ethics principles onto Alinsky’s tactical logic, then illustrate these dynamics through two case studies—the San Francisco facial recognition ban and a New York City gig worker data strike. Finally, we discuss methodological limits, ethical risks, and future research avenues and issue a manifesto: proper AI accountability arises not from compliance alone but from collective contestation of algorithmic power.

View PDFchevron_right
Human-centricity in AI governance: A systemic approach
Anton Sigfrids

Frontiers in Artificial Intelligence

Human-centricity is considered a central aspect in the development and governance of artificial intelligence (AI). Various strategies and guidelines highlight the concept as a key goal. However, we argue that current uses of Human-Centered AI (HCAI) in policy documents and AI strategies risk downplaying promises of creating desirable, emancipatory technology that promotes human wellbeing and the common good. Firstly, HCAI, as it appears in policy discourses, is the result of aiming to adapt the concept of human-centered design (HCD) to the public governance context of AI but without proper reflection on how it should be reformed to suit the new task environment. Second, the concept is mainly used in reference to realizing human and fundamental rights, which are necessary, but not sufficient for technological emancipation. Third, the concept is used ambiguously in policy and strategy discourses, making it unclear how it should be operationalized in governance practices. This article ...

View PDFchevron_right
Varieties of transparency: Exploring agency within AI systems
Gloria Andrada

AI and Society, 2022

AI systems play an increasingly important role in shaping and regulating the lives of millions of human beings across the world. Calls for greater transparency from such systems have been widespread. However, there is considerable ambiguity concerning what "transparency" actually means, and therefore what greater transparency might entail. While, according to some debates, transparency requires seeing through the artefact or device, widespread calls for transparency imply seeing into different aspects of AI systems. These two notions are in apparent tension with each other, and they are present in two lively but largely disconnected debates. In this paper we aim to further analyse what these calls for transparency entail, and in so doing, clarify the sorts of transparency that we should want from AI systems. We do so by offering a taxonomy that classifies different notions of transparency. After a careful exploration of the different varieties of transparency, we show how this taxonomy can help us to navigate various domains of human-technology interactions, and more usefully discuss the relationship between technological transparency and human agency. We conclude by arguing that all of these different notions of transparency should be taken into account for designing more ethically adequate AI systems.

View PDFchevron_right
Meaningful human control over AI systems: beyond talking the talk
Deborah Forster

ArXiv, 2021

How can humans remain in control of artificial intelligence (AI)-based systems designed to have autonomous capabilities? Such systems are increasingly ubiquitous, creating benefits but also undesirable situations where moral responsibility for their actions cannot be properly attributed to any particular person or group. The concept of meaningful human control has been proposed to address responsibility gaps and mitigate them by establishing conditions that enable a proper attribution of responsibility for humans (e.g., users, designers and developers, manufacturers, legislators). However, the relevant discussions around meaningful human control have so far not resulted in clear requirements for researchers, designers, and engineers. As a result, there is no consensus on how to assess whether a designed AI system is under meaningful human control, making the practical development of AI-based systems that remain under meaningful human control challenging. In this paper, we address th...

View PDFchevron_right
Responsible AI by Design
Daniel Sierra

ArXiv, 2019

Recently, a lot of attention has been given to undesired consequences of Artificial Intelligence (AI), such as unfair bias leading to discrimination, or the lack of explanations of the results of AI systems. There are several important questions to answer before AI can be deployed at scale in our businesses and societies. Most of these issues are being discussed by experts and the wider communities, and it seems there is broad consensus on where they come from. There is, however, less consensus on, and experience with how to practically deal with those issues in organizations that develop and use AI, both from a technical and organizational perspective. In this paper, we discuss the practical case of a large organization that is putting in place a company-wide methodology to minimize the risk of undesired consequences of AI. We hope that other organizations can learn from this and that our experience contributes to making the best of AI while minimizing its risks.

View PDFchevron_right
Governing AI - attempting to herd cats? Introduction to the special issue on the Governance of Artificial Intelligence
Tim Büthe, Christian Djeffal

2022

Artificial Intelligence raises new, distinct governance challenges, as well as familiar governance challenges in novel ways. The governance of AI, moreover, is not an issue of distant futures, it is well underwayand it has characteristics akin to 'herding cats' with a mind of their own. This essay introduces the contributions to the special issue, situating them in broader political and social science literatures. It then provides a sketch of an interdisciplinary research agenda. It highlights the limits of 'explainable AI', makes the case for considering AI ethics and AI governance simultaneously, identifies as an underappreciated risk 'system effects' that arise from the introduction of AI applications, and calls for policymakers to consider both the opportunities and the risks of AI. Focusing on the (ab)uses of AI, rather than the complex, rapidly changing and hard-to-predict technology as such, might provide a superior approach to governing AI.

View PDFchevron_right
AI and the Conditions of Design: Towards A New Set of Design Ideals
Elisa Giaccardi

Proceedings of DRS

The five papers in the DRS 2022 track "AI and the Conditions of Design: Towards A New Set of Design Ideals" offer radical lenses to change the narrative around AI and open pathways towards pluralist digital futures, signaling redirections for experimenting with more inclusive and imaginative design practices.

View PDFchevron_right
A Layered Model for AI Governance
Michael Andrews

AI-based systems are "black boxes," resulting in massive information asymmetries between the developers of such systems and consumers and policymakers. In order to bridge this information gap, this article proposes a conceptual framework for thinking about governance for AI. M any sectors of society rapidly adopt digital technologies and big data, resulting in the quiet and often seamless integration of AI, autonomous systems, and algorith-mic decision-making into billions of human lives. 1,2 AI and algorithmic systems already guide a vast array of decisions in both private and public sectors. For example, private global platforms, such as Google and Facebook, use AI-based filtering algorithms to control access to information. AI algorithms that control self-driving cars must decide on how to weigh the safety of passengers and pedestrians. 3 Various applications, including security and safety decision-making systems, rely heavily on AI-based face recognition algorithms. And a recent study from Stanford University describes an AI algorithm that can deduce the sexuality of people on a dating site with up to 91 percent accuracy. 4 Voicing alarm at the capabilities of AI evidenced within this study, and as AI technologies move toward broader adoption, some voices in society have expressed concern about the unintended consequences and potential downsides of widespread use of these technologies. To ensure transparency, accountability, and explainability for the AI ecosystem, our governments , civil society, the private sector, and academia must be at the table to discuss gover-nance mechanisms that minimize the risks and possible downsides of AI and autonomous systems while harnessing the full potential of this technology. 5 Yet the process of designing a gov-ernance ecosystem for AI, autonomous systems, and algorithms is complex for several reasons. As researchers at the University of Oxford point out, 3 separate regulation solutions for decision-making algorithms, AI, and robot-ics could misinterpret legal and ethical challenges as unrelated, which is no longer accurate in today's systems. Algorithms, hardware, software, and data are always part of AI and autonomous systems. To regulate ahead of time is difficult for any kind of industry. Although AI technologies are evolving rapidly, they are still in the development stages. A global AI governance system must be flexible enough to accommodate cultural differences and bridge gaps across different national legal systems. While there are many approaches we can take to design a governance structure for AI, one option is to take inspiration from the development and evolution of governance structures that act on the Internet environment. Thus, here we discuss different issues associated with gov-ernance of AI systems, and introduce a conceptual framework for thinking about governance for AI, autonomous systems, and algorithmic decision-making processes. The Nature of AI Although AI-based applications are increasingly adopted in hospitals, courtrooms, schools, at home, and on the road to support (and in some instances, even guide) human decision-making,

View PDFchevron_right
AI Philosophy: Sources of Legitimacy to Analyze Artificial Intelligence
Olga Gil

2023

The following pages aim to reflect upon how to analyze the governance of artificial intelligence in a comparative perspective. In doing so, a dashboard is developed for the analysis and for eventual comparisons between democratic and non democratic regimes. The Gil dashboard of legitimacy would allow us to assess key features that determine the governance model for artificial intelligence at the national level, for local governments and other participant actors. The framework also allows us to appraise aims of the governance strategy, and what aims are left aside. The work opens windows to discuss 1) the complex reality of AI command and control 2) uncertainties about future society and the polity against AI development and 3) cultural values enshrined in countries' AI development. The theoretical framework could be of use to advance case studies globally, and comparative endeavors.

View PDFchevron_right

References (89)

  1. Interactive control over automated decisions Bayamlıoğlu (2021), Hirsch et al. (2017), Jewell (2018), Vaccaro et al. (2019, 2020) Explanations of system behavior Aler Tubella et al. (2020), Almada (2019), Bayamlıoğlu (2021), Brkan (2019), Crawford (2016), Edwards and Veale (2018), Henin and Le Métayer (2021), Hirsch et al. (2017), Lyons et al. (2021), Ploug and Holm (2020), Sarra (2020), Vaccaro et al. (2019, 2021)
  2. Human review and intervention requests Almada (2019), Bayamlıoğlu (2021), Brkan (2019), Edwards and Veale (2018), Elkin-Koren (2020), Henin and Le Métayer (2021), Hirsch et al. (2017), Lyons et al. (2021), Ploug and Holm (2020), Sarra (2020), Vaccaro et al. (2019, 2020, 2021), Walmsley (2021)
  3. Tools for scrutiny by subjects or third parties Aler Tubella et al. (2020), Almada (2019), Bayamlıoğlu (2021), Henin and Le Métayer (2021), Hirsch et al. (2017), Kariotis and Mir (2020), Lyons et al. (2021), Vaccaro et al. (2019, 2020)
  4. Table 4 Practices contributing to contestable AI Practice Sources Ex-ante safeguards Aler Tubella et al. (2020), Almada (2019), Bayamlıoğlu (2021), Brkan (2019), Edwards and Veale (2018), Henin and Le Métayer (2021), Hirsch et al. (2017), Kariotis and Mir (2020), Lyons et al. (2021), Sarra (2020), Walmsley (2021) Agonistic approaches to ML development Almada (2019), Henin and Le Métayer (2021), Kariotis and Mir (2020), König and Wenzelburger (2021), Vaccaro et al. (2019, 2021) Quality assurance during development Almada (2019), Elkin-Koren (2020), Hirsch et al. (2017), Kariotis and Mir (2020), Ploug and Holm (2020), Vaccaro et al. (2019, 2020), Walmsley (2021) Quality assurance after deployment Aler Tubella et al. (2020), Almada (2019), Bayamlıoğlu (2021), Hirsch et al. (2017), Vaccaro et al. (2020, 2021), Walmsley (2021) Risk mitigation strategies Hirsch et al. (2017), Lyons et al. (2021), Ploug and Holm (2020), Vaccaro et al. (2019, 2020)
  5. Third-party oversight Bayamlıoğlu (2021), Crawford (2016), Edwards and Veale (2018), Elkin-Koren (2020), Lyons et al. (2021), Vaccaro et al. (2019, 2020)
  6. References Aler Tubella, A., Theodorou, A., Dignum, V., et al. (2020). Contestable black boxes. In V. Gutiérrez- Basulto, T. Kliegr, A. Soylu, et al. (Eds.), Rules and reasoning (Vol. 12173). Springer.
  7. Almada, M. (2019). Human intervention in automated decision-making: Toward the construction of con- testable systems. In Proceedings of the 17th International Conference on Artificial Intelligence and Law, ICAIL 2019, pp 2-11
  8. Ananny, M., & Crawford, K. (2018). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media and Society, 20(3), 973-989.
  9. Applebee, A. N., & Langer, J. A. (1983). Instructional scaffolding: Reading and writing as natural lan- guage activities. Language Arts, 60(2), 168-175 http:// www. jstor. org/ stable/ 41961 447.
  10. Bayamlıoğlu, E. (2021). The right to contest automated decisions under the General Data Protection Reg- ulation: Beyond the so-called "right to explanation". Regulation and Governance.
  11. Binns, R., & Gallo, V. (2019). An overview of the Auditing Framework for Artificial Intelligence and its core components. https:// ico. org. uk/ about-the-ico/ news-and-events/ ai-blog-an-overv iew-of-the- audit ing-frame work-for-artifi cial-intel ligen ce-and-its-core-compo nents/
  12. Braun, M., Bleher, H., & Hummel, P. (2021). A leap of faith: Is there a formula for "trustworthy'' AI? Hastings Center Report, 51(3), 17-22. https:// doi. org/ 10. 1002/ hast. 1207.
  13. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychol- ogy, 3(2), 77-101.
  14. Brkan, M. (2019). Do algorithms rule the world? Algorithmic decision-making and data protection in the framework of the GDPR and beyond. International Journal of Law and Information Technology, 27(2), 91-121.
  15. Cavalcante Siebert, L., Lupetti, M. L., & Aizenberg, E., et al. (2022). Meaningful human control: Action- able properties for AI system development. AI and Ethics.
  16. Chiusi, F., Fischer, S., & Kayser-Bril, N., et al. (2020). Automating Society Report 2020. Tech. rep., Algorithm Watch, https:// autom ating socie ty. algor ithmw atch. org
  17. Cobbe, J., Lee, M. S. A., & Singh, J. (2021). Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, Virtual Event, pp 598- 609, https:// doi. org/ 10. 1145/ 34421 88. 34459 21
  18. Cowgill, B., & Tucker, C. (2017). Algorithmic bias: A counterfactual perspective. Working Paper: NSFTrustworthy Algorithms p 3. http:// trust worthy-algor ithms. org/ white papers/ Bo Cowgi ll. pdf
  19. Crawford, K. (2016). Can an algorithm be agonistic? Ten scenes from life in calculated publics. Science, Technology, & Human Values, 41(1), 77-92
  20. Crawford, K., Dobbe, R., & Dryer, T., et al. (2019). AI now 2019 report. Technical report, AI Now Insti- tute. https:// ainow insti tute. org/ AI_ Now_ 2019_ Report. html
  21. Davis, J. (2009). Design methods for ethical persuasive computing. In Proceedings of the 4th interna- tional conference on persuasive technology. Association for Computing Machinery, Persuasive '09.
  22. de Sio, F. S., & van den Hoven, J. (2018). Meaningful human control over autonomous systems: A philo- sophical account. Frontiers Robotics AI, 5, 1-14.
  23. Dorst, K., & Cross, N. (2001). Creativity in the design process: Co-evolution of problem-solution. Design Studies, 22(5), 425-437.
  24. Dourish, P. (2004). What we talk about when we talk about context. Personal and Ubiquitous Computing, 8(1), 19-30.
  25. Edwards, L., & Veale, M. (2018). Enslaving the algorithm: From a "right to an explanation'' to a "right to better decisions''? IEEE Security & Privacy, 16(3), 46-54.
  26. Elkin-Koren, N. (2020). Contesting algorithms: Restoring the public interest in content filtering by artifi- cial intelligence. Big Data & Society, 7(2), 205395172093,229.
  27. Franssen, M. (2015). Design for values and operator roles in sociotechnical systemsociotechnical sys- tems. In: van den Hoven J, Vermaas PE, van de Poel I (Eds.) Handbook of Ethics, Values, and Tech- nological Design. Springer, pp 117-149, https:// doi. org/ 10. 1007/ 978-94-007-6970-0_8
  28. Gebru, T., Morgenstern, J., & Vecchione, B., et al. (2020). Datasheets for datasets. arXiv: 1803. 09010 [cs]
  29. Geuens, J., Geurts, L., Swinnen, T. W., et al. (2018). Turning tables: A structured focus group method to remediate unequal power during participatory design in health care. In Proceedings of the 15th participatory design conference: Short papers, situated actions, workshops and tutorial -Volume 2. ACM, Hasselt and Genk, pp 1-5.
  30. Goodman, B. (2016). Economic models of (algorithmic) discrimination. In 29th conference on neural information processing systems
  31. Henin, C., & Le Métayer, D. (2021). Beyond explainability: Justifiability and contestability of algorith- mic decision systems. AI & Society
  32. Hildebrandt, M. (2017). Privacy as protection of the incomputable self: Agonistic machine learning. SSRN Electronic Journal 1-33.
  33. Hirsch, T., Merced, K., Narayanan, S., et al. (2017). Designing contestability: Interaction design, machine learning, and mental health. In DIS 2017 -Proceedings of the 2017 ACM conference on designing interactive systems. ACM Press, pp 95-99.
  34. Höök, K., Karlgren, J., & Waern, A., et al. (1998). A glass box approach to adaptive hypermedia. In: Brusilovsky P, Kobsa A, Vassileva J (Eds.) Adaptive hypertext and hypermedia. Springer, pp 143- 170, https:// doi. org/ 10. 1007/ 978-94-017-0617-9_6
  35. Höök, K., & Löwgren, J. (2012). Strong concepts: Intermediate-level knowledge in interaction design research. ACM Transactions on Computer-Human Interaction, 19(3), 1-18.
  36. Hutchinson, B., Smart, A., & Hanna, A., et al. (2021). Towards accountability for machine learning data- sets: Practices from software engineering and infrastructure. In Proceedings of the 2021 ACM con- ference on fairness, accountability, and transparency. ACM, Virtual Event Canada, pp 560-575.
  37. Jewell, M. (2018). Contesting the decision: Living in (and living with) the smart city. International Review of Law, Computers and Technology.
  38. Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389-399.
  39. Johnson, D. W. (2003). Social interdependence: Interrelationships among theory, research, and practice. American Psychologist, 58(11), 934-945.
  40. Kamarinou, D., Millard, C., & Singh, J. (2016). Machine learning with personal data. Queen Mary School of Law Legal Studies Research Paper, 1(247), 23.
  41. Kariotis, T., & Mir, D. J. (2020). Fighting back algocracy: The need for new participatory approaches to technology assessment. In Proceedings of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise -Volume 2. ACM, Manizales Colombia, pp 148-153.
  42. Katell, M., Young, M., & Dailey, D., et al. (2020). Toward situated interventions for algorithmic equity: Lessons from the field. In Proceedings of the 2020 conference on fairness, accountability, and transparency. Association for Computing Machinery, pp 45-55, https:// doi. org/ 10. 1145/ 33510 95. 33728 74
  43. Kluttz, D., Kohli, N., & Mulligan, D. K. (2018). Contestability and professionals: From explanations to engagement with algorithmic systems. SSRN Electronic Journal. https:// doi. org/ 10. 2139/ ssrn. 33118
  44. Kluttz, D. N., & Mulligan, D. K. (2019). Automated decision support technologies and the legal profes- sion. Berkeley Technology Law Journal, 34(3), 853. https:// doi. org/ 10. 15779/ Z3815 4DP7K.
  45. Kluttz, D. N., Mulligan, D. K., Mulligan, D. K., et al. (2019). Shaping our tools: Contestability as a means to promote responsible algorithmic decision making in the professions. SSRN Electronic Journal. https:// doi. org/ 10. 2139/ ssrn. 33118 94.
  46. König, P. D., & Wenzelburger, G. (2021). The legitimacy gap of algorithmic decision-making in the pub- lic sector: Why it arises and how to address it. Technology in Society, 67(101), 688.
  47. Kroes, P., Franssen, M., van de Poel, I., et al. (2006). Treating socio-technical systems as engineering systems: Some conceptual problems. Systems Research and Behavioral Science, 23(6), 803-814.
  48. Kroll, J. A., Barocas, S., Felten, E. W., et al. (2016). Accountable algorithms. U Pa L Rev, 165, 633.
  49. Leahu, L. (2016). Ontological surprises: A relational perspective on machine learning. In Proceedings of the 2016 ACM conference on designing interactive systems. ACM, pp 182-186
  50. Leydens, J. A., & Lucena, J. C. (2018). Engineering justice: Transforming engineering education and practice. IEEE PCS Professional Engineering Communication Series. Wiley.
  51. Löwgren, J., Gaver, B., & Bowers, J. (2013). Annotated Portfolios and other forms of intermediate-level knowledge. Interactions pp 30-34.
  52. Lyons, H., Velloso, E., & Miller, T. (2021). Conceptualising contestability: Perspectives on contesting algorithmic decisions. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1-25.
  53. Mahendran, A., & Vedaldi, A. (2015). Understanding deep image representations by inverting them. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)
  54. Matias, J. N., Johnson, A., Boesel, W. E., et al. (2015). Reporting, reviewing, and responding to harass- ment on twitter. https:// doi. org/ 10. 48550/ ARXIV. 1505. 03359
  55. Mendoza, I., & Bygrave, L. A. (2017). The right not to be subject to automated decisions based on profil- ing. In EU Internet Law. Springer, pp 77-98
  56. Methnani, L., Aler Tubella, A., & Dignum, V., et al. (2021). Let me take over: Variable autonomy for meaningful human control. Frontiers in Artificial Intelligence 4.
  57. Mitchell, M., Wu, S., Zaldivar, A., et al. (2019). Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency. ACM, pp 220-229
  58. Moher, D., Liberati, A., Tetzlaff, J., et al. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000,097.
  59. Mohseni, S. (2019). Toward design and evaluation framework for interpretable machine learning systems. In Proceedings of the 2019 AAAI/ACM conference on AI, ethics, and society. ACM, pp. 553-554.
  60. Morley, J., Floridi, L., Kinsey, L., et al. (2019). From what to how. An overview of AI ethics tools, methods and research to translate principles into practices. Science and Engineering Ethics, 26, 2141-2168.
  61. Myers West, S. (2018). Censored, suspended, shadowbanned: User interpretations of content moderation on social media platforms. New Media & Society, 20(11), 4366-4383.
  62. Nissenbaum, H. (2011). A contextual approach to privacy online. Daedalus, 140(4), 32-48.
  63. Norman, D. A., & Stappers, P. J. (2015). DesignX: Complex sociotechnical systems. She Ji: The Journal of Design, Economics, and Innovation, 1(2), 83-106.
  64. Novick, D. G., & Sutton, S. (1997). What is mixed-initiative interaction. In Proceedings of the AAAI spring symposium on computational models for mixed initiative interaction, p 12.
  65. Obrenović, Ž. (2011). Design-based research: What we learn when we engage in design of interactive systems. Interactions, 18(5), 56-59.
  66. Ouzzani, M., Hammady, H., Fedorowicz, Z., et al. (2016). Rayyan: A web and mobile app for systematic reviews. Systematic Reviews, 5(1), 210.
  67. Ploug, T., & Holm, S. (2020). The four dimensions of contestable AI diagnostics: A patient-centric approach to explainable AI. Artificial Intelligence in Medicine, 107(101), 901.
  68. Raji, I. D., Smart, A., White, R. N., et al. (2020). Closing the AI accountability gap: Defining an end-to- end framework for internal algorithmic auditing. FAT* 2020-Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency pp 33-44
  69. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why should i trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 1135-1144.
  70. Rouvroy, A. (2012). The end(s) of critique: Data-behaviourism vs. due-process. In: Hildebrandt M, De Vries E (Eds.) Privacy, due process and the computational turn. Philosophers of Law Meet Phi- losophers of Technology.
  71. Salehi, N., Teevan, J., Iqbal, S., et al. (2017). Communicating context to the crowd for complex writing tasks. In: Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing. ACM, Portland, pp 1890-1901.
  72. Sandvig, C., Hamilton, K., Karahalios, K., et al. (2014). Auditing algorithms: Research methods for detecting discrimination on internet platforms. In: Data and discrimination: Converting critical concerns into productive inquiry.
  73. Sarra, C. (2020). Put dialectics into the machine: Protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist, 20(3), 20200,003.
  74. Schot, J., & Rip, A. (1997). The past and future of constructive technology assessment. Technological Forecasting and Social Change, 54(2-3), 251-268.
  75. Selbst, A. D., & Barocas, S. (2018). The intuitive appeal of explainable machines. SSRN Electronic Journal.
  76. Shneiderman, B. (2020). Bridging the gap between ethics and practice: Guidelines for reliable, safe, and trustworthy human-centered AI systems. ACM Transactions on Interactive Intelligent Systems, 10(4), 1-31.
  77. Sloane, M., Moss, E., Awomolo, O., et al. (2020). Participation is not a design fix for machine learning. arXiv: 2007. 02423 [cs]
  78. Stolterman, E., & Wiberg, M. (2010). Concept-driven interaction design research. Human-Computer Interaction, 25(2), 95-118.
  79. Suchman, L. (2018). Corporate accountability. https:// robot futur es. wordp ress. com/ 2018/ 06/ 10/ corpo rate- accou ntabi lity/
  80. Tickle, A., Andrews, R., Golea, M., et al. (1998). The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks. IEEE Transactions on Neural Networks, 9(6), 1057-1068.
  81. Tonkinwise, C. (2016). The interaction design public intellectual. Interactions, 23(3), 24-25).
  82. Umbrello, S. (2021). Coupling levels of abstraction in understanding meaningful human control of auton- omous weapons: A two-tiered approach. Ethics and Information Technology, 23(3), 455-464.
  83. Vaccaro, K., Karahalios, K., Mulligan, D. K., et al. (2019). Contestability in algorithmic systems. In Con- ference companion publication of the 2019 on computer supported cooperative work and social computing. ACM, pp 523-527
  84. Vaccaro, K., Sandvig, C., & Karahalios, K. (2020). At the end of the day Facebook does what it wants: How users experience contesting algorithmic content moderation. In Proceedings of the ACM on human-computer interaction 4.
  85. Vaccaro, K., Xiao, Z., Hamilton, K., et al. (2021). Contestability for content moderation. In: Proceedings of the ACM on human-computer interaction, pp 1-28.
  86. van de Poel, I. (2020). Embedding values in artificial intelligence (AI) systems. Minds and machines Verbeek, P. P. (2015). Beyond interaction: A short introduction to mediation theory. Interactions, 22(3), 26-31.
  87. Verdiesen, I., Santoni de Sio, F., & Dignum, V. (2021). Accountability and control over autonomous weapon systems: A framework for comprehensive human oversight. Minds and Machines, 31(1), 137-163.
  88. Walmsley, J. (2021). Artificial intelligence and the value of transparency. AI & SOCIETY, 36(2), 585-595.
  89. Winner, L. (1980). Do artifacts have politics? Daedalus, 109(1), 121-136.

Related papers

Put Dialectics into the Machine: Protection against Automatic-decision-making through a Deeper Understanding of <i>Contestability by Design</i>
Claudio Sarra

Global jurist, 2020

This paper endorses the idea that the right to contest provided for by art. 22, § 3 GDPR, actually is the apex of a progressive set of tools the data subject has at his disposal to cope with automatic decisions and it should work as an architectural principle to create contestable systems. But in order to achieve that important role, it cannot be reduce to the right of human intervention, also provided for by art.22, § 3, nor to a generic opposition to the outcome of the automatic processing. Thus, drawing from a thorough analysis of the relationships among the rights included in art. 22, § 3 GDPR as well as from the juridical proper meaning of "contestatio", it is concluded that the right to contest has its own proper nature as a hybrid substantial-processual right that is able to give concrete shape to all the other rights indicated in art. 22, § 3, included the much discussed right to explanation.

View PDFchevron_right
More-Than-Human Design and AI
Elisa Giaccardi

Companion Publication of the 2020 ACM Designing Interactive Systems Conference, 2020

View PDFchevron_right
Contestable Camera Cars: A Speculative Design Exploration of Public AI That Is Open and Responsive to Dispute
Neelke Doorn

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Local governments increasingly use artifcial intelligence (AI) for automated decision-making. Contestability, making systems responsive to dispute, is a way to ensure they respect human rights to autonomy and dignity. We investigate the design of public urban AI systems for contestability through the example of camera cars: human-driven vehicles equipped with image sensors. Applying a provisional framework for contestable AI, we use speculative design to create a concept video of a contestable camera car. Using this concept video, we then conduct semi-structured interviews with 17 civil servants who work with AI employed by a large northwestern European city. The resulting data is analyzed using refexive thematic analysis to identify the main challenges facing the implementation of contestability in public AI. We describe how civic participation faces issues of representation, public AI systems should integrate with existing democratic practices, and cities must expand capacities for responsible AI development and operation. CCS CONCEPTS • Human-centered computing → Empirical studies in interaction design; • Applied computing → Computing in government.

View PDFchevron_right
Beyond black boxes: tackling artificial intelligence as a design material
Joanna Berzowska

DRS2018: Catalyst

The growth and development of artificial intelligence is fast becoming a powerful influence on the global economy and society overall. While advancements in AI are successfully reshaping many transactional contexts such as image search and purchase recommendations, the progression of this technology is somewhat slower in contexts that involve multidimensional experiences aimed at advancing human intelligence and the overall human condition. Issues of transparency, ethics, bias, and privacy are more and more emerging as topics of public debate, while the ultimate role of accountability, responsibility, and eventual consequence still requires some interrogative work. In this Conversation, we examine AI as a formative material to design with, often requiring a nuanced, pragmatic, or indeed sceptical mindset. We will use examples from our experiences in digital healthcare, smart textiles, and curatorial practice to seed a wider discussion about the form, function, and promise of AI in design practice.

View PDFchevron_right
Towards Contestable AI: Embedding Accountability in Indonesia's National AI Strategy
Ilma A Fiddien, Karina Dewi Oktaviani

ACM CSCW 2024 Workshop - From Stem to Stern: Contestability Along AI Value Chains, 2024

As Indonesia pursues an ambitious national strategy for artificial intelligence (AI), ensuring the development of contestable and accountable AI systems is crucial. This paper critically examines the current state of AI contestability in Indonesia's policy landscape, identifying challenges, opportunities, and recommendations. Through document analysis, comparative review, and application of the Contestable AI by Design framework, we find that while Indonesia's AI strategy shows promising intent, gaps remain in institutionalizing contestability principles. To address these gaps, we propose a policy roadmap spanning regulatory reforms, public empowerment, and cross-sectoral partnerships. Key recommendations include mandating Algorithmic Impact Assessments, establishing AI ombudsperson offices, and fostering multi-stakeholder collaboration. We argue that by embedding contestability safeguards and adaptive governance, Indonesia can foster an inclusive and socially responsive AI ecosystem. This paper advances the research agenda on AI governance in the Global South. It offers policymakers and practitioners diagnostic tools and pathways for aligning Indonesia's AI ambitions with democratic accountability, contributing to the transnational dialogue on ethical AI.

View PDFchevron_right

Related topics

  • Philosophy and Religious Studies
  • Psychology and Cognitive Sciences
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved