Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Conclusion
References
All Topics
Computer Science
Artificial Intelligence

The Pragmatic Turn in Explainable Artificial Intelligence (XAI)

Profile image of Andrés PáezAndrés Páez

2019, MInds and Machines, 29(3), 441-459

https://doi.org/10.1007/S11023-019-09502-W
visibility

…

description

28 pages

link

1 file

Abstract

In this paper I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. Intuitively, the purpose of providing an explanation of a model or a decision is to make it understandable to its stake-holders. But without a previous grasp of what it means to say that an agent understands a model or a decision, the explanatory strategies will lack a well-defined goal. Aside from providing a clearer objective for XAI, focusing on understanding also allows us to relax the factivity condition on explanation, which is impossible to fulfill in many machine learning models, and to focus instead on the pragmatic conditions that determine the best fit between a model and the methods and devices deployed to understand it. After an examination of the different types of understanding discussed in the philosophical and psychological literature, I conclude that interpretative or approximation models not only provide the best way to achieve the objectual understanding of a machine learning model, but are also a necessary condition to achieve post-hoc interpretability. This conclusion is partly based on the shortcomings of the purely functionalist approach to post-hoc interpretability that seems to be predominant in most recent literature.

Related papers

A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable
Mark Neerincx

Explainable and Transparent AI and Multi-Agent Systems, 2021

Because of recent and rapid developments in Artificial Intelligence (AI), humans and AI-systems increasingly work together in human-agent teams. However, in order to effectively leverage the capabilities of both, AI-systems need to be understandable to their human teammates. The branch of eXplainable AI (XAI) aspires to make AIsystems more understandable to humans, potentially improving humanagent teamwork. Unfortunately, XAI literature suffers from a lack of agreement regarding the definitions of and relations between the four key XAI-concepts: transparency, interpretability, explainability, and understandability. Inspired by both XAI and social sciences literature, we present a two-dimensional framework that defines and relates these concepts in a concise and coherent way, yielding a classification of three types of AI-systems: incomprehensible, interpretable, and understandable. We also discuss how the established relationships can be used to guide future research into XAI, and how the framework could be used during the development of AI-systems as part of human-AI teams.

View PDFchevron_right
Explainable AI
Britta Wrede

KI - Künstliche Intelligenz

View PDFchevron_right
The Role of Human Knowledge in Explainable AI
Marco Brambilla

Data

As the performance and complexity of machine learning models have grown significantly over the last years, there has been an increasing need to develop methodologies to describe their behaviour. Such a need has mainly arisen due to the widespread use of black-box models, i.e., high-performing models whose internal logic is challenging to describe and understand. Therefore, the machine learning and AI field is facing a new challenge: making models more explainable through appropriate techniques. The final goal of an explainability method is to faithfully describe the behaviour of a (black-box) model to users who can get a better understanding of its logic, thus increasing the trust and acceptance of the system. Unfortunately, state-of-the-art explainability approaches may not be enough to guarantee the full understandability of explanations from a human perspective. For this reason, human-in-the-loop methods have been widely employed to enhance and/or evaluate explanations of machine...

View PDFchevron_right
Explaining Explanations in AI
Chris Russell, Brent Mittelstadt

Recent work on interpretability in machine learning and AI has focused on the building of simplified models that approximate the true criteria used to make decisions. These models are a useful pedagogical device for teaching trained professionals how to predict what decisions will be made by the complex system, and most importantly how the system might break. However, when considering any such model it's important to remember Box's maxim that "All models are wrong but some are useful." We focus on the distinction between these models and explanations in philosophy and sociology. These models can be understood as a "do it yourself kit" for explanations, allowing a practitioner to directly answer "what if questions" or generate contrastive explanations without external assistance. Although a valuable ability, giving these models as explanations appears more difficult than necessary, and other forms of explanation may not have the same trade-offs. We contrast the different schools of thought on what makes an explanation, and suggest that machine learning might benefit from viewing the problem more broadly.

View PDFchevron_right
A global taxonomy of interpretable AI: unifying the terminology for the technical and social sciences
Lode Lauwaert

AI and Transparency, 2023

Since its emergence in the 1960s, Artificial Intelligence (AI) has grown to conquer many technology products and their fields of application. Machine learning, as a major part of the current AI solutions, can learn from the data and through experience to reach high performance on various tasks. This growing success of AI algorithms has led to a need for interpretability to understand opaque models such as deep neural networks. Various requirements have been raised from different domains, together with numerous tools to debug, justify outcomes, and establish the safety, fairness and reliability of the models. This variety of tasks has led to inconsistencies in the terminology with, for instance, terms such as interpretable, explainable and transparent being often used interchangeably in methodology papers. These words, however, convey different meanings and are "weighted" differently across domains, for example in the technical and social sciences. In this paper, we propose an overarching terminology of interpretability of AI systems that can be referred to by the technical developers as much as by the social sciences community to pursue clarity and efficiency in the definition of regulations for ethical and reliable AI development. We show how our taxonomy and definition of interpretable AI differ from the ones in previous research and how they apply with high versatility to several domains and use cases, proposing a-highly needed-standard for the communication among interdisciplinary areas of AI.

View PDFchevron_right
Reviewing the Need for Explainable Artificial Intelligence (xAI)
Arisa Shollo

Proceedings of the Annual Hawaii International Conference on System Sciences, 2021

The diffusion of artificial intelligence (AI) applications in organizations and society has fueled research on explaining AI decisions. The explainable AI (xAI) field is rapidly expanding with numerous ways of extracting information and visualizing the output of AI technologies (e.g. deep neural networks). Yet, we have a limited understanding of how xAI research addresses the need for explainable AI. We conduct a systematic review of xAI literature on the topic and identify four thematic debates central to how xAI addresses the black-box problem. Based on this critical analysis of the xAI scholarship we synthesize the findings into a future research agenda to further the xAI body of knowledge.

View PDFchevron_right
Explainable Artificial Intelligence in Machine Learning
Gabriel Banaggia

Capstone Project Report, 2024

Explainable Artificial Intelligence aims to assist in opening the blackbox of opaque algorithms, especially present in machine learning systems, so they can earn due trustworthiness. This work employs a systematic literature review in order to understand how the concepts of explainability and interpretability, among others, are defined in this context. It offers a structured vocabulary to better comprehend and classify artificial intelligence systems and models, as well as the techniques used to make them more intelligible. Inspired by Human-Centered Computing notions, it becomes crucial to consider the audience to which explanations are owed, emphasizing that it is composed of a community of heterogeneous users.

View PDFchevron_right
XAI Handbook: Towards a Unified Framework for Explainable AI
Sheraz Ahmed

ArXiv, 2021

The field of explainable AI (XAI) has quickly become a thriving and prolific community. However, a silent, recurrent and acknowledged issue in this area is the lack of consensus regarding its terminology. In particular, each new contribution seems to rely on its own (and often intuitive) version of terms like “explanation” and “interpretation”. Such disarray encumbers the consolidation of advances in the field towards the fulfillment of scientific and regulatory demands e.g., when comparing methods or establishing their compliance w.r.t. biases and fairness constraints. We propose a theoretical framework that not only provides concrete definitions for these terms, but it also outlines all steps necessary to produce explanations and interpretations. The framework also allows for existing contributions to be recontextualized such that their scope can be measured, thus making them comparable to other methods. We show that this framework is compliant with desiderata on explanations, on ...

View PDFchevron_right
Explainable AI and the philosophy and practice of explanation
Kieron O'Hara

Computer Law & Security Review, 2020

Considerations of the nature of explanation and the law are brought together to argue that computed accounts of AI systems' outputs cannot function on their own as explanations of decisions informed by AI. The important context for this inquiry is set by Article 22(3) of GDPR. The paper looks at the question of what an explanation is from the point of view of the philosophy of science -i.e. it asks not what counts as explanatory in legal terms, or what an AI system might compute using provenance metadata, but rather what explanation as a social practice consists in, arguing that explanation is an illocutionary act, and that it should be considered as a process, not a text. It cannot therefore be computed, although computed accounts of AI systems are likely to be important inputs to the explanatory process.

View PDFchevron_right
A Literature Review and Research Agenda on Explainable Artificial Intelligence (XAI
Srinivas Publication

Purpose: When Artificial Intelligence is penetrating every walk of our affairs and business, we face enormous challenges and opportunities to adopt this revolution. Machine learning models are used to make the important decisions in critical areas such as medical diagnosis, financial transactions. We need to know how they make decisions to trust the systems powered by these models. However, there are challenges in this area of explaining predictions or decisions made by machine learning model. Ensembles like Random Forest, Deep learning algorithms make the matter worst in terms of explaining the outcomes of decision even though these models produce more accurate results. We cannot accept the black box nature of AI models as we encounter the consequences of those decisions. In this paper, we would like to open this Pandora box and review the current challenges and opportunities to explain the decisions or outcome of AI model. There has been lot of debate on this topic with headlines as Explainable Artificial Intelligence (XAI), Interpreting ML models, Explainable ML models etc. This paper does the literature review of latest findings and surveys published in various reputed journals and publications. Towards the end, we try to bring some open research agenda in these findings and future directions. Methodology: The literature survey on the chosen topic has been exhaustively covered to include fundamental concepts of the research topic. Journals from multiple secondary data sources such as books and research papers published in various reputable publications which are relevant for the work were chosen in the methodology. Findings/Result: While there are no single approaches currently solve the explainable ML model challenges, some model algorithms such as Decision Trees, KNN algorithm provides built in interpretations. However there is no common approach and they cannot be used in all the problems. Developing model specific interpretations will be complex and difficult for the user to make them adopt. Model specific explanations may lead to multiple explanations on same predictions which will lead to ambiguity of the outcome. In this paper, we have conceptualized a common approach to build explainable models that may fulfill current challenges of XAI. Originality: After the literature review, the knowledge gathered in the form of findings were used to model a theoretical framework for the research topic. Then concerted effort was made to develop a conceptual model to support the future research work. Paper Type: Literature Review.

View PDFchevron_right

References (68)

  1. Achinstein, P. (1983). The nature of explanation. New York: Oxford University Press.
  2. Allahyari, H., & Lavesson, N. (2011). User-oriented assessment of classification model understandability. Proceedings of the 11th Scandinavian Conference on Artificial Intelligence. Amsterdam: IOS Press.
  3. Carter, J. A., & Gordon, E. C. (2016). Objectual understanding, factivity and belief. In: M. Grajner & P. Schmechtig (Eds.), Epistemic reasons, norms and goals (pp. 423-442). Berlin: De Gruyter.
  4. Caruana, R., Kangarloo, H., Dionisio, J. D. N., Sinha, U., & Johnson, D. (1999). Case-based explanations of non-case-based learning methods. In Proceedings of the AMIA Symposium (p. 212). American Medical Informatics Association.
  5. Castelfranchi, C., & Tan, Y-H. (Eds.). (2001). Trust and deception in virtual societies (pp. 157-168). Dordrecht: Kluwer Academic Publisher.
  6. Darwin, C. (1860/1903). Letter to Henslow, May 1860. In F. Darwin (Ed.), More letters of Charles Darwin, vol. I. New York: D. Appleton.
  7. De Graaf, M. M., & Malle, B. F. (2017). How people explain action (and Autonomous Intelligent Systems should too). In AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction (pp. 19-26). Palo Alto: The AAAI Press.
  8. de Regt, H. W., & Dieks, D. (2005). A contextual approach to scientific understanding. Synthese, 144, 137-170.
  9. de Regt, H. W., Leonelli, S., & Eigner, K. (Eds.). (2009). Scientific understanding: Philosophical perspectives. Pittsburgh: University of Pittsburgh Press.
  10. Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
  11. Ehsan, U., Harrison, B., Chan, L., & Riedl, M. O. (2018). Rationalization: A neural machine translation approach to generating natural language explanations. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (pp. 81-87). New York: ACM.
  12. Elgin, C. Z. (2004). True enough. Philosophical Issues, 14, 113-131
  13. Elgin, C. Z. (2007). Understanding and the facts. Philosophical Studies,132, 33-42.
  14. Elgin, C. Z. (2008). Exemplification, idealization, and scientific understanding. In M. Suárez (Ed.), Fictions in science: Philosophical essays on modelling and idealization (pp. 77- 90). London: Routledge.
  15. Elgin, C. Z. (2017). True enough. Cambridge: MIT Press.
  16. Falcone R., & Castelfranchi, C. (2001). Social trust: A cognitive approach. In C. Castelfranchi, & Tan, Y.-H. (Eds), Trust and deception in virtual societies (pp. 55-90). Springer: Dordrecht.
  17. Freitas, A. A. (2014). Comprehensible classification models: a position paper. ACM SIGKDD explorations newsletter, 15(1), 1-10.
  18. Fürnkranz, J., Kliegr, T., & Paulheim, H. (2018). On cognitive preferences and the plausibility of rule-based models. arXiv preprint arXiv:1803.01316.
  19. Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2019). Explaining explanation. An overview of interpretability of machine learning. arXiv preprint arXiv:1806.00069v3.
  20. Greco, J. (2010). Achieving knowledge. Cambridge: Cambridge University Press.
  21. Greco, J. (2012). Intellectual virtues and their place in philosophy. In C. Jäger & W. Löffler (Eds.), Epistemology: Contexts, values, disagreement: Proceedings of the 34 th International Wittgenstein Symposium (pp. 117-130). Heusenstamm: Ontos.
  22. Grimm, S. R. (2006). Is understanding a species of knowledge? British Journal for the Philosophy of Science, 57, 515-535.
  23. Grimm, S. R. (2011). Understanding. In S. Bernecker & D. Pritchard (Eds.), The Routledge companion to epistemology (pp. 84-94). New York: Routledge.
  24. Grimm, S. R. (2014). Understanding as knowledge of causes. In A. Fairweather (Ed.), Virtue epistemology naturalized: Bridges between virtue epistemology and philosophy of science. Dordrecht: Springer.
  25. Grimm, S. R. (Ed.). (2018). Making sense of the world: New essays on the philosophy of understanding. New York: Oxford University Press.
  26. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51(5), Article 93.
  27. Hempel, C. G. (1965). Aspects of scientific explanation. New York: The Free Press.
  28. Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., & B. Baesens (2011). An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decision Support Systems, 51(1), 141-154.
  29. Kelemen, D. (1999). Functions, goals, and intentions: Children's teleological reasoning about objects. Trends in Cognitive Science, 12, 461-468.
  30. Khalifa, K. (2012). Inaugurating understanding or repackaging explanation. Philosophy of Science, 79, 15-37.
  31. Kim, B. (2015). Interactive and interpretable machine learning models for human machine collaboration. PhD thesis, Massachusetts Institute of Technology.
  32. Kliegr, T., Bahník, Š., & Fürnkranz, J. (2018). A review of possible effects of cognitive biases on interpretation of rule-based machine learning models. arXiv preprint arXiv:1804.02969.
  33. Krening, S., Harrison, B., Feigh, K., Isbell, C., Riedl, M., & Thomaz, A. (2016). Learning from explanations using sentiment and advice in RL. IEEE Transactions on Cognitive and Developmental Systems, 9(1), 44-55.
  34. Kvanvig, J. (2003). The value of knowledge and the pursuit of understanding. New York: Cambridge University Press.
  35. Kvanvig, J. (2009). Response to critics. In A. Haddock, A. Millar, & D. Pritchard (Eds.), Epistemic value (pp. 339-351). New York: Oxford University Press.
  36. Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S., & Doshi-Velez, F. (2019). An Evaluation of the Human-Interpretability of Explanation. arXiv preprint arXiv:1902.00006.
  37. Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K. R. (2019). Unmasking Clever Hans predictors and assessing what machines really learn. Nature communications, 10(1), 1096.
  38. Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2017). Fair, transparent, and accountable algorithmic decision-making processes: The premise, the proposed solutions, and the open challenges. Philosophy & Technology, 31, 611-627.
  39. Lewis, D. K. (1986). Causal explanation. In Philosophical papers, vol. II (pp. 214-240). New York: Oxford University Press.
  40. Lipton, P. (2009). Understanding without explanation. In H. W. de Regt, S. Leonelli, & K. Eigner (Eds.), Scientific understanding: Philosophical perspectives (pp. 43-63). Pittsburgh: University of Pittsburgh Press.
  41. Lipton, Z. C. (2016). The mythos of model interpretability. arXiv preprint arXiv:1606.03490.
  42. Lombrozo, T. & Gwynne, N. Z. (2014). Explanation and inference: Mechanistic and functional explanations guide property generalization. Frontiers in Human Neuroscience, 8, 700.
  43. Lombrozo, T., & Wilkenfeld, D. A. (forthcoming). Mechanistic vs. functional understanding. In S. R. Grimm (Ed.), Varieties of understanding: New perspectives from philosophy, psychology, and theology. New York: Oxford University Press.
  44. McAuley, J., & Leskovec, J. (2013). Hidden factors and hidden topics: understanding rating dimensions with review text. In Proceedings of the 7th ACM conference on recommender systems (pp. 165-172). New York: ACM.
  45. Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267, 1-18.
  46. Miller, T., Howe, P., & Sonenberg, L. (2017). Explainable AI: Beware of inmates running the asylum. In Proceedings of the IJCAI-17 Workshop on Explainable AI (XAI) (pp. 36- 42). Accessed March 10, 2019 http://www.intelligentrobots.org/files/IJCAI2017/IJCAI- 17_XAI_WS_Proceedings.pdf
  47. Mittelstadt, B., Russell, C., & Wachter, S. (2019). Explaining explanations in AI. In Proceedings of the Conference on Fairness, Accountability, and Transparency (pp. 279- 288). New York: ACM.
  48. Mizrahi, M. (2012). Idealizations and scientific understanding. Philosophical Studies, 160, 237-252.
  49. Páez, A. (2006). Explanations in K. An analysis of explanation as a belief revision operation. Oberhausen: Athena Verlag.
  50. Páez, A. (2009). Artificial explanations: The epistemological interpretation of explanation in AI. Synthese, 170, 131-146.
  51. Pazzani, M. (2000). Knowledge discovery from data? IEEE Intelligent Systems, 15(2), 10- 13.
  52. Piltaver, R., Luštrek, M., Gams, M., & Martinčić-Ipšić, S. (2016). What makes classification trees comprehensible? Expert Systems with Applications: An International Journal, 62(C), 333-346.
  53. Potochnik, A. (2017). Idealization and the aims of science. Chicago: University of Chicago Press.
  54. Pritchard, D. (2008). Knowing the answer, Understanding and epistemic value. Grazer Philosophische Studien, 77, 325-339.
  55. Pritchard, D. (2014). Knowledge and understanding. In A. Fairweather (Ed.), Virtue scientia: Bridges between virtue epistemology and philosophy of science (pp. 315-328). Dordrecht: Springer.
  56. Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., & Lawrence, N. D. (Eds.). (2009). Dataset shift in machine learning. Cambridge: MIT Press.
  57. Reiss, J. (2012). The explanation paradox. Journal of Economic Methodology, 19, 43-62.
  58. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why should I trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135-1144). New York: ACM.
  59. Salmon, W. C. (1971). Statistical explanation. In W. C. Salmon (Ed.), Statistical explanation and statistical relevance. Pittsburgh: Pittsburgh University Press.
  60. Salmon, W. C. (1984). Scientific explanation and the causal structure of the world. Princeton: Princeton University Press.
  61. Samek, W., Wiegand, T., & Müller, K. R. (2017). Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296.
  62. Strevens, M. (2013). No understanding without explanation. Studies in the History and Philosophy of Science, 44, 510-515.
  63. van Fraassen, B. (1980). The scientific image. Oxford: Clarendon Press.
  64. Wilkenfeld, D. (2013). Understanding as representation manipulability. Synthese, 190, 997- 1016.
  65. Woodward, J. (2003). Making things happen. A theory of causal explanation. New York: Oxford University Press.
  66. Zagzebski, L. (2001). Recovering understanding. In M. Steup (Ed.), Knowledge, truth, and duty: Essays on epistemic justification, responsibility, and virtue. New York: Oxford University Press.
  67. Zagzebski, L. (2009). On epistemology. Belmont: Wadsworth.
  68. Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. In 13 th European Conference on Computer Vision ECCV 2014 (pp. 818-833). Cham: Springer.

Related papers

A7.Developing interpretable models and techniques for explainable
neelam kumar

The Scientific Temper , 2023

The rapid proliferation of artificial intelligence (AI) technologies across various industries and decision-making processes has undeniably transformed the way of approaching complex problems and tasks. AI systems have proven their prowess in areas such as healthcare, finance, and autonomous systems, revolutionizing how decisions are made. Nevertheless, this proliferation of AI has raised critical concerns regarding the transparency, accountability, and fairness of these systems, as many of the state-of-the-art AI models often resemble complex black boxes. These intricate models, particularly deep learning neural networks, harbor non-linear relationships that are difficult for human users to decipher, thereby raising concerns about bias, fairness, and overall trustworthiness in AI-driven decisions. The urgency of this issue is underscored by the realization that AI should not merely be accurate; it should also be interpretable. Explainable AI (XAI) has emerged as a vital field of research, emphasizing the development of models and techniques that render AI systems comprehensible and transparent in their decision-making processes. This paper investigates into the relevance and significance of XAI across various domains, including healthcare, finance, and autonomous systems, where the ability to understand the rationale behind AI decisions is paramount. In healthcare, where AI assists in diagnosis and treatment, the interpretability of AI models is crucial for clinicians to make informed decisions. In finance, applications like credit scoring and investment analysis demand transparent AI to ensure fairness and accountability. In the realm of autonomous systems, transparency is indispensable to guarantee safety and compliance with regulations. Moreover, government agencies in areas such as law enforcement and social services require interpretable AI to maintain ethical standards and accountability. This paper also highlights the diverse array of research efforts in the XAI domain, spanning from model-specific interpretability methods to more general approaches aimed at unveiling complex AI models. Interpretable models like decision trees and rule-based systems have gained attention for their inherent transparency, while integrating explanation layers into deep neural networks strives to balance accuracy with interpretability. The study emphasizes the significance of this burgeoning field in bridging the gap between AI’s advanced capabilities and human users’ need for comprehensible AI systems. It seeks to contribute to this field by exploring the design, development, and practical applications of interpretable AI models and techniques, with the ultimate goal of enhancing the trust and understanding of AI-driven decisions.

View PDFchevron_right
Notions of explainability and evaluation approaches for explainable artificial intelligence
Giulia Vilone, Luca Longo

Explainable Artificial Intelligence (XAI) has experienced a significant growth over the last few years. This is due to the widespread application of machine learning, particularly deep learning, that has led to the development of highly accurate models that lack explainability and interpretability. A plethora of methods to tackle this problem have been proposed, developed and tested, coupled with several studies attempting to define the concept of explainability and its evaluation. This systematic review contributes to the body of knowledge by clustering all the scientific studies via a hierarchical system that classifies theories and notions related to the concept of explainability and the evaluation approaches for XAI methods. The structure of this hierarchy builds on top of an exhaustive analysis of existing taxonomies and peer-reviewed scientific material. Findings suggest that scholars have identified numerous notions and requirements that an explanation should meet in order to be easily understandable by end-users and to provide actionable information that can inform decision making. They have also suggested various approaches to assess to what degree machine-generated explanations meet these demands. Overall, these approaches can be clustered into human-centred evaluations and evaluations with more objective metrics. However, despite the vast body of knowledge developed around the concept of explainability, there is not a general consensus among scholars on how an explanation should be defined, and how its validity and reliability assessed. Eventually, this review concludes by critically discussing these gaps and limitations, and it defines future research directions with explainability as the starting component of any artificial intelligent system.

View PDFchevron_right
What do we want from Explainable Artificial Intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research
Kevin Baum

Artificial Intelligence

Previous research in Explainable Artificial Intelligence (XAI) suggests that a main aim of explainability approaches is to satisfy specific interests, goals, expectations, needs, and demands regarding artificial systems (we call these 'stakeholders' desiderata') in a variety of contexts. However, the literature on XAI is vast, spreads out across multiple largely disconnected disciplines, and it often remains unclear how explainability approaches are supposed to achieve the goal of satisfying stakeholders' desiderata. This paper discusses the main classes of stakeholders calling for explainability of artificial systems and reviews their desiderata.

View PDFchevron_right
Innovations in Explainable AI : Bridging the Gap Between Complexity and Understanding
Keshav Jena, International Journal of Scientific Research in Computer Science, Engineering and Information Technology IJSRCSEIT

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2023

The integration of Artificial Intelligence (AI) into various domains has witnessed remarkable advancements, yet the opacity of complex AI models poses challenges for widespread acceptance and application. This research paper delves into the field of Explainable AI (XAI) and explores innovative strategies aimed at bridging the gap between the intricacies of advanced AI algorithms and the imperative for human comprehension. We investigate key developments, including interpretable model architectures, local and visual explanation techniques, natural language explanations, and model-agnostic approaches. Emphasis is placed on ethical considerations to ensure transparency and fairness in algorithmic decision-making. By surveying and analyzing these innovations, this research contributes to the ongoing discourse on making AI systems more accessible, accountable, and trustworthy, ultimately fostering a harmonious collaboration between humans and intelligent machines in an increasingly AI-driven world.

View PDFchevron_right
What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
Sarah Schulz

ArXiv, 2017

We characterize three notions of explainable AI that cut across research fields: opaque systems that offer no insight into its algo- rithmic mechanisms; interpretable systems where users can mathemat- ically analyze its algorithmic mechanisms; and comprehensible systems that emit symbols enabling user-driven explanations of how a conclusion is reached. The paper is motivated by a corpus analysis of NIPS, ACL, COGSCI, and ICCV/ECCV paper titles showing differences in how work on explainable AI is positioned in various fields. We close by introducing a fourth notion: truly explainable systems, where automated reasoning is central to output crafted explanations without requiring human post processing as final step of the generative process.

View PDFchevron_right

Related topics

  • Artificial Intelligence
  • Machine Learning
  • Scientific Models
  • XAI - Explainable Artificial Int...
  • Model Transparency

    • Cited by

    Explainable Artificial Intelligence: Concepts, Applications, Research Challenges and Visions
    Randy Goebel

    Lecture Notes in Computer Science, 2020

    The development of theory, frameworks and tools for Explainable AI (XAI) is a very active area of research these days, and articulating any kind of coherence on a vision and challenges is itself a challenge. At least two sometimes complementary and colliding threads have emerged. The first focuses on the development of pragmatic tools for increasing the transparency of automatically learned prediction models, as for instance by deep or reinforcement learning. The second is aimed at anticipating the negative impact of opaque models with the desire to regulate or control impactful consequences of incorrect predictions, especially in sensitive areas like medicine and law. The formulation of methods to augment the construction of predictive models with domain knowledge can provide support for producing human understandable explanations for predictions. This runs in parallel with AI regulatory concerns, like the European Union General Data Protection Regulation, which sets standards for the production of explanations from automated or semi-automated decision making. Despite the fact that all this research activity is the growing acknowledgement that the topic of explainability is essential, it is important to recall that it is also among the oldest fields of computer science. In fact, early AI was re-traceable, interpretable, thus understandable by and explainable to humans. The goal of this research is to articulate the big picture ideas and their role in advancing the development of XAI systems, to acknowledge their historical roots, and to emphasise the biggest challenges to moving forward.

    View PDFchevron_right
    Sources of Understanding in Supervised Machine Learning Models
    Paulo Pirozelli

    Philosophy & Technology, 2022

    In the last decades, supervised machine learning has seen the widespread growth of highly complex, non-interpretable models, of which deep neural networks are the most typical representative. Due to their complexity, these models have showed an outstanding performance in a series of tasks, as in image recognition and machine translation. Recently, though, there has been an important discussion over whether those non-interpretable models are able to provide any sort of understanding whatsoever. For some scholars, only interpretable models can provide understanding. More popular, however, is the idea that understanding can come from a careful analysis of the dataset or from the model's theoretical basis. In this paper, I wish to examine the possible forms of obtaining understanding of such non-interpretable models. Two main strategies for providing understanding are analyzed. The first involves understanding without interpretability, either through external evidence for the model's inner functioning or through analyzing the data. The second is based on the artificial production of interpretable structures, through three main forms: post hoc models, hybrid models, and quasi-interpretable structures. Finally, I consider some of the conceptual difficulties in the attempt to create explanations for these models, and their implications for understanding.

    View PDFchevron_right
    The State Space of Artificial Intelligence
    Holger Lyre

    Minds and Machines, 2020

    The goal of the paper is to develop and propose a general model of the state space of AI. Given the breathtaking progress in AI research and technologies in recent years, such conceptual work is of substantial theoretical interest. The present AI hype is mainly driven by the triumph of deep learning neural networks. As the distinguishing feature of such networks is the ability to self-learn, self-learning is identified as one important dimension of the AI state space. Another dimension is recognized as generalization, the possibility to go over from specific to more general types of problems. A third dimension is semantic grounding. Our overall analysis connects to a number of known foundational issues in the philosophy of mind and cognition: the blockhead objection, the Turing test, the symbol grounding problem, the Chinese room argument, and use theories of meaning. It shall finally be argued that the dimension of grounding decomposes into three sub-dimensions. And the dimension o...

    View PDFchevron_right
    Automated opioid risk scores: a case for machine learning-induced epistemic injustice in healthcare
    Giorgia Pozzi

    Ethics and Information Technology

    Artificial intelligence-based (AI) technologies such as machine learning (ML) systems are playing an increasingly relevant role in medicine and healthcare, bringing about novel ethical and epistemological issues that need to be timely addressed. Even though ethical questions connected to epistemic concerns have been at the center of the debate, it is going unnoticed how epistemic forms of injustice can be ML-induced, specifically in healthcare. I analyze the shortcomings of an ML system currently deployed in the USA to predict patients’ likelihood of opioid addiction and misuse (PDMP algorithmic platforms). Drawing on this analysis, I aim to show that the wrong inflicted on epistemic agents involved in and affected by these systems’ decision-making processes can be captured through the lenses of Miranda Fricker’s account of hermeneutical injustice. I further argue that ML-induced hermeneutical injustice is particularly harmful due to what I define as an automated hermeneutical appro...

    View PDFchevron_right
    A Redhead Walks into a Bar: Experiences of Writing Fiction with Artificial Intelligence
    Jeffrey Bardzell

    25th International Academic Mindtrek conference

    Human creativity has been often aided and supported by artificial tools, spanning traditional tools such as ideation cards, pens, and paper, to computed and software. Tools for creativity are increasingly using artificial intelligence to not only support the creative process, but also to act upon the creation with a higher level of agency. This paper focuses on writing fiction as a creative activity and explores human-AI co-writing through a research product, which employs a natural language processing model, the Generative Pre-trained Transformer 3 (GPT-3), to assist the co-authoring of narrative fiction. We report on two progressive-not comparative-autoethnographic studies to attain our own creative practices in light of our engagement with the research product: (1) a co-writing activity initiated by basic textual prompts using basic elements of narrative and (2) a co-writing activity initiated by more advanced textual prompts using elements of narrative, including dialects and metaphors undertaken by one of the authors of this paper who has doctoral training in literature. In both studies, we quickly came up against the limitations of the system; then, we repositioned our goals and practices to maximize our chances of success. As a result, we discovered not only limitations but also hidden capabilities, which not only altered our creative practices and outcomes, but which began to change the ways we were relating to the AI as collaborator. CCS CONCEPTS • Human-centered computing → HCI design and evaluation methods.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved