Academia.eduAcademia.edu

Outline

Title
Abstract
Figures
Related Work
Universal Context for Xai Methods
Conclusions
Recommendations
References
All Topics
Computer Science

XAI Handbook: Towards a Unified Framework for Explainable AI

Profile image of Sebastian PalacioSebastian Palacio

2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

Abstract
sparkles

AI

The field of explainable AI (XAI) has quickly become a thriving and prolific community. However, a silent, recurrent and acknowledged issue in this area is the lack of consensus regarding its terminology. In particular, each new contribution seems to rely on its own (and often intuitive) version of terms like "explanation" and "interpretation". Such disarray encumbers the consolidation of advances in the field towards the fulfillment of scientific and regulatory demands e.g., when comparing methods or establishing their compliance w.r.t. biases and fairness constraints. We propose a theoretical framework that not only provides concrete definitions for these terms, but it also outlines all steps necessary to produce explanations and interpretations. The framework also allows for existing contributions to be recontextualized such that their scope can be measured, thus making them comparable to other methods. We show that this framework is compliant with desiderata on explanations, on interpretability and on evaluation metrics. We present a use-case showing how the framework can be used to compare LIME, SHAP and MDNet, establishing their advantages and shortcomings. Finally, we discuss relevant trends in XAI as well as recommendations for future work, all from the standpoint of our framework.

Figures (4)
XAI Handbook: Towards a Unified Framework for Explainable AI Figure 1. Overview of our proposed framework. A task defined in the high-level domain gets an under-specified characterization, leaving out non-functional requirements. “Explanations” are meth- ods that probe for said requirements. Interpretations are mappings from low- to the high-level domain.
XAI Handbook: Towards a Unified Framework for Explainable AI Figure 1. Overview of our proposed framework. A task defined in the high-level domain gets an under-specified characterization, leaving out non-functional requirements. “Explanations” are meth- ods that probe for said requirements. Interpretations are mappings from low- to the high-level domain.
to ask what kind of semantic entity an explanation is: is it an action, an outcome, a process or an object? For most textbook definitions (see Table 2), explanations are seen as statements. In turn, such statements are nothing but descrip- tions about an already existing entity (the explanandum or the one which is subject to description). From a functional perspective, an explanation is therefore the process by which an explanandum is described. Finally, to avoid confusions between the process of explaining and its output, we refer to the former as explanation and the latter as the explanans. As we seek explanations for AI models, we find such suit- able facts in the form of low-level mathematical primitives used to build the models themselves: support vector ma- chines have a decision boundary equation, coordinates of the support vectors, the tolerance threshold, etc. A Neural Network has values for each individual parameter, the equa- tions governing how they connect with each other, the value of the cost function when computed on a particular input, the gradient that can be computed on that loss, etc. All of these are concrete, undisputed facts (assuming there are no bugs) that are suitable explanandums.
to ask what kind of semantic entity an explanation is: is it an action, an outcome, a process or an object? For most textbook definitions (see Table 2), explanations are seen as statements. In turn, such statements are nothing but descrip- tions about an already existing entity (the explanandum or the one which is subject to description). From a functional perspective, an explanation is therefore the process by which an explanandum is described. Finally, to avoid confusions between the process of explaining and its output, we refer to the former as explanation and the latter as the explanans. As we seek explanations for AI models, we find such suit- able facts in the form of low-level mathematical primitives used to build the models themselves: support vector ma- chines have a decision boundary equation, coordinates of the support vectors, the tolerance threshold, etc. A Neural Network has values for each individual parameter, the equa- tions governing how they connect with each other, the value of the cost function when computed on a particular input, the gradient that can be computed on that loss, etc. All of these are concrete, undisputed facts (assuming there are no bugs) that are suitable explanandums.
Table 2. Definitions of “explanation” and “interpretation” according to various dictionaries. Explanation is referred to as an object while interpretation is more commonly associated to an action. Accessed on 05.01.2021. requirements. Asking why something happens, inescapably relates back to causal effects. While these kind of relationships are among the most useful to discover (as it allows for more control over the effect by adjusting the cause), other non-causal relationships remain valuable in the toolset of explainability. Finding out that a model is unfair, without knowing what the cause of it is, can already be helpful in high-stake sce- narios (e.g., by preventing its use). We say that the why relates to the nature of an interpretation. In short, if the interpretation bares a causal meaning, then the why is being defined by the causal link. If the meaning is limited to a correlation, the why is left out of the scope of that particular interpretation. Note that, if the explanation method is al- ready based on causal theory (Lopez-Paz et al., 2017; Chang et al., 2019), the assigned meaning (i.e., the link between the explanans and the high-level (non-)functional requirement) will be more direct and therefore, more likely to withstand scientific scrutiny. The why is therefore not mandatory in explanations generated by XAI methods. In any case, the explanation’s context can and should be defined, be it causal or based solely on correlations. Proponents of XAI methods are responsible for clearly stating the context in which their explanations can be interpreted.
Table 2. Definitions of “explanation” and “interpretation” according to various dictionaries. Explanation is referred to as an object while interpretation is more commonly associated to an action. Accessed on 05.01.2021. requirements. Asking why something happens, inescapably relates back to causal effects. While these kind of relationships are among the most useful to discover (as it allows for more control over the effect by adjusting the cause), other non-causal relationships remain valuable in the toolset of explainability. Finding out that a model is unfair, without knowing what the cause of it is, can already be helpful in high-stake sce- narios (e.g., by preventing its use). We say that the why relates to the nature of an interpretation. In short, if the interpretation bares a causal meaning, then the why is being defined by the causal link. If the meaning is limited to a correlation, the why is left out of the scope of that particular interpretation. Note that, if the explanation method is al- ready based on causal theory (Lopez-Paz et al., 2017; Chang et al., 2019), the assigned meaning (i.e., the link between the explanans and the high-level (non-)functional requirement) will be more direct and therefore, more likely to withstand scientific scrutiny. The why is therefore not mandatory in explanations generated by XAI methods. In any case, the explanation’s context can and should be defined, be it causal or based solely on correlations. Proponents of XAI methods are responsible for clearly stating the context in which their explanations can be interpreted.
Table 3. Recontextualization of three popular XAI methods with the help of our proposed framework.
Table 3. Recontextualization of three popular XAI methods with the help of our proposed framework.

Related papers

Explainable AI (XAI)
Avwerosuoghene P . Akiti

@KidOnWallSt, 2025

Theoretical foundations and definitions  Methodological approaches (inherently interpretable models, post-hoc methods, attention-based techniques)  Technical challenges and limitations  Domain-specific applications (healthcare, finance, criminal justice, education)  Regulatory and ethical frameworks  Current research trends and future directions  Implementation challenges and barriers  Case studies and practical applications  Policy and societal implications  Comprehensive conclusion  Academic references

View PDFchevron_right
Explainable Artificial Intelligence (XAI): Methods, Tools, and Applications
Soumitra Das, Simanta Das

Zenodo, 2025

Explainable AI helps users understand how AI systems make decisions. This survey paper considered popular XAI methodologies such as LIME, SHAP, and saliency maps, along with tools to facilitate their implementation. It highlighted applications in the realms of healthcare and finance. We also include discussion in the contention of accuracy over interpretability, and some dominant ethical issues like fairness and privacy. The main insight is that in combination, most methods provide better explanations, thereby making AI more trustworthy and somewhat responsible.

View PDFchevron_right
xxAI - Beyond Explainable Artificial Intelligence
Randy Goebel

xxAI - Beyond Explainable AI

The success of statistical machine learning from big data, especially of deep learning, has made artificial intelligence (AI) very popular. Unfortunately, especially with the most successful methods, the results are very difficult to comprehend by human experts. The application of AI in areas that impact human life (e.g., agriculture, climate, forestry, health, etc.) has therefore led to an demand for trust, which can be fostered if the methods can be interpreted and thus explained to humans. The research field of explainable artificial intelligence (XAI) provides the necessary foundations and methods. Historically, XAI has focused on the development of methods to explain the decisions and internal mechanisms of complex AI systems, with much initial research concentrating on explaining how convolutional neural networks produce image classification predictions by producing visualizations which highlight what input patterns are most influential in activating hidden units, or are most ...

View PDFchevron_right
Explainable AI: The New 42?
Randy Goebel

Machine Learning and Knowledge Extraction, 2018

their un-debuggability, and their inability to "explain" their decisions in a human understandable and reconstructable way. So while AlphaGo or DeepStack can crush the best humans at Go or Poker, neither program has any internal model of its task; its representations defy interpretation by humans, there is no mechanism to explain their actions and behaviour, and furthermore, there is no obvious instructional value. .. the high performance systems can not help humans improve. Even when we understand the underlying mathematical scaffolding of current machine learning architectures, it is often impossible to get insight into the internal working of the models; we need explicit modeling and reasoning tools to explain how and why a result was achieved. We also know that a significant challenge for future AI is contextual adaptation, i.e., systems that incrementally help to construct explanatory models for solving real-world problems. Here it would be beneficial not to exclude human expertise, but to augment human intelligence with artificial intelligence.

View PDFchevron_right
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Mohammed Berrada

IEEE Access

At the dawn of the fourth industrial revolution, we are witnessing a fast and widespread adoption of artificial intelligence (AI) in our daily life, which contributes to accelerating the shift towards a more algorithmic society. However, even with such unprecedented advancements, a key impediment to the use of AI-based systems is that they often lack transparency. Indeed, the black-box nature of these systems allows powerful predictions, but it cannot be directly explained. This issue has triggered a new debate on explainable AI (XAI). A research field holds substantial promise for improving trust and transparency of AI-based systems. It is recognized as the sine qua non for AI to continue making steady progress without disruption. This survey provides an entry point for interested researchers and practitioners to learn key aspects of the young and rapidly growing body of research related to XAI. Through the lens of the literature, we review the existing approaches regarding the topic, discuss trends surrounding its sphere, and present major research trajectories. INDEX TERMS Explainable artificial intelligence, interpretable machine learning, black-box models. I. INTRODUCTION A. CONTEXT

View PDFchevron_right
A Comprehensive Review on Explainable AI Techniques, Challenges, and Future Scope
Dr Bhavna Bajpai

Intelligent Computing and Networking

Artificial Intelligence (AI) has been making remarkable advancements in recent years and has the potential to revolutionize many aspects of our lives. From self-driving cars to healthcare systems, AI can make tasks easier, faster, and more accurate. However, the increasing reliance on AI has raised concerns about its transparency, accountability, and interpretability. eXplainable AI (XAI) is a field that focuses on explaining the predictions made by AI systems. This has become increasingly important as AI is being used in sensitive and critical applications such as medical diagnoses, financial risk assessments, and criminal justice decisions. It is essential to ensure that the decisions made by AI systems are transparent, trustworthy, and can be justified to stakeholders. The paper explores the challenges associated with creating explainable AI systems and the different techniques that are being developed to overcome these challenges. Further, it presents a summary of the strengths and weaknesses of various XAI techniques. The paper will provide an overview of the state-of-the-art in XAI and highlight the need for further research in this field.

View PDFchevron_right
Axe the X in XAI: A Plea for Understandable AI
Andrés Páez

Philosophy of science for machine learning: Core issues and new perspectives, 2025

In a recent paper, Erasmus et al. (2021) defend the idea that the ambiguity of the term "explanation" in explainable AI (XAI) can be solved by adopting any of four different extant accounts of explanation in the philosophy of science: the Deductive Nomological, Inductive Statistical, Causal Mechanical, and New Mechanist models. In this chapter, I show that the authors' claim that these accounts can be applied to deep neural networks as they would to any natural phenomenon is mistaken. I also provide a more general argument as to why the notion of explainability as it is currently used in the XAI literature bears little resemblance to the traditional concept of scientific explanation. It would be more fruitful to use the label "understandable AI" to avoid the confusion that surrounds the goal and purposes of XAI. In the second half of the chapter, I argue for a pragmatic conception of understanding that is better suited to play the central role attributed to explanation in XAI. Following De Regt (2017) and Kuorikoski and Ylikoski (2015), the conditions of satisfaction for understanding an ML system are fleshed out in terms of an agent's success in using the system.

View PDFchevron_right
Innovations in Explainable AI : Bridging the Gap Between Complexity and Understanding
Keshav Jena, International Journal of Scientific Research in Computer Science, Engineering and Information Technology IJSRCSEIT

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2023

The integration of Artificial Intelligence (AI) into various domains has witnessed remarkable advancements, yet the opacity of complex AI models poses challenges for widespread acceptance and application. This research paper delves into the field of Explainable AI (XAI) and explores innovative strategies aimed at bridging the gap between the intricacies of advanced AI algorithms and the imperative for human comprehension. We investigate key developments, including interpretable model architectures, local and visual explanation techniques, natural language explanations, and model-agnostic approaches. Emphasis is placed on ethical considerations to ensure transparency and fairness in algorithmic decision-making. By surveying and analyzing these innovations, this research contributes to the ongoing discourse on making AI systems more accessible, accountable, and trustworthy, ultimately fostering a harmonious collaboration between humans and intelligent machines in an increasingly AI-driven world.

View PDFchevron_right
The role of ontologies and knowledge in Explainable AI
Roberto Confalonieri

Semantic web, 2024

View PDFchevron_right
Explainable Artificial Intelligence: Concepts, Applications, Research Challenges and Visions
Randy Goebel

Lecture Notes in Computer Science, 2020

The development of theory, frameworks and tools for Explainable AI (XAI) is a very active area of research these days, and articulating any kind of coherence on a vision and challenges is itself a challenge. At least two sometimes complementary and colliding threads have emerged. The first focuses on the development of pragmatic tools for increasing the transparency of automatically learned prediction models, as for instance by deep or reinforcement learning. The second is aimed at anticipating the negative impact of opaque models with the desire to regulate or control impactful consequences of incorrect predictions, especially in sensitive areas like medicine and law. The formulation of methods to augment the construction of predictive models with domain knowledge can provide support for producing human understandable explanations for predictions. This runs in parallel with AI regulatory concerns, like the European Union General Data Protection Regulation, which sets standards for the production of explanations from automated or semi-automated decision making. Despite the fact that all this research activity is the growing acknowledgement that the topic of explainability is essential, it is important to recall that it is also among the oldest fields of computer science. In fact, early AI was re-traceable, interpretable, thus understandable by and explainable to humans. The goal of this research is to articulate the big picture ideas and their role in advancing the development of XAI systems, to acknowledge their historical roots, and to emphasise the biggest challenges to moving forward.

View PDFchevron_right

References (57)

  1. Al-Shedivat, M., Dubey, A., and Xing, E. Contextual expla- nation networks. Journal of Machine Learning Research, 21(194):1-44, 2020. 1, 4, 6
  2. Alvarez-Melis, D. and Jaakkola, T. S. Towards robust in- terpretability with self-explaining neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 7786-7795, 2018. 7, 8
  3. Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58:82-115, 2020. 1
  4. Athalye, A., Carlini, N., and Wagner, D. Obfuscated gradients give a false sense of security: Circumvent- ing defenses to adversarial examples. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Pro- ceedings of Machine Learning Research, pp. 274-283, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR. 2
  5. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., and Samek, W. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015. 6
  6. Baraniuk, C. The 'creepy Facebook AI' story that capti- vated the media, Aug 2017. URL https://www.bbc. com/news/technology-40790258. 5
  7. Bau, D., Zhou, B., Khosla, A., Oliva, A., and Torralba, A. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6541-6549, 2017. 6
  8. Bibal, A. and Frénay, B. Interpretability of machine learning models and representations: an introduction. In ESANN, 2016. 1
  9. Biran, O. and Cotton, C. Explanation and justification in machine learning: A survey. In IJCAI-17 workshop on explainable AI (XAI), volume 8, pp. 8-13, 2017. 4
  10. Bussmann, N., Giudici, P., Marinelli, D., and Pa- penbrock, J. Explainable Machine Learning in Credit Risk Management. Computational Economics, September 2020. ISSN 1572-9974. doi: 10.1007/ s10614-020-10042-0. URL https://doi.org/10. 1007/s10614-020-10042-0. 1
  11. Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Madry, A., and Kurakin, A. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705, 2019. 2
  12. Carrieri, A. P., Haiminen, N., Maudsley-Barton, S., Gar- diner, L.-J., Murphy, B., Mayes, A., Paterson, S., Grimshaw, S., Winn, M., Shand, C., et al. Explainable ai reveals key changes in skin microbiome associated with menopause, smoking, aging and skin hydration. bioRxiv, 2020. 2
  13. Chang, C.-H., Creager, E., Goldenberg, A., and Duvenaud, D. Explaining image classifiers by counterfactual genera- tion. In International Conference on Learning Represen- tations, 2019. 7
  14. Ciatto, G., Calvaresi, D., Schumacher, M. I., and Omicini, A. An abstract framework for agent-based explanations in ai. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pp. 1816- 1818, 2020. 1
  15. Dam, H. K., Tran, T., and Ghose, A. Explainable soft- ware analytics. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results, pp. 53-56, 2018. 2, 4
  16. D'Amour, A., Heller, K., Moldovan, D., Adlam, B., Ali- panahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395, 2020. 3
  17. de Sousa, I. P., Vellasco, M. M. B. R., and da Silva, E. C. Evolved explainable classifications for lymph node metas- tases. arXiv preprint arXiv:2005.07229, 2020. 2
  18. Deng, J., Ding, N., Jia, Y., Frome, A., Murphy, K., Bengio, S., Li, Y., Neven, H., and Adam, H. Large-scale object classification using label relation graphs. In European conference on computer vision, pp. 48-64. Springer, 2014.
  19. Dombrowski, A.-K., Alber, M., Anders, C., Ackermann, M., Müller, K.-R., and Kessel, P. Explanations can be manipulated and geometry is to blame. In Advances in Neural Information Processing Systems, pp. 13589- 13600, 2019. 2
  20. Doshi-Velez, F. and Kim, B. Considerations for evaluation and generalization in interpretable machine learning. In Explainable and Interpretable Models in Computer Vision and Machine Learning, pp. 3-17. Springer, 2018. 1, 2, 4, 7, 8
  21. Esser, P., Rombach, R., and Ommer, B. A disentangling invertible interpretation network for explaining latent rep- resentations. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 9223-9232, 2020. 6
  22. Frye, C., de Mijolla, D., Cowton, L., Stanley, M., and Feige, I. Shapley-based explainability on the data manifold. arXiv preprint arXiv:2006.01272, 2020. 9
  23. Ghorbani, A., Abid, A., and Zou, J. Interpretation of neural networks is fragile. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 33, pp. 3681-3688, 2019. 2
  24. Jain, S. and Wallace, B. C. Attention is not explanation. In Proceedings of the 2019 Conference of the North Amer- ican Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 3543-3556, 2019. 10
  25. Josephson, J. R. and Josephson, S. G. Abductive inference: Computation, philosophy, technology. Cambridge Uni- versity Press, 1996. 2, 4
  26. Kang, M.-J. and Kang, J.-W. Intrusion detection system using deep neural network for in-vehicle network security. PloS one, 11(6):e0155781, 2016. 1
  27. Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., et al. Interpretability beyond feature attribu- tion: Quantitative testing with concept activation vectors (TCAV). In International conference on machine learn- ing, pp. 2668-2677. PMLR, 2018. 6, 8
  28. King, T. D. Human color perception, cognition, and culture: why red is always red. In Color imaging X: processing, hardcopy, and applications, volume 5667, pp. 234-242. International Society for Optics and Photonics, 2005. 5
  29. Lakkaraju, H., Kamar, E., Caruana, R., and Leskovec, J. Faithful and customizable explanations of black box mod- els. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 131-138, 2019. 2, 4
  30. Leavitt, M. L. and Morcos, A. Towards falsifiable inter- pretability research, 2020. 1, 10
  31. Lewis, D. Causal explanation, philosophical papers, vol. 2, 1986. 2, 4
  32. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ra- manan, D., Dollár, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014. 3
  33. Lipton, Z. C. The mythos of model interpretability. Queue, 2018. 1, 2
  34. Lombrozo, T. The structure and function of explanations. Trends in cognitive sciences, 10(10):464-470, 2006. 2, 3, 4, 5, 6
  35. Lopez-Paz, D., Nishihara, R., Chintala, S., Scholkopf, B., and Bottou, L. Discovering causal signals in images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6979-6987, 2017. 7
  36. Lundberg, S. M. and Lee, S.-I. A unified approach to inter- preting model predictions. In Advances in neural infor- mation processing systems, pp. 4765-4774, 2017. 1, 2, 4, 6, 8
  37. Miller, T. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 2019. 1, 2, 6, 7
  38. Miller, T., Howe, P., and Sonenberg, L. Explainable AI: Be- ware of inmates running the asylum. In IJCAI 2017 Work- shop on Explainable Artificial Intelligence (XAI), 2017. URL http://people.eng.unimelb.edu.au/ tmiller/pubs/explanation-inmates.pdf.
  39. Montavon, G., Samek, W., and Müller, K.-R. Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 2018. 1, 2, 4, 10 of the European Union, C. and Parliament, E. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation), 2016. 1
  40. Palacio, S., Folz, J., Hees, J., Raue, F., Borth, D., and Den- gel, A. What do deep networks like to see. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018. 8
  41. Ras, G., van Gerven, M., and Haselager, P. Explanation methods in deep learning: Users, values, concerns and challenges. In Explainable and Interpretable Models in Computer Vision and Machine Learning, pp. 19-36. Springer, 2018. 5
  42. Ribeiro, M. T., Singh, S., and Guestrin, C. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016. 2, 8
  43. Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206- 215, 2019. 1, 2, 6, 9, 10
  44. Rudin, C. and Ustun, B. Optimized scoring systems: To- ward trust in machine learning for healthcare and criminal justice. Interfaces, 48(5):449-466, 2018. 1
  45. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Imagenet large scale visual recognition chal- lenge. International journal of computer vision, 115(3): 211-252, 2015. 3
  46. Schmid, U. and Finzel, B. Mutual explanations for co- operative decision making in medicine. KI-Künstliche Intelligenz, pp. 1-7, 2020. 4
  47. Schneeberger, D., Stöger, K., and Holzinger, A. The euro- pean legal framework for medical ai. In Holzinger, A., Kieseberg, P., Tjoa, A. M., and Weippl, E. (eds.), Ma- chine Learning and Knowledge Extraction, pp. 209-226, Cham, 2020. Springer International Publishing. ISBN 978-3-030-57321-8. 2
  48. Slack, D., Hilgard, S., Jia, E., Singh, S., and Lakkaraju, H. Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp. 180-186, 2020. 2, 6, 8
  49. Smeulders, A. W., Worring, M., Santini, S., Gupta, A., and Jain, R. Content-based image retrieval at the end of the early years. IEEE Transactions on pattern analysis and machine intelligence, 22(12):1349-1380, 2000. 3
  50. Sokol, K. and Flach, P. Explainability fact sheets: A framework for systematic assessment of explainable ap- proaches. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* '20, pp. 56-67, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450369367. doi: 10.1145/3351095.3372870. 5
  51. Sowa, J. F. Conceptual structures: information processing in mind and machine. Addison-Wesley Pub., Reading, MA, 1983. 3
  52. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
  53. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. Atten- tion is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017. 6
  54. Vilone, G. and Longo, L. Explainable artificial intelligence: a systematic review. arXiv preprint arXiv:2006.00093, 2020. 2, 4
  55. Wiegreffe, S. and Pinter, Y. Attention is not not explana- tion. In Proceedings of the 2019 Conference on Empir- ical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 11-20, 2019. 10
  56. Xie, N., Ras, G., van Gerven, M., and Doran, D. Explainable deep learning: A field guide for the uninitiated. arXiv preprint arXiv:2004.14545, 2020. 2
  57. Zhang, Z., Xie, Y., Xing, F., McGough, M., and Yang, L. Mdnet: A semantically and visually interpretable medical image diagnosis network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6428-6436, 2017. 8

Related papers

Explainable Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
Adrien Bennetot

Information Fusion

In the last few years, Artificial Intelligence (AI) has achieved a notable momentum that, if harnessed appropriately, may deliver the best of expectations over many application sectors across the field. For this to occur shortly in Machine Learning, the entire community stands in front of the barrier of explainability, an inherent problem of the latest techniques brought by sub-symbolism (e.g. ensembles or Deep Neural Networks) that were not present in the last hype of AI (namely, expert systems and rule based models). Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is widely acknowledged as a crucial feature for the practical deployment of AI models. The overview presented in this article examines the existing literature and contributions already done in the field of XAI, including a prospect toward what is yet to be reached. For this purpose we summarize previous efforts made to define explainability in Machine Learning, establishing a novel definition of explainable Machine Learning that covers such prior conceptual propositions with a major focus on the audience for which the explainability is sought. Departing from this definition, we propose and discuss about a taxonomy of recent contributions related to the explainability of different Machine Learning models, including those aimed at explaining Deep Learning methods for which a second dedicated taxonomy is built and examined in detail. This critical literature analysis serves as the motivating background for a series of challenges faced by XAI, such as the interesting crossroads of data fusion and explainability. Our prospects lead toward the concept of Responsible Artificial Intelligence, namely, a methodology for the large-scale implementation of AI methods in real organizations with fairness, model explainability and accountability at its core. Our ultimate goal is to provide newcomers to the field of XAI with a thorough taxonomy that can serve as reference material in order to stimulate future research advances, but also to encourage experts and professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any prior bias for its lack of interpretability.

View PDFchevron_right
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
Sergio Gil-Lopez

Information Fusion, 2020

In the last few years, Artificial Intelligence (AI) has achieved a notable momentum that, if harnessed appropriately, may deliver the best of expectations over many application sectors across the field. For this to occur shortly in Machine Learning, the entire community stands in front of the barrier of explainability, an inherent problem of the latest techniques brought by sub-symbolism (e.g. ensembles or Deep Neural Networks) that were not present in the last hype of AI (namely, expert systems and rule based models). Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is widely acknowledged as a crucial feature for the practical deployment of AI models. The overview presented in this article examines the existing literature and contributions already done in the field of XAI, including a prospect toward what is yet to be reached. For this purpose we summarize previous efforts made to define explainability in Machine Learning, establishing a novel definition of explainable Machine Learning that covers such prior conceptual propositions with a major focus on the audience for which the explainability is sought. Departing from this definition, we propose and discuss about a taxonomy of recent contributions related to the explainability of different Machine Learning models, including those aimed at explaining Deep Learning methods for which a second dedicated taxonomy is built and examined in detail. This critical literature analysis serves as the motivating background for a series of challenges faced by XAI, such as the interesting crossroads of data fusion and explainability. Our prospects lead toward the concept of Responsible Artificial Intelligence, namely, a methodology for the large-scale implementation of AI methods in real organizations with fairness, model explainability and accountability at its core. Our ultimate goal is to provide newcomers to the field of XAI with a thorough taxonomy that can serve as reference material in order to stimulate future research advances, but also to encourage experts and professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any prior bias for its lack of interpretability.

View PDFchevron_right
Explainable AI
Britta Wrede

KI - Künstliche Intelligenz

View PDFchevron_right
A Practical Tutorial on Explainable AI Techniques
Silvia Tulli

ArXiv, 2021

Last years have been characterized by an upsurge of opaque automatic decision support systems, such as Deep Neural Networks (DNNs). Although they have great generalization and prediction skills, their functioning does not allow obtaining detailed explanations of their behaviour. As opaque machine learning models are increasingly being employed to make important predictions in critical environments, the danger is to create and use decisions that are not justifiable or legitimate. Therefore, there is a general agreement on the importance of endowing machine learning models with explainability. The reason is that EXplainable Artificial Intelligence (XAI) techniques can serve to verify and certify model outputs and enhance them with desirable notions such as trustworthiness, accountability, transparency and fairness. This tutorial is meant to be the go-to handbook for any audience with a computer science background aiming at getting intuitive insights of machine learning models, accompa...

View PDFchevron_right
Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
Andrés Páez

Information Fusion, 106, 2024

As systems based on opaque Artificial Intelligence (AI) continue to flourish in diverse real-world applications, understanding these black box models has become paramount. In response, Explainable AI (XAI) has emerged as a field of research with practical and ethical benefits across various domains. This paper not only highlights the advancements in XAI and its application in real-world scenarios but also addresses the ongoing challenges within XAI, emphasizing the need for broader perspectives and collaborative efforts. We bring together experts from diverse fields to identify open problems, striving to synchronize research agendas and accelerate XAI in practical applications. By fostering collaborative discussion and interdisciplinary cooperation, we aim to propel XAI forward, contributing to its continued success. Our goal is to put forward a comprehensive proposal for advancing XAI. To achieve this goal, we present a manifesto of 27 open problems categorized into nine categories. These challenges encapsulate the complexities and nuances of XAI and offer a road map for future research. For each problem, we provide promising research directions in the hope of harnessing the collective intelligence of interested stakeholders.

View PDFchevron_right

Related topics

  • Computer Science
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved