Academia.eduAcademia.edu

Figure 8 – uploaded by Francesco Bodria

See full PDFdownloadDownload figure
Fig. 10: (Top) Explanations of DEEP-SHAP on mnist. (Bottom) Explanations of GRAD-SHAP on imagenet.

Figure 10 (Top) Explanations of DEEP-SHAP on mnist. (Bottom) Explanations of GRAD-SHAP on imagenet.

Related Figures (21)

Fig. 1: Existing taxonomy for the classification of explanation methods.
Fig. 1: Existing taxonomy for the classification of explanation methods.
4 Explanations for Tabular Data
4 Explanations for Tabular Data
Fig. 2: TOP: LIME application on the same record for adult (a/b), german (c/d): a/c are the LG model explanation and b/d the CAT model explanation. All the models correctly predicted the output class. BOTTOM: Force plot returned by SHAP explaining XGB on two records of adult: (e), labeled as class 1 (> 504) and, (f), labeled as class 0 (< 504’). Only the features that contributed more (i.e. higher SHAP’s values) to the classification are reported.
Fig. 2: TOP: LIME application on the same record for adult (a/b), german (c/d): a/c are the LG model explanation and b/d the CAT model explanation. All the models correctly predicted the output class. BOTTOM: Force plot returned by SHAP explaining XGB on two records of adult: (e), labeled as class 1 (> 504) and, (f), labeled as class 0 (< 504’). Only the features that contributed more (i.e. higher SHAP’s values) to the classification are reported.
Fig. 3: SHAP application on adult: a record labelled > 50K (top-left) and one as < 50K (top-right). They are obtained applying the TreeExplainer on a XGB model and then the decision plot, in which all the input features are shown. At the bottom, the application of SHAP to explain the outcome of a set of record by XGB on adult. The interaction values among the features are reported.
Fig. 3: SHAP application on adult: a record labelled > 50K (top-left) and one as < 50K (top-right). They are obtained applying the TreeExplainer on a XGB model and then the decision plot, in which all the input features are shown. At the bottom, the application of SHAP to explain the outcome of a set of record by XGB on adult. The interaction values among the features are reported.
Fig. 4: Explanations of DALEX for two records of adult: b(x) = 0 (< 50) (left), b(~) = 1 (> 50K) (right) to explain an XGB in form of Shapely values (top), break down plots (bottom). The y-axis is the features important, the x-axis the positive/negative contribution.
Fig. 4: Explanations of DALEX for two records of adult: b(x) = 0 (< 50) (left), b(~) = 1 (> 50K) (right) to explain an XGB in form of Shapely values (top), break down plots (bottom). The y-axis is the features important, the x-axis the positive/negative contribution.
Fig. 5: TOP: Results of EBM on adult: overall global explanation (left), example of a global expla- nation for education number (right). BOTTOM: Local explanations of EBM on adult: left, a record classified as 1 (> 50K); right a record classified as 0 (< 50).
Fig. 5: TOP: Results of EBM on adult: overall global explanation (left), example of a global expla- nation for education number (right). BOTTOM: Local explanations of EBM on adult: left, a record classified as 1 (> 50K); right a record classified as 0 (< 50).
Table 3: Comparison on the fidelity and the faithfulness metrics of different explanation methods. For every evaluation we report the mean and the standard deviation over a subset of 50 test set records. Table 4: Comparison on the stability metric. We report the mean and the standard deviation over a subset of 30 test records.
Table 3: Comparison on the fidelity and the faithfulness metrics of different explanation methods. For every evaluation we report the mean and the standard deviation over a subset of 50 test set records. Table 4: Comparison on the stability metric. We report the mean and the standard deviation over a subset of 30 test records.
4.6 Quantitative Comparison
4.6 Quantitative Comparison
Table 5: Explanation runtime expressed in seconds for explainers of tabular classifiers approximated as order of magnitude. Table 6: Explainers for black-boxes classifying image data sorted by explanation type: Saliency Maps (SM), Concept Attributions (CA), Counterfactuals (CF), and Prototypes (PR). For every method is indicated if is possible it for images (IMG) only, or for ANY type of data, if it is an Intrinsic (IN) or a Post-Hoc (PH) model, Local (L) or Global (G), and if it is model Agnostic (A) or model-Specific (S).
Table 5: Explanation runtime expressed in seconds for explainers of tabular classifiers approximated as order of magnitude. Table 6: Explainers for black-boxes classifying image data sorted by explanation type: Saliency Maps (SM), Concept Attributions (CA), Counterfactuals (CF), and Prototypes (PR). For every method is indicated if is possible it for images (IMG) only, or for ANY type of data, if it is an Intrinsic (IN) or a Post-Hoc (PH) model, Local (L) or Global (G), and if it is model Agnostic (A) or model-Specific (S).
common end-user due to their logical structure and the similarity by example they exploit. This is particularly true in decision rules correlated by counterfactual ones, like in LORE. The end-user can understand why she received that outcome, but she also has a suggestion about what to change to achieve another classification. However, fewer methods are proposed in this context w.r.t. feature importance explanations. In particular, the majority of rule and prototype-based explanators are intrinsic. For the few post-hoc ones, on average, they require more time to provide an explanation w.r.t. feature importance ones. Regarding the post-hoc prototype-based models, there are some interesting approaches, but there is no code for them, highlighting that they are still in an early stage of development. During the past few years, counterfactuals have witnessed a particularly great interest. Overall, even if the rules, prototypes, and counterfactuals seem to be the best solution, there are still several open questions and challenges in this research area such as improving the efficiency and the accuracy of these explanation algorithms as well as considering the constraints of the domain in which the model is being employed.
common end-user due to their logical structure and the similarity by example they exploit. This is particularly true in decision rules correlated by counterfactual ones, like in LORE. The end-user can understand why she received that outcome, but she also has a suggestion about what to change to achieve another classification. However, fewer methods are proposed in this context w.r.t. feature importance explanations. In particular, the majority of rule and prototype-based explanators are intrinsic. For the few post-hoc ones, on average, they require more time to provide an explanation w.r.t. feature importance ones. Regarding the post-hoc prototype-based models, there are some interesting approaches, but there is no code for them, highlighting that they are still in an early stage of development. During the past few years, counterfactuals have witnessed a particularly great interest. Overall, even if the rules, prototypes, and counterfactuals seem to be the best solution, there are still several open questions and challenges in this research area such as improving the efficiency and the accuracy of these explanation algorithms as well as considering the constraints of the domain in which the model is being employed.
Fig. 8: Examples of saliency maps obtained with the algorithm exposed in Section 5.1 on various datasets. The first row are the original images of the dataset and on top of them we have the predicted class from the original model.
Fig. 8: Examples of saliency maps obtained with the algorithm exposed in Section 5.1 on various datasets. The first row are the original images of the dataset and on top of them we have the predicted class from the original model.
Fig. 9: Visual Comparison of saliency maps obtained by taking the gradient of the output y w.r.t. the input image x (center) and SMOOTHGRAD (bottom). On the three image in the center the saliency map changes drastically. On all three cases is focusing to the subject of the image completely changing original values. This is true also for the seashore image on the far right.
Fig. 9: Visual Comparison of saliency maps obtained by taking the gradient of the output y w.r.t. the input image x (center) and SMOOTHGRAD (bottom). On the three image in the center the saliency map changes drastically. On all three cases is focusing to the subject of the image completely changing original values. This is true also for the seashore image on the far right.
Table 7: Insertion (left) and deletion (right) metrics expressed as AUC of accuracy vs. percentage of removed/inserted pixels. slowly inserted pixels while substituting with black pixels for deletion. For every substitution we made, we query the image to the black-box, obtaining an accuracy. The final score is obtained by taking the area under the curve (AUC) [62] of accuracy as a function of the percentage of removed pixels. In Figure 11 we have an example of this metric computed on the hockey figure of imagenet. For every dataset, we performed this metric calculation for a set of 100 samples, and then we averaged. The results are shown in Table 7. Insertion scores decrease while augmenting the dataset image dimension because we have higher information and more pixels have to be inserted to higher the accuracy. On the other hand, deletion scores decrease. This fact could be because since we have greater information, it is easier to decrease accuracy. The best methods are highlighted in bold, and we can see that RISE is the best in three out of five experiments. RISE is followed by DEEPLIFT, and €-LRP. Segmentation based methods (LIME, XRAI, GRADCAM, GRADCAM+-+) struggles when using low-resolution images.
Table 7: Insertion (left) and deletion (right) metrics expressed as AUC of accuracy vs. percentage of removed/inserted pixels. slowly inserted pixels while substituting with black pixels for deletion. For every substitution we made, we query the image to the black-box, obtaining an accuracy. The final score is obtained by taking the area under the curve (AUC) [62] of accuracy as a function of the percentage of removed pixels. In Figure 11 we have an example of this metric computed on the hockey figure of imagenet. For every dataset, we performed this metric calculation for a set of 100 samples, and then we averaged. The results are shown in Table 7. Insertion scores decrease while augmenting the dataset image dimension because we have higher information and more pixels have to be inserted to higher the accuracy. On the other hand, deletion scores decrease. This fact could be because since we have greater information, it is easier to decrease accuracy. The best methods are highlighted in bold, and we can see that RISE is the best in three out of five experiments. RISE is followed by DEEPLIFT, and €-LRP. Segmentation based methods (LIME, XRAI, GRADCAM, GRADCAM+-+) struggles when using low-resolution images.
Fig. 11: Example of Insertion (on the left) and Deletion (on the right) metric computation per- formed on LIME and the hockey image. The area under the curve is 0.2156 for deletion and 0.5941 for Insertion.
Fig. 11: Example of Insertion (on the left) and Deletion (on the right) metric computation per- formed on LIME and the hockey image. The area under the curve is 0.2156 for deletion and 0.5941 for Insertion.
Fig. 12: TCAV scores for three concepts: ice, Hockey player, and cheering people (fans) for the class puck of imagenet. On the left the query image; on the center some sample of the image tested in TCAV as concepts, and on the right the histogram of the scores with errors. The hockey players has been classified as a puck, but the saliency maps are very different alongside methods. Here we can see that the ice and the hockey players are important concepts, while the background fans are not significant.
Fig. 12: TCAV scores for three concepts: ice, Hockey player, and cheering people (fans) for the class puck of imagenet. On the left the query image; on the center some sample of the image tested in TCAV as concepts, and on the right the histogram of the scores with errors. The hockey players has been classified as a puck, but the saliency maps are very different alongside methods. Here we can see that the ice and the hockey players are important concepts, while the background fans are not significant.
Fig. 13: Criticism (on the left) and prototypes (on the right), output of MMD-CRITIC from cifar. On the criticisms we have a lot of planes on white background, so the sky background is important for the plane.
Fig. 13: Criticism (on the left) and prototypes (on the right), output of MMD-CRITIC from cifar. On the criticisms we have a lot of planes on white background, so the sky background is important for the plane.
Table 8: Explanation runtime expressed in seconds for explainers of image classifiers approximated as order of magnitude. a value for a group of pixels called patches. If the value is positive, a group contributed positively to the prediction. Otherwise, it contributed negatively.
Table 8: Explanation runtime expressed in seconds for explainers of image classifiers approximated as order of magnitude. a value for a group of pixels called patches. If the value is positive, a group contributed positively to the prediction. Otherwise, it contributed negatively.
Fig. 15: Example of sentence highlighting, on top we have the score produce by IntGrad and below we have in order, LIME, DeepLift and the baseline which consists of multiplying the input with the gradient w.r.t. input. The sentence is taken from imdb
Fig. 15: Example of sentence highlighting, on top we have the score produce by IntGrad and below we have in order, LIME, DeepLift and the baseline which consists of multiplying the input with the gradient w.r.t. input. The sentence is taken from imdb
Table 10: Deletion (right) and Insertion (left) metrics and computed on Sentence Highlighting for different datasets.
Table 10: Deletion (right) and Insertion (left) metrics and computed on Sentence Highlighting for different datasets.
Fig. 16: Saliency heat-map matrix generated from the method presented in [30]. The row and the columns of the matrix correspond to the words in the sentence ‘Read the book, for- get the movie!”. Each value of the matrix shows the attention weight a,; of the annotation of the i-th word w.r.t. the j-th. Fig.17: Representation of the attention in BERT for a sentence taken from imdb using the visualization of [66]. The greater the attention between two words, the bigger the line. Here is selected only the attention related to the word “sucks”.
Fig. 16: Saliency heat-map matrix generated from the method presented in [30]. The row and the columns of the matrix correspond to the words in the sentence ‘Read the book, for- get the movie!”. Each value of the matrix shows the attention weight a,; of the annotation of the i-th word w.r.t. the j-th. Fig.17: Representation of the attention in BERT for a sentence taken from imdb using the visualization of [66]. The greater the attention between two words, the bigger the line. Here is selected only the attention related to the word “sucks”.
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved