In the vignette below we will see how methods implemented in DALEX R packages focuses on variables in selected model. For our model we consider Break Down, LIME and Shapley values.

library(DALEX)
library(lime)

1 Data

To illustrate the differences in explanation of prediction between methods we will use an artificial dataset created below.

set.seed(123)
N <- 10000

x1 <- 2*(runif(N) - 0.5)
x2 <- 2*(runif(N) - 0.5)

y1 <- sign(x1) + sign(x2) + rnorm(N)
y2 <- sign(x1) * sign(x2) + rnorm(N)

df <- data.frame(y1, y2, x1, x2)

We also want to find the differences between interactions and additive effect in model.

2 Models

df_train <- df[1:1000,]
df_test <- df[1001:nrow(df),]

library(mlr)

regr_task_y1 <- makeRegrTask(id = "df_y1", data = df_train[,-2], target = "y1")
regr_lrn_rf_y1 <- makeLearner("regr.randomForest")
regr_rf_y1 <- train(regr_lrn_rf_y1, regr_task_y1)

regr_task_y2 <- makeRegrTask(id = "df_y2", data = df_train[,-1], target = "y2")
regr_lrn_rf_y2 <- makeLearner("regr.randomForest")
regr_rf_y2 <- train(regr_lrn_rf_y2, regr_task_y2)

3 Methods

In this vignette we would like to compare three approaches to analysis of a single prediction: Break Down, LIME and Shapley value.

3.1 Break Down

Break Down is the methodology delevoped by Przemysław Biecek and Mateusz Staniak. A description of the method can be found here.

This methodology is implemented by Przemysław Biecek and Mateusz Staniak in the R package breakDown. Currently available in the R package DALEX.

The breakDown package is a model agnostic tool for decomposition of predictions from black boxes. Break Down Table shows contributions of every variable to a final prediction. Break Down Profile presents variable contributions in a concise graphical way.

3.2 LIME (Local Interpretable Model-agnostic Explanations)

LIME is the methodology developed by Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. A description of the method can be found here.

The purpose of LIME is to explain the predictions of black box classifiers. What this means is that for any given prediction and any given classifier it is able to determine a small set of features in the original data that has driven the outcome of the prediction. Using here package lime is an R port of the Python lime package (https://github.com/marcotcr/lime) developed by the authors of the lime approach for black-box model explanations.

3.3 Shapley value

A methodology based on the use of Shapley value known from game theory to explain predictions is developed by Scott Lundberg and Su-In Lee. A description of the method can be found here.

The Shapley value is a method that gives a solution to the following problem: A coalition of players play a game, for which they get a payout, but it is not clear how to distribute the payout fairly among the players. The Shapley value solves this problem by trying out different coalitions of players to decide how much each player changes the amount of the payout. What does this have to do with machine learning? In machine learning, the features (= players) work together to get the payout (=predicted value). The Shapley value tells us, how much each feature contributed to the prediction.

We use Shapley value implementation in the R package DALEX.

4 Explainer object

We create a explainer which is an object/adapter that wraps the model and creates an uniform structure and interface for operation. We use a DALEXtra R package which is an extension of DALEX and supports models built in mlr.

explainer_regr_rf_y1 <- DALEXtra::explain_mlr(regr_rf_y1, data = df_test[,-c(1,2)], y = df_test$y1, label = "randomforest_1")
## Preparation of a new explainer is initiated
##   -> model label       :  randomforest_1 
##   -> data              :  9000  rows  2  cols 
##   -> target variable   :  9000  values 
##   -> predict function  :  yhat.WrappedModel  will be used (  default  )
##   -> predicted values  :  numerical, min =  -3.201019 , mean =  -0.02157578 , max =  3.251853  
##   -> model_info        :  package mlr , ver. 2.17.0 , task regression (  default  ) 
##   -> residual function :  difference between y and yhat (  default  )
##   -> residuals         :  numerical, min =  -3.954667 , mean =  -0.02344241 , max =  4.018775  
##   A new explainer has been created! 
explainer_regr_rf_y2 <- DALEXtra::explain_mlr(regr_rf_y2, data = df_test[,-c(1,2)], y = df_test$y2, label = "randomforest_2")
## Preparation of a new explainer is initiated
##   -> model label       :  randomforest_2 
##   -> data              :  9000  rows  2  cols 
##   -> target variable   :  9000  values 
##   -> predict function  :  yhat.WrappedModel  will be used (  default  )
##   -> predicted values  :  numerical, min =  -2.272377 , mean =  -0.02608881 , max =  2.174938  
##   -> model_info        :  package mlr , ver. 2.17.0 , task regression (  default  ) 
##   -> residual function :  difference between y and yhat (  default  )
##   -> residuals         :  numerical, min =  -3.879196 , mean =  0.01098456 , max =  4.207064  
##   A new explainer has been created! 

5 Model with additive effect

Now for the three new observations we will compare results of Break Down, LIME and Shapley value for model with additive effect between variables x1 and x2.

5.1 x1 = -1, x2 = -1

5.1.1 Explainers

new_observation_1 <- data.frame(x1 = -1, x2 = -1)

df_test <- rbind(df_test, data.frame(y1 = sign(-1) + sign(-1) + rnorm(1) , y2 = sign(-1)*sign(-1) + rnorm(1), x1 = -1, x2 = -1),  data.frame(y1 = sign(-1) + sign(1) + rnorm(1) , y2 = sign(-1)*sign(1) + rnorm(1), x1 = -1, x2 = 1),
                 data.frame(y1 = sign(0) + sign(0) + rnorm(1) , y2 = sign(0)*sign(0) + rnorm(1), x1 = 0, x2 = 0))

bd_new_observation_1 <- predict_parts(explainer_regr_rf_y1, new_observation_1, type = "break_down")

explainer <- lime(df_train[,-c(1,2)], regr_rf_y1)
lime_new_observation_1 <- lime::explain(new_observation_1, explainer, n_features = 2)

sh_new_observation_1 <- predict_parts(explainer_regr_rf_y1, new_observation_1, type = "shap")

5.1.2 Comparison

plot(bd_new_observation_1)

plot_features(lime_new_observation_1)

lime_new_observation_1
## # A tibble: 2 x 11
##   model_type case  model_r2 model_intercept model_prediction feature
##   <chr>      <chr>    <dbl>           <dbl>            <dbl> <chr>  
## 1 regression 1        0.301           0.260            -2.06 x1     
## 2 regression 1        0.301           0.260            -2.06 x2     
## # ... with 5 more variables: feature_value <dbl>, feature_weight <dbl>,
## #   feature_desc <chr>, data <list>, prediction <dbl>
plot(sh_new_observation_1)

We see that in all methods both, x1 and x2 contradicts the predicted value of y1.

5.2 x1 = -1, x2 = 1

5.2.1 Explainers

new_observation_2 <- data.frame(x1 = -1, x2 = 1)

bd_new_observation_2 <- predict_parts(explainer_regr_rf_y1, new_observation_2, type = "break_down")

lime_new_observation_2 <- lime::explain(new_observation_2, explainer, n_features = 2)

sh_new_observation_2 <- predict_parts(explainer_regr_rf_y1, new_observation_2, type = "shap")

5.2.2 Comparison

plot(bd_new_observation_2)

plot_features(lime_new_observation_2)

lime_new_observation_2
## # A tibble: 2 x 11
##   model_type case  model_r2 model_intercept model_prediction feature
##   <chr>      <chr>    <dbl>           <dbl>            <dbl> <chr>  
## 1 regression 1        0.168           0.441            0.288 x1     
## 2 regression 1        0.168           0.441            0.288 x2     
## # ... with 5 more variables: feature_value <dbl>, feature_weight <dbl>,
## #   feature_desc <chr>, data <list>, prediction <dbl>
plot(sh_new_observation_2)

For all methods we see that variable x1 contradicts the prediction and x2 supports it.

5.3 x1 = 0, x2 = 0

5.3.1 Explainers

new_observation_3 <- data.frame(x1 = 0, x2 = 0)

bd_new_observation_3 <- predict_parts(explainer_regr_rf_y1, new_observation_3, type = "break_down")

lime_new_observation_3 <- lime::explain(new_observation_3, explainer, n_features = 2)

sh_new_observation_3 <- predict_parts(explainer_regr_rf_y1, new_observation_3, type = "shap")

5.3.2 Comparison

plot(bd_new_observation_3)

plot_features(lime_new_observation_3)

lime_new_observation_3
## # A tibble: 2 x 11
##   model_type case  model_r2 model_intercept model_prediction feature
##   <chr>      <chr>    <dbl>           <dbl>            <dbl> <chr>  
## 1 regression 1        0.306          -0.733             1.86 x1     
## 2 regression 1        0.306          -0.733             1.86 x2     
## # ... with 5 more variables: feature_value <dbl>, feature_weight <dbl>,
## #   feature_desc <chr>, data <list>, prediction <dbl>
plot(sh_new_observation_3)

For Break Down and Shapley value we see that both variables contradicts the prediction, but in LIME method we see that both variables support the value of y1.

6 Model with interaction

Now for the three new observations we will compare results of Break Down, LIME and Shapley value for the model with interaction between variables x1 and x2.

6.1 x1 = -1, x2 = -1

6.1.1 Explainers

new_observation_1 <- data.frame(x1 = -1, x2 = -1)

bd_new_observation_1 <- predict_parts(explainer_regr_rf_y2, new_observation_1, type = "break_down")

explainer <- lime(df_train[,-c(1,2)], regr_rf_y2)
lime_new_observation_1 <- lime::explain(new_observation_1, explainer, n_features = 2)

sh_new_observation_1 <- predict_parts(explainer_regr_rf_y2, new_observation_1, type = "shap")

6.1.2 Comparison

plot(bd_new_observation_1)

plot_features(lime_new_observation_1)

lime_new_observation_1
## # A tibble: 2 x 11
##   model_type case  model_r2 model_intercept model_prediction feature
##   <chr>      <chr>    <dbl>           <dbl>            <dbl> <chr>  
## 1 regression 1       0.0415          -0.239            0.367 x1     
## 2 regression 1       0.0415          -0.239            0.367 x2     
## # ... with 5 more variables: feature_value <dbl>, feature_weight <dbl>,
## #   feature_desc <chr>, data <list>, prediction <dbl>
plot(sh_new_observation_1)

We see that in LIME and Shapley value x1 and x2 support the predicted value of y2. In Break Down x1 supports and x2 contradicts prediction.

6.2 x1 = -1, x2 = 1

6.2.1 Explainers

new_observation_2 <- data.frame(x1 = -1, x2 = 1)

bd_new_observation_2 <- predict_parts(explainer_regr_rf_y2, new_observation_2, type = "break_down")

lime_new_observation_2 <- lime::explain(new_observation_2, explainer, n_features = 2)

sh_new_observation_2 <- predict_parts(explainer_regr_rf_y2, new_observation_2, type = "shap")

6.2.2 Comparison

plot(bd_new_observation_2)

plot_features(lime_new_observation_2)

lime_new_observation_2
## # A tibble: 2 x 11
##   model_type case  model_r2 model_intercept model_prediction feature
##   <chr>      <chr>    <dbl>           <dbl>            <dbl> <chr>  
## 1 regression 1       0.0166          0.0878           -0.622 x1     
## 2 regression 1       0.0166          0.0878           -0.622 x2     
## # ... with 5 more variables: feature_value <dbl>, feature_weight <dbl>,
## #   feature_desc <chr>, data <list>, prediction <dbl>
plot(sh_new_observation_2)

For all methods we see that both variables contradict the prediction.

6.3 x1 = 0, x2 = 0

6.3.1 Explainers

new_observation_3 <- data.frame(x1 = 0, x2 = 0)

bd_new_observation_3 <- predict_parts(explainer_regr_rf_y2, new_observation_3, type = "break_down")

lime_new_observation_3 <- lime::explain(new_observation_3, explainer, n_features = 2)

sh_new_observation_3 <- predict_parts(explainer_regr_rf_y2, new_observation_3, type = "shap")

6.3.2 Comparison

plot(bd_new_observation_3)

plot_features(lime_new_observation_3)

lime_new_observation_3
## # A tibble: 2 x 11
##   model_type case  model_r2 model_intercept model_prediction feature
##   <chr>      <chr>    <dbl>           <dbl>            <dbl> <chr>  
## 1 regression 1      0.00291          0.0170           -0.113 x1     
## 2 regression 1      0.00291          0.0170           -0.113 x2     
## # ... with 5 more variables: feature_value <dbl>, feature_weight <dbl>,
## #   feature_desc <chr>, data <list>, prediction <dbl>
plot(sh_new_observation_3)

For Break Down and Shapley values we see that both variables support the prediction, but in LIME method we see that both variables contradict the value of y2.

7 Comparison for additive and interaction effect

After plotting the values of explainers for Break Down, LIME and Shapley value we see that for model with additive effect all three methods produce similar results (only in the case of observation with x1 = 0 and x2 = 0). On the other hand, when considering the model with interactions, we see that the results for our methods are usually different (only in the case of observation with x1 = -1 and x2 = 1 we have similar results).

8 Session info

sessionInfo()
## R version 3.6.3 (2020-02-29)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 18363)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=Polish_Poland.1250  LC_CTYPE=Polish_Poland.1250   
## [3] LC_MONETARY=Polish_Poland.1250 LC_NUMERIC=C                  
## [5] LC_TIME=Polish_Poland.1250    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] mlr_2.17.0        ParamHelpers_1.13 lime_0.5.1        DALEX_2.0.1      
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_1.0.4          lattice_0.20-38     utf8_1.1.4         
##  [4] assertthat_0.2.1    glmnet_3.0-2        digest_0.6.25      
##  [7] foreach_1.4.8       mime_0.9            R6_2.4.1           
## [10] backports_1.1.5     evaluate_0.14       ggplot2_3.3.0      
## [13] pillar_1.4.3        rlang_0.4.6         data.table_1.12.8  
## [16] Matrix_1.2-18       checkmate_2.0.0     DALEXtra_2.0       
## [19] reticulate_1.14     rmarkdown_2.1       labeling_0.3       
## [22] shinythemes_1.1.2   splines_3.6.3       gower_0.2.1        
## [25] stringr_1.4.0       htmlwidgets_1.5.1   munsell_0.5.0      
## [28] shiny_1.4.0         compiler_3.6.3      httpuv_1.5.2       
## [31] xfun_0.12           pkgconfig_2.0.3     shape_1.4.4        
## [34] BBmisc_1.11         htmltools_0.4.0     tidyselect_1.1.0   
## [37] tibble_2.1.3        codetools_0.2-16    randomForest_4.6-14
## [40] XML_3.99-0.3        fansi_0.4.1         crayon_1.3.4       
## [43] dplyr_1.0.0         later_1.0.0         grid_3.6.3         
## [46] jsonlite_1.6.1      xtable_1.8-4        gtable_0.3.0       
## [49] lifecycle_0.2.0     magrittr_1.5        scales_1.1.0       
## [52] cli_2.0.2           stringi_1.4.6       farver_2.0.3       
## [55] iBreakDown_1.3.1    promises_1.1.0      parallelMap_1.4    
## [58] generics_0.0.2      vctrs_0.3.1         fastmatch_1.1-0    
## [61] iterators_1.0.12    tools_3.6.3         glue_1.3.2         
## [64] purrr_0.3.3         parallel_3.6.3      fastmap_1.0.1      
## [67] survival_3.1-8      yaml_2.2.1          colorspace_1.4-1   
## [70] knitr_1.28