Details of your forest

## 
## Call:
##  randomForest(formula = medv ~ ., data = Boston, ntree = 1000,      localImp = TRUE) 
##                Type of random forest: regression
##                      Number of trees: 1000
## No. of variables tried at each split: 4
## 
##           Mean of squared residuals: 9.718371
##                     % Var explained: 88.49

Distribution of minimal depth

The plot below shows the distribution of minimal depth among the trees of your forest. Note that:

  • the mean of the distribution is marked by a vertical bar with a value label on it (the scale for it is different than for the rest of the plot),

  • the scale of the X axis goes from zero to the maximum number of trees in which any variable was used for splitting.

Minimal depth for a variable in a tree equals to the depth of the node which splits on that variable and is the closest to the root of the tree. If it is low than a lot of observations are divided into groups on the basis of this variable

Importance measures

Below you can explore the measures of importance for all variables in the forest:

Multi-way importance plot

The multi-way importance plot shows the relation between three measures of importance and labels 10 variables which scored best when it comes to these three measures (i.e. for which the sum of the ranks for those measures is the lowest).

The first multi-way importance plot focuses on three importance measures that derive from the structure of trees in the forest:

  • mean depth of first split on the variable,

  • number of trees in which the root is split on the variable,

  • the total number of nodes in the forest that split on that variable.

The second multi-way importance plot shows two importance measures that derive from the role a variable plays in prediction: with the additional information on the \(p\)-value based on a binomial distribution of the number of nodes split on the variable assuming that variables are randomly drawn to form splits (i.e. if a variable is significant it means that the variable is used for splitting more often than would be the case if the selection was random).

Compare importance measures

The plot below shows bilateral relations between the following importance measures: , if some variables are strongly related to each other it may be worth to consider focusing only on one of them.

Compare rankings of variables

The plot below shows bilateral relations between the rankings of variables according to chosen importance measures. This approach might be useful as rankings are more evenly spread than corresponding importance measures. This may also more clearly show where the different measures of importance disagree or agree.

Variable interactions

Conditional minimal depth

The plot below reports 30 top interactions according to mean of conditional minimal depth – a generalization of minimal depth that measures the depth of the second variable in a tree of which the first variable is a root (a subtree of a tree from the forest). In order to be comparable to normal minimal depth 1 is subtracted so that 0 is the minimum.

For example value of 0 for interaction x:y in a tree means that if we take the highest subtree with the root splitting on x then y is used for splitting immediately after x (minimal depth of x in this subtree is 1). The values presented are means over all trees in the forest.

Note that:

  • the plot shows only 30 interactions that appeared most frequently,

  • the horizontal line shows the minimal value of the depicted statistic among interactions for which it was calculated,

  • the interactions considered are ones with the following variables as first (root variables): and all possible values of the second variable.

You can explore the data used for plotting by interacting with the following table:

Prediction on a grid

The plots below show predictions of the random forest depending on values of components of an interaction (the values of remaining predictors are sampled from their empirical distribution) for up to 3 most frequent interactions that consist of two numerical variables.