The revival of bagged trees: hierarchical shrinkage as a regularization tool

Abstract

We present methodological refinements and new insights into hierarchical shrinkage (HS), a post-hoc regularization technique for decision trees that shrinks node predictions toward ancestral means. By contrasting HS with the implicit regularization from feature sub-sampling in random forests (RFs), we clarify when bagging and RFs exhibit similar regularization behavior and explore how the optimal shrinkage parameter depends on the signal-to-noise ratio (SNR). Our analysis also highlights key interactions between tree depth, feature cardinality, and degrees of freedom. To address limitations of the original HS method, such as its insensitivity to split informativeness and bias toward high-cardinality features, we propose a novel method called adaptive HS (adHS), which adjusts shrinkage based on feature entropy and estimated overfitting. We further examine the tunability of RFs across SNR regimes and show that bagging combined with HS can recover tunability gains in certain data regimes when it comes to predictive performance, offering a computationally efficient post-hoc alternative to repeated refitting for structural hyperparameter tuning in RFs. Additionally, adHS can improve the stability and interpretability of feature importance rankings derived from SHAP values and mean decrease in impurity.