TemplateGBM: Learning Gradient Boosting Decision Trees with Predefined Structures

Abstract

Gradient boosting decision trees (GBDTs) are highly effective for tabular data, but their data-dependent and irregular tree construction limits mini-batch training, incremental updates, and efficient GPU execution. We present TemplateGBM, a GBDT learning framework that separates tree-structure selection from leaf-weight optimization. TemplateGBM first initializes a pool of predefined tree structures, maps training instances to leaves once, and then optimizes leaf weights with mini-batch stochastic gradient descent and back-propagation. This stable computation graph supports user-defined, oblivious, and n-ary trees while enabling reusable instance-to-leaf mappings and parallel GPU execution. We provide five initialization strategies—random, median, oblivious, information-gain, and batch initialization—and show that training converges under a diminishing learning-rate schedule. Experiments on seven real-world regression and classification data sets demonstrate competitive predictive quality, up to 5 × faster inference than XGBoost with oblivious templates, and up to 13.5 × CPU-to-GPU training speedup. TemplateGBM also supports efficient incremental model updates without full retraining and can fine-tune tree ensembles produced by existing GBDT systems.