Skip to content

Convolution Smoothed Quantile Regression for XGBoost

Aug 2026 · arXiv.org · Vol abs/2608.15290 · 1 citation · 31 references
Computer Science Mathematics

TL;DR

QXGB is developed, a quantile-based gradient boosting framework, and a convolution smoothed loss is introduced within it that estimates conditional quantiles for constructing dense cumulative distribution functions (CDFs), exceedance probabilities, and tail behaviour relevant to extreme outcomes.

Abstract

The increasing availability of large and complex datasets across many scientific disciplines has led to widespread adoption of machine learning (ML) for prediction. However, most ML algorithms focus on point estimation and provide limited information about predictive uncertainty or the conditional distribution of the response, restricting their ability to characterize rare or extreme outcomes. We develop QXGB, a quantile-based gradient boosting framework, and introduce a convolution smoothed loss within it that estimates conditional quantiles for constructing dense cumulative distribution functions (CDFs), exceedance probabilities, and tail behaviour relevant to extreme outcomes. This approach preserves the computational efficiency of extreme gradient boosting while restoring the Hessian information XGBoost relies on for tree splitting, in turn providing interpretable measures of extreme value and exceedance probability predictions. We derive the gradients and Hessians needed to integrate convolution smoothed quantile loss with different kernel specifications into XGBoost, and with simulated data, benchmark this approach against alternative smoothed quantile regression losses, the native quantile objective in the XGBoost Python package, and independent versus multi-output tree estimation. The practical relevance is illustrated in an application predicting fine particulate matter (PM$_{2.5}$) in northern California, including periods where levels were elevated due to wildfire smoke. Our results show that convolution smoothed QXGB, particularly when paired with multi-output trees, delivers accurate predictions with near-zero quantile crossing, well-calibrated CDF and exceedance probability estimates, and useful tail characterization for extreme values. Interval estimation is also evaluated as a measure of data spread.

View source

Similar papers

Preprint Oct 2026

Existence and consistency of weighted maximum likelihood estimator for extreme quantile regression

Estimating extreme conditional quantiles faces two major challenges: understanding complex nonlinear relationships between variables and accurately extrapolating into the tails of the distribution, where data are sparse. To address both issues, we introduce a novel approach that combines the theoretical framework of bl...

Lucien M. Vidagbandji, Alexandre Berred, Cyrille Bertelle et al. · 0 citations
Preprint Oct 2026

Sparse-group Convoluted Smoothed Quantile Regression in High Dimension

In many applications, covariates are naturally organized into groups, yet conventional group-selection methods often retain all variables within selected groups, failing to achieve within-group sparsity. To address this limitation, we propose an adaptive two-stage sparse-group selection procedure for high-dimensional q...

Ji-Ah-Ua Lv, An-Xin Wang · 0 citations
Preprint Oct 2026

QuantileMix: A Python package for quantile g-computation

Health researchers often study how a mixture of pollutants or nutrients relates to health. Quantile g-computation addresses this question by giving each exposure an ordered score and fitting a model for the outcome. For example, dividing an exposure into four quantile groups gives scores 0, 1, 2, and 3. The model can c...

Nilson Chapagain · 0 citations
Preprint Oct 2026

Scalable and Adaptive Distribution Regression via Random Features

Distribution regression seeks to learn mappings from probability distributions to response variables and has emerged as a unifying framework for kernel methods, multi-instance learning, and set-valued learning. Despite its success, kernel-based distribution regression remains computationally expensive and sensitive to...

Yin-Song Wang, Xiao Liu, Yu Ding · 0 citations
Sep 2026

Contextual Conditional Value-at-Risk: Estimation and Optimization

This work exploits the Rockafellar–Uryasev representation to formulate both contextual CVaR estimation and optimization as optimization problems involving conditional expectations, and develops a unified data-driven predict-then-optimize framework.

Heng Luo, Ni-Fei Lin, L. J. Hong · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.