Aug 2026· arXiv.org· Vol abs/2608.15290· 1 citation· 31 references
Computer ScienceMathematics
TL;DR
QXGB is developed, a quantile-based gradient boosting framework, and a convolution smoothed loss is introduced within it that estimates conditional quantiles for constructing dense cumulative distribution functions (CDFs), exceedance probabilities, and tail behaviour relevant to extreme outcomes.
Abstract
The increasing availability of large and complex datasets across many scientific disciplines has led to widespread adoption of machine learning (ML) for prediction. However, most ML algorithms focus on point estimation and provide limited information about predictive uncertainty or the conditional distribution of the response, restricting their ability to characterize rare or extreme outcomes. We develop QXGB, a quantile-based gradient boosting framework, and introduce a convolution smoothed loss within it that estimates conditional quantiles for constructing dense cumulative distribution functions (CDFs), exceedance probabilities, and tail behaviour relevant to extreme outcomes. This approach preserves the computational efficiency of extreme gradient boosting while restoring the Hessian information XGBoost relies on for tree splitting, in turn providing interpretable measures of extreme value and exceedance probability predictions. We derive the gradients and Hessians needed to integrate convolution smoothed quantile loss with different kernel specifications into XGBoost, and with simulated data, benchmark this approach against alternative smoothed quantile regression losses, the native quantile objective in the XGBoost Python package, and independent versus multi-output tree estimation. The practical relevance is illustrated in an application predicting fine particulate matter (PM$_{2.5}$) in northern California, including periods where levels were elevated due to wildfire smoke. Our results show that convolution smoothed QXGB, particularly when paired with multi-output trees, delivers accurate predictions with near-zero quantile crossing, well-calibrated CDF and exceedance probability estimates, and useful tail characterization for extreme values. Interval estimation is also evaluated as a measure of data spread.
Estimating extreme conditional quantiles faces two major challenges: understanding complex nonlinear relationships between variables and accurately extrapolating into the tails of the distribution, where data are sparse. To address both issues, we introduce a novel approach that combines the theoretical framework of bl...
Lucien M. Vidagbandji, Alexandre Berred, Cyrille Bertelle et al.· 0 citations
In many applications, covariates are naturally organized into groups, yet conventional group-selection methods often retain all variables within selected groups, failing to achieve within-group sparsity. To address this limitation, we propose an adaptive two-stage sparse-group selection procedure for high-dimensional q...
Health researchers often study how a mixture of pollutants or nutrients relates to health. Quantile g-computation addresses this question by giving each exposure an ordered score and fitting a model for the outcome. For example, dividing an exposure into four quantile groups gives scores 0, 1, 2, and 3. The model can c...
Distribution regression seeks to learn mappings from probability distributions to response variables and has emerged as a unifying framework for kernel methods, multi-instance learning, and set-valued learning. Despite its success, kernel-based distribution regression remains computationally expensive and sensitive to...
This work exploits the Rockafellar–Uryasev representation to formulate both contextual CVaR estimation and optimization as optimization problems involving conditional expectations, and develops a unified data-driven predict-then-optimize framework.
Heng Luo, Ni-Fei Lin, L. J. Hong· Asia-Pacific Journal of Oper...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.