Statistical Inference for Gaussian Kernel Robust Regression with the gkrreg Package
TL;DR
It is formally established that GKRReg belongs to the family of redescending M-estimators, providing the theoretical foundation for the inferential procedures that follow and proposing a pairs bootstrap that re-estimates the kernel width hyper-parameter gamma^2 on every replicate, capturing variability that the sandwich ignores.
Abstract
The Gaussian Kernel Robust Regression method (GKRReg) is a robust regression estimator that iteratively re-weights observations via a Gaussian kernel so that outliers and leverage points receive near-zero weight, with convergence of the estimation algorithm theoretically guaranteed. Despite a thorough study of estimation, the original work leaves open the problem of statistical inference for the regression coefficients. We fill this gap with three contributions. First, we formally establish that GKRReg belongs to the family of redescending M-estimators, providing the theoretical foundation for the inferential procedures that follow. Second, we derive a closed-form analytic sandwich variance estimator based on the theory of generalised M-estimators, corresponding to the HC0 class of heteroskedasticity-robust covariance matrices; we show that a finite-sample correction analogous to HC3 requires the weighted hat matrix of the converged IRWLS step, and identify this as a direction for future work. Third, we propose a pairs bootstrap that re-estimates the kernel width hyper-parameter gamma^2 on every replicate, capturing variability that the sandwich ignores. All procedures are implemented in the R package gkrreg, which also provides four estimators for gamma^2 and an automatic data-driven selection procedure, comprehensive diagnostic plots, and six real datasets from the robust regression literature. Applications to real data sets and comparison with traditional robust regression models highlight the potential of the GKRReg and the usability of the R package.