Local polynomial smoothing is commonly used in non-parametric regression, but local linear derivative estimation still has a bias of order $O(h^2)$. This paper proposes an iterative data sharpening method to reduce the bias of derivative estimates while retaining the simplicity of local linear fitting. The method is based on two expectation operators: $L_0$, acting on the regression function, and $L_1$, acting on the first-order derivative. By repeatedly applying the residual operator $R=I-L_0$, a series of sharpened derivative estimates can be constructed. After $l$ sharpening steps, the bias order can be reduced from $O(h^2)$ to $O(h^{2l+2})$. For the Gaussian kernel, all sharpening coefficients equal 1, giving a simple closed-form single-bandwidth expression. Simulation experiments on three smooth test functions show that this method can significantly reduce the estimation bias while revealing a bias-variance trade-off.
Kernel ridge regression is a standard method for functional data analysis, but its exact behavior is less understood. We study tensor-product kernel ridge regression for estimating the $r$-th moment function of a random function based on noisy discrete observations. The formulation includes mean estimation, covariance...
We present a simple Gaussian approximation to the finite-sample distribution of the classical ridge regression estimator. Our approximation captures the fact that, in finite samples, the ridge regression estimator trades off bias and variance to reduce estimation and prediction error. Our approximation is based on nons...
J. M. Olea, Ryan Strong, Amilcar Velez et al.· 0 citations
For finite-dimensional linear inverse problems where the variables are Gaussian, it is well-known that the minimum-mean-square error estimator takes the form of a regularized least-squares data fit. In this chapter, we show that this equivalence extends to a much broader infinite-dimensional setting where generalized s...
Exhaustive moment fitting in this constant-dimensional space produces a proper mixture and, together with the dimension-free moment characterization of Gaussian mixtures, achieves the optimal Hellinger rate in polynomial arithmetic time for every fixed $k$.
This work constructs a counterexample empirically showing that smoothness alone is not sufficient for the sequential convergence of AdaGrad-type algorithms, and suggesting that additional geometric hypotheses are indispensable for sequential convergence results.
A general recursive framework that includes the Chambolle--Pock algorithm and related primal--dual splitting methods is developed, which proves finite-sample guarantees for both estimators and establishes a matched-Gaussian universality result beyond Gaussian designs.
Kai Tan, Pierre C. Bellec· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
