Matching the Lower Bounds: Stochastic Contracting Cubic Newton and Its Optimal Acceleration

Abstract

We study second-order methods for convex stochastic optimization, where gradients and Hessians are available only through stochastic estimates with variances $\sigma_1^2$ and $\sigma_2^2$, respectively. First, we propose the Stochastic Contracting Cubic Newton method. At each iteration, it minimizes a cubic model with additional quadratic regularization and then contracts the step toward the current point. After $T$ iterations, the method achieves the expected convergence rate $\mathcal{O}(\sigma_1/\sqrt{T}+\sigma_2/T+1/T^2)$. Building on this construction, we develop an accelerated variant achieving $\mathcal{O}(\sigma_1/\sqrt{T}+\sigma_2/T^2+1/T^{7/2})$, matching the known lower bounds of Agafonov et al. (2024) in all three terms.