Sampled-Max Subgradient Method for Convex Finite-Max Optimization

Abstract

We study the Sampled-Max Subgradient Method (SMax-SGM) for large convex finite-max problems. Each iteration maximizes over a fresh random subset of the $N$ components and takes one subgradient of the sampled maximizer. The method is therefore stochastic subgradient descent on a sampled-max surrogate. We bound the surrogate error by an average-top-$k$ gap plus the probability of missing all top-$k$ components. A localization argument requires these quantities only on a near-optimal sublevel set. Under a bounded subgradient-moment assumption, this yields $O(\varepsilon^{-2})$ subgradient queries and, when $k$ components remain nearly active in that set, $\widetilde O((N/k)\varepsilon^{-2})$ component-value queries, capped by the full-scan cost. Conversely, a one-dimensional affine construction shows that $\Omega(N/k)$ component-value queries can be necessary even when the average-top-$k$ gap vanishes everywhere and subgradient queries are unlimited. A tensor-grid specialization explains when the subset size can become independent of grid cardinality. Experiments on finite maxima with $2\times10^5$ and $5\times10^6$ components show that, under the reported protocols, SMax-SGM reaches a 5% numerical-reference gap with fewer component-value queries and lower optimizer time than each comparison method that reaches the target.