Branch Geometry and Finite-Radius Sensitivity of Hard-ReLU Training
This work characterize the intervening regime in which the perturbation radius is proportional to the GD step, and identifies sufficient response regimes: differentiating training is a choice of perturbation resolution as well as a choice of derivative.