Wei-Min Huang

Backpropagation-Free Trunk Training via the Split Forward Gradients

Split Forward Gradient is introduced, which splits a network at an intermediate representation: it computes the output head gradient exactly and estimates only the trunk gradient with a Jacobian--vector product, which reduces estimator variance and requires no backward pass through the trunk, while retaining an Adam-st...