Omri Barak

Learning reveals invisible structure in low-rank RNNs

This work extends the low-rank framework from activity to learning by deriving gradient-descent dynamics directly in a reduced overlap space, forming a closed-form, low-dimensional system of ODEs that governs learning in this space, exact for linear RNNs and asymptotically exact for nonlinear RNNs in the large-$N$ Gaus...