Jul 2026
How are linear representations learned? Exact solutions to the dynamics of abstraction
A striking attenuation law is proved: both nonlinearities weaken abstraction in activations relative to preactivations, and the theory is applied to improve linear probe generalization in LLMs.
William Yang, Andrew M. Saxe, Peter E. Latham
· arXiv.org · 0 citations