Content-Addressed Recursive State: Resolving the DenseRetrieval Trade-o with a Cubic-Composition Readout
Recurrent architectures that compress history into a single state are cheap but have classically been unable to exactly retrieve far past key–value bindings, whereas attention content-addresses the whole past at quadratic cost. We show that the conflict between dense language modeling and long-range associative recall...