Shallow Encoding Emits Deep Memories
SEED makes the prefill shallow in depth and the decode state shallow in representation, and proves that the open-loop error of the periodic compressions back to low rank is bounded independently of their number.
2 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
SEED makes the prefill shallow in depth and the decode state shallow in representation, and proves that the open-loop error of the periodic compressions back to low rank is bounded independently of their number.
State-space models (SSMs) maintain a fixed-size recurrent state, but reading and writing it at every decoding step can bottleneck hybrid models. This eager state management causes large state I/O with low arithmetic intensity, leaves speculative decoding no cheap rollback, and feeds state quantization error into subseq...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.