Latent-Condensed Transformer for Efficient Long Context Modeling
LCA is proposed, which directly condenses context within MLA's latent space, where the representation is disentangled into semantic latent vectors and positional keys, and jointly reduces both computational cost and KV cache without adding parameters.
Zeng You, Yao-Fo Chen, Qiuwu Chen et al.
· Annual Meeting of the Associ... · 0 citations