Kwai Summary Attention V2 Technical Report
Summary-token architectures such as Kwai Summary Attention (KSA) compress distant context into learnable Summary token per chunk, and hybrid variants interleave these layers with full attention to retain exact access to the original text. In such hybrid KSA-Full architecture, the interleaving inserted summary tokens ar...
OneRec Team, Chenglong Chu, Guo-Hong Mu et al.
· 0 citations