Lngram v2: Latent N-Gram Memory with Interpretable Discrete Representations
Transformers lack a native lookup mechanism, requiring repeated dense computation to recognize and reuse local static patterns. Lngram v1 introduces tokenizer-independent conditional memory through discrete latent n-gram addressing, but its memory capacity is coupled with the backbone width, limiting scalability due to...
Yun Zheng, Bin Wen, Xiao-Jie Wang et al.
· 0 citations