The COUNT: Counting N-grams Unlocks Scaling on Host Memory and Training-Free Adaptation
The scaling relationship of compute and performance for language models (LMs) has been extensively studied; more recently, models are being designed to scale memory use, as well. In particular, adding embeddings for n-grams scales memory separately from compute use, because each pass only processes individual embedding...