RiLM: Parameter-Efficient Language Modeling via Geodesic Decoding
Language models under one million parameters matter for edge deployment, domain adaptation, and reproducible research, yet a two-layer LSTM or Transformer at embedding width d = 128 still spends roughly one third of its capacity on the output matrix W_out in R^(d x |V|). We propose Riemannian Language Models (RiLM), wh...
Fang Li
· 0 citations