Seung-Yoon Kang

Fast LM Head via Coarse-Fine Decomposition and Tile-Wise Weight Clustering

Large language models (LLMs) reuse each generated token as the next input during autoregressive decoding, making per-token latency a critical factor in practical inference performance. In this process, the LM Head performs a large-scale linear projection from the hidden state to the full vocabulary space at every decod...