When Tokenization is Secretly Output Supervision
It is argued that framing tokenization as output supervision provides a principled account of why tokenization consistently affects model performance, and that differences in task performance, training dynamics, and model internals are induced by output tokenization and largely invariant to input tokenization.