Jul 2026
Training Continuous Chain of Thought Models: A Tale of Two Regimes
C-MTP is introduced, a simpler, faster direct supervision approach that models each latent as an average of the embeddings in the CoT traces to be compressed, and outperforms a prior direct supervision method that approximates the distribution of compressed tokens.
V. Yerram, He He, Eunsol Choi
· arXiv.org · 1 citation