NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction
NCP-ArchPreview is introduced, a latent-space language model that pushes autoregressive pretraining beyond standard next-token prediction (NTP) and learns through Next Concept Prediction (NCP) to predict discrete concepts that span multiple tokens, introducing an explicit and more challenging concept-level objective wh...
T.-A. Cao, Chi-Yu Chen, Shuang Cheng et al.
· 0 citations