Jingyuan Li

UNIFUSION: Adapting Autoregressive Language Models into Discrete Diffusion under a Unified Reverse-Rate Objective

This work proposes a simple continual pre-training approach for directly adapting pretrained GPT2 checkpoints to uniform-noise diffusion, and establishes connections among SEDD, MDLM/GIDD, M2S, and Neural CTMC by expressing their conditional losses as a single generalized Kullback--Leibler objective over model reverse rates.