Code editing requires a model to decide where to make changes, generate the new content, and preserve everything else. We study how masked diffusion language models divide these responsibilities across four editing interfaces: whole-file rewriting, search-and-replace, locate-then-infill, and token-level editing. Experi...
Xi-Jia Tao, Zi-Rui Liu, Shansan Gong et al.· 0 citations
On-policy Verbal Distillation is introduced, a framework that uses verbal scores from black-box teachers to rank student-generated sub-trajectories, retaining high-scoring ones and replacing low-scoring ones with teacher-generated continuations and suggests that retaining student-generated prefixes helps preserve explo...
Jing Xiong, Hui Shen, Shansan Gong et al.· arXiv.org· 8 citations
Prefilling-dLLM is proposed, a training-free prefill-decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top-K most relevant chunks with intra-chunk token sparsity for decoding, showing that sparse prefilling can outperform dense atte...
Jingfei Xiong, Qilong Han, Shansan Gong et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.