ORL is introduced, a framework for on-policy residual learning in DLMs that learns from model-induced residual trajectories on the fly and introduces an entropy-contraction objective that sharpens predictions as the model revisits its own predictions.
Yi-Fan Yu, Yu-Qing Jian, Ji-Sen Li et al.· 0 citations
Large language model (LLM) inference is increasingly moving toward lower precision to realize the throughput of hardware accelerators, but aggressive post-training quantization (PTQ) can degrade model quality. We present QATFactory, an open-source framework for deployment-aligned quantization-aware distillation (QAD) a...
BRISK-DLM is introduced, a framework that addresses both mismatches between how these tokens are predicted and how they ultimately contribute to generation by optimizing proposal learning and selection for verified progress.
Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distribution for a single target model, and their acceptance rate collapses under workload shifts. This is a striking inversion of modern LLM development, where target models...
Fengxiang Bie, Yu-Qing Jian, Yi-Fan Yu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.