This work reviews data strategies across the training lifecycle and formulates them as policies that allocate a training budget over candidate examples, then divides methods by their decision granularity, model dependence, and temporal adaptivity, then compares their computational requirements and empirical support.
Yuan-Jian Xu, Jia-Ning Hao, Tian-Ze Sun et al.· 0 citations
Multi-Head Latent Attention (MLA) enables expressive multi-head attention with compact caches for its content and decoupled RoPE paths, yet cache memory still scales linearly with context length and batch size. In this work, we establish a systematic model of MLA's dual-path quantization errors, characterizing their di...
Zun-Hai Su, Yuxuan Sun, Jian-Chao Tan et al.· 0 citations
G$^2$PTQ is presented, a unified PTQ framework with Generalized Gradient Compensation that integrates both first- and second-order information under a globally supervised, block-wise optimization objective and enables better alignment with the full-precision model, outperforming state-of-the-art baselines.
Rui-Kang Liu, Hao-Li Bai, Yuxuan Sun et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.