Post-training compression of LLM attention is often formulated as independent matrix approximation, ignoring both the shared structure among attention projections and the representation shift introduced by earlier compression. We propose FTC, a sequential structured compression framework that adapts the approximation t...
Jiang-Feng Chen, Xin-Yu Wang, Tian-Shuo Yan et al.· 0 citations
Group-wise post-training quantizers for large language models round weights onto a grid that is not refit to the resulting integer codes. We show that this leaves accuracy on the table: the best grid depends on the codes, input correlations couple the errors of different groups, and useful code changes often involve ma...
RRC introduces two complementary strategies: self-competitive ranking, which exploits comparisons among sampled responses, and anchor-guided ranking, which enables scalable ranking-based reward construction with a small set of reference responses.
Chenglong Wang, Ziming Zhu, Yifu Huo et al.· 2 citations
Results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost, and AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph, improves end-to-end speed at both tested target scales.
OJBKQ is proposed, a layer-wise PTQ method that formulates weight quantization as a joint optimization problem over activations and weights, yielding a multiple-right-hand-side box-constrained integer least squares (BILS) problem per layer.
Xin-Yu Wang, Zi-Yu Zhao, Peng Lu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
