By coupling QAD's stable low-bit initialization with OPD's on-policy reasoning recovery, the framework provides a comprehensive sub-3-bit solution that preserves broad capabilities while restoring long-form reasoning.
Yuan-Teng Chen, Zhi-Lei Liu, Pei-Song Wang et al.· 0 citations
It is found that expert selection is far more redundant than the field's operating points assume, and how much expert computation fine-grained MoEs can dispense with is revealed, and when dynamic allocation earns its complexity is established, informing both practical deployment and future pruning methods.
Yuan-Teng Chen, Qi-Wei Lai, Tian-Qi Chen et al.· 1 citation
Experiments show that PRQuant substantially reduces down-projection reconstruction error, with scaling and residual compensation providing the main numerical gains while permutation enables a hardware-friendly contiguous layout.
Pei-Ran Wang, An-Qi Wang, Jia-Ying Zhao et al.· 0 citations
BiSCo-LLM is presented, a codebook-free binary spherical coding framework for extreme low-bit LLM weight compression and its reported storage budget includes binary codes, neural decoders, protected-channel payloads, LoRA adapters, and metadata.
Yuantian Shao, Peisong Wang, Zhilei Liu et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.