Jul 2026
MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
4-bit quantization enables efficient LLM inference, but suffers from significant accuracy degradation due to outliers. Prior work addresses this problem via data rotation or mixed-precision integer quantization, but often relies on software-managed scaling and frequent dequantization, incurring substantial overhead. Mi...
Simla Burcu Harma, D. Mishin, Zhengyuan Su et al.
· arXiv.org · 0 citations