PhaseQuant: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics

Abstract

Post-training quantization (PTQ) is essential for efficient large language model (LLM) inference, but reliably quantizing activations remains challenging when weights, activations, and KV caches are all quantized to 4-bit precision. A key difficulty lies in massive activations, whose extreme values dominate the activation range and amplify quantization error. State-of-the-art methods mitigate massive activations through transformation-based smoothing, such as orthogonal rotations and affine scaling, but overlook the cross-layer dynamics of the residual stream. In this paper, we show that massive activations emerge and disappear in a phase-wise pattern across network depth, injecting abrupt updates into the residual stream. These updates cause the newly injected layer-wise signal to dominate the 4-bit quantization scale and weaken the accumulated residual information. To characterize this behavior, we introduce two metrics, the Update Dominance Ratio (UDR) and the Historical Retention Score (HRS), and show that static transformation-based smoothing cannot fully resolve the resulting dynamic quantization instability. Based on this analysis, we propose PhaseQuant, a phase-aware mixed-precision post-training quantization policy for activation quantization. PhaseQuant identifies quantization-sensitive layers from residual-stream dynamics and assigns 8-bit activation precision only to these layers, while keeping weights, KV caches, and all remaining activations at 4-bit precision. It can be directly integrated with strong PTQ baselines such as QuaRot, SpinQuant, and FlatQuant. Experiments on LLaMA-2 and LLaMA-3 show that PhaseQuant consistently improves perplexity and zero-shot QA performance under W4A4KV4 quantization, while achieving 1.05$\times$--1.07$\times$ throughput improvement with modest memory overhead. These results demonstrate a practical path toward robust low-bit LLM inference.