Preprint

ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads

Suayp Talha Kocabay Talha Rüzgar Akkus Kamer Ali Yüksel

Aug 2026 · 1 citation · 23 references

Computer Science

TL;DR

ARCHead is presented, a packed LM-head compressor that combines a quantized low-rank core, group-wise INT4 residuals, and a low-rank correction fitted in an activation-derived metric to complements block quantizers by compressing the large output projection they can leave untouched.

Abstract

Weight-only quantization substantially reduces the storage of large language model (LLM) transformer blocks, but practical backends often retain the final language-modeling head (LM-head) in BF16 or FP16. Quantizing this projection naively can strongly perturb the vocabulary-logit distribution. We present ARCHead, a packed LM-head compressor that combines a quantized low-rank core, group-wise INT4 residuals, and a low-rank correction fitted in an activation-derived metric. ARCHead stores no dense BF16 head and reduces persistent LM-head storage by 3.7-3.9x. On Qwen3-8B-Base, it uses 25.6% of BF16 head storage while attaining 1.007 relative perplexity; storage-matched naive INT4 yields 1.14-1.16. Replacing the BF16 head left by AWQ or bitsandbytes adds only 0.006-0.007 cross-entropy, with less than 2% throughput change in our measurements. ARCHead therefore complements block quantizers by compressing the large output projection they can leave untouched. Code is available at https://github.com/suayptalha/archead.

arXiv Semantic Scholar

View source

Similar papers

Jul 2026

RDQ: Residual Distribution Quantization for Large Language Models

RDQ (Residual Distribution Quantization), a PTQ framework whose central contribution is Cascaded Error Compensation, a sequential calibration procedure that captures the actual drifted activations each layer receives and fits per-channel AWQ-style scales against those drifted inputs, with scales folded into preceding R...

P. Singh · arXiv.org · 0 citations

#machine learning Preprint Sep 2026

UE5M3 FP4 Block Scaling for Stable Language Model Pretraining

Stable 4-bit floating-point (FP4) pretraining is difficult because the E2M1 payload represents only a narrow range of magnitudes. NVIDIA's Transformer Engine \nv{} recipe addresses this with current-tensor scaling, a randomized Hadamard transform (RHT), and bfloat16 (BF16) final layers, adding work outside the FP4 matr...

R. Hu, Carlo Luschi, Paul Balanca · 1 citation

Preprint Aug 2026

Every Expert Counts: ExactMoE for Memory-Efficient W4A16 Inference

ExactMoE, an inference design that applies symmetric group-128 four-bit weight quantization only to routed experts, stores those experts in kernel-native MARLIN form in pinned host memory, and executes all selected experts through a configurable GPU-resident slot cache and fused grouped MoE kernels, identifies a practi...

Amjad Saab · 0 citations

#small language model Preprint Aug 2026

SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs

SHIFT-LLM, a training-free post-pruning correction framework that inserts a Linear Residual Adapter at each pruning site, consistently recovers accuracy lost to depth pruning across most configurations, achieving gains up to +15.7 points on Llama-3.1-8B-Instruct.

Ali Bahri, Hang Li, Hongliang Li et al. · 0 citations

Preprint Aug 2026

F-WANDA: Fisher-Reweighted Post-Training Pruning for Sustainable Deployment of Large Language Models

F-WANDA is introduced, a drop-in modification of WANDA that reallocates the per-row keep budget across output neurons in proportion to the empirical Fisher information of the pre-activation, placing F-WANDA on the Pareto frontier of quality versus pruning cost for sustainable LLM compression.

Himanshu Mishra · 0 citations

Preprint Jul 2026

Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models

Scaling Large Language Models (LLMs) has been driven mainly by enlarging the Transformer backbone, but for an already-strong model this requires another round of costly pretraining. We study whether an existing backbone can keep improving by allocating more computation to each token while leaving the Transformer backbo...

Aiwei Liu, Cheng Shi, Chuhan Wu et al. · 2 citations