Skip to content
Book Open access

EPTQ: Fast and Accurate 2-bit Post-Training Quantization via Factored E8 Lattice

Nov 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · pp. 3873-3883 · 0 citations · 12 references

Abstract

Two-bit Post-Training Quantization (PTQ) offers an attractive path to deploying Large Language Models under strict memory budgets, yet scalar quantization suffers from catastrophic performance collapse at this extreme bit-width. While Vector Quantization (VQ) methods resolve the accuracy issue, existing approaches suffer from practical inefficiencies: complex optimization pipelines result in hours-long quantization, and some methods exhibit severely limited inference throughput. We propose EPTQ, a 2-bit PTQ framework available in two complementary variants (E8 and FE8), achieving high accuracy and practical efficiency through three contributions: (1) Factored E8 (FE8) lattice quantization, which decomposes the E8 codebook into a compact 4, KB structure that fits in the GPU L1 cache, enabling fast codebook lookup at inference time; (2) Lightweight scale normalization via row/column scaling vectors (g, h), which aligns weight distributions to maximize E8 quantization efficiency, requiring only element-wise per-token scaling at inference; and (3) Adaptive critical weight preservation via knee-point threshold detection, which more accurately identifies Hessian-sensitive weights than approaches relying on fixed global criteria, consistently improving perplexity across models, with zero bit-overhead by encoding the preservation mask into the existing codebook index space. Evaluated across multiple LLMs including Llama-2 and Llama-3 families, EPTQ (E8) achieves accuracy competitive with or surpassing QTIP on Llama-2 models, while EPTQ (FE8) surpasses competing VQ methods in decode throughput on most models, achieving up to 38% faster inference than FP16, with both variants reducing quantization time by over 80% compared to competing approaches.

Read PDF

Similar papers

#artificial intelligence Preprint Oct 2026

E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks

Low-bit quantization reduces the memory footprint of large language models, but preserving accuracy with efficient training remains challenging. Scalar quantization encodes weights independently, while explicit-codebook vector quantization introduces costly search during training. We propose E8-QAT, a 2-bit quantizatio...

Xinyu Lei, Hong Huang, Jie-Bin Zhang et al. · 1 citation
#artificial intelligence Preprint Oct 2026

E8 Lattice Key Quantization for Low-Bit KV-Cache Compression

Compressing the key-value (KV) cache is essential for serving long-context large language models (LLMs) under memory constraints. While current scalar quantization methods achieve significant compression, they typically face a quality-fidelity trade-off, either suffering from performance instability at 2-bit precision...

Sasho Nedelkoski, Alexander Acker, Soeren Becker et al. · 0 citations

JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization

JustQuant is proposed, a simple yet effective framework that moves the complexity of low-bit quantization from deployment-time operators into the training process and introduces Theseus QAD, a quantization-aware distillation method that progressively applies multi-level supervision, analogous to the gradual replacement...

Kai-Cheng Yang, Kai-Sen Yang, Chun-Yu Liu et al. · 0 citations
#machine learning Preprint Oct 2026

SQuAT: Spherical Lattice Vector Quantization-Aware Training

Vector quantization (VQ), in particular lattice-based methods, offer strong rate-distortion trade-offs for low-bit large language models (LLMs), yet their behavior under quantization-aware training (VQ-QAT) remains poorly understood. We scale lattice VQ-QAT to billion-parameter LLMs and multi-billion-token training reg...

Sebastian Loeschcke, Ivan Koryakovskiy, T. V. D. van der Ouderaa et al. · 0 citations
#machine learning Preprint Nov 2026

HADVQ: Hessian Aware Directional Vector Quantization

Ultra-low-bit quantization of large language models is a challenging task because aggressive compression severely distorts model parameters and disproportionately harms a small number of highly sensitive channels. We present a vector quantization method tailored for this regime. Our approach begins with random precondi...

Daniil Li, Mars Davletshin, Vadim Klyzhenko et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.