Preprint
Jul 2026
StreamDQ: Near-Memory Weight DeQuantization in Custom HBM for Scalable AI Inference Acceleration
StreamDQ is proposed, a lightweight architectural enhancement that enables on-the-fly dequantization in the memory subsystem for high-throughput, large-batch LLM inference and reduces latency and improves decode throughput for end-to-end LLM inference.
Minki Jeong, Daegun Yoon, Soohong Ahn et al.
· 0 citations