Preprint
Aug 2026
FlashQuant: Sparse-Dense Fusion for Memory-Efficient Outlier-Aware LLM Inference
FlashQuant fuses the dense GEMM and sparse outlier SpMM paths into a single GPU kernel, enabling on-chip reuse of activation and output tiles across heterogeneous computations, and introduces three key techniques: sparse-dense tiling, which aligns outlier processing with dense GEMM tiles, and pipelined scheduling, which overlaps computation with data movement.
Jun-Qing Lin, Jingwei Sun, Zhengding Hu et al.
· 0 citations