Preprint
Aug 2026
HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Management
HiSparse is merged into upstream SGLang and evaluated across three sparse-attention families (DSA, NSA, and Quest) on H200, B200, and GH200 platforms: it improves peak generation throughput by up to 4.7x on long-context workloads while preserving comparable per-token latency and reducing time-to-first-token at high load.
Zhiqiang Xie, Zhangheng Huang, Ting-Jun Huang et al.
· 2 citations