This work proposes SalQ-VLM, an importance-aware PTQ framework that prioritizes salient tokens and suppresses redundant vision tokens during calibration, and derives a gradient-driven importance factor that captures token-level importance variance and is theoretically grounded in the relationship among loss perturbatio...
Yufei Xue, Yu-Shi Huang, Lu Zhu et al.· 5 citations
PIVOT, Proxy Indexing Via One full-prefix Traversal, a training-free, drop-in replacement for the DSA indexer that shares one prefix scan across a group of nearby queries, matches the accuracy of the dense DSA indexer while accelerating it by up to 4x and reducing end-to-end latency by up to 1.6x at long context.
Hong Liu, Yuan Cheng, Lin Niu et al.· arXiv.org· 3 citations
CoSA is proposed, a two-stage training-free Sparse Attention under proxy-kernel CO-design, which couples a Kernel-Aware Proxy (KAP) with an Ordered-Skipping Kernel (OSK) and achieves a 4.93% attention speedup and reduces end-to-end Time-to-First-Token by 2.53% with negligible performance degradation.
Yufei Xue, Lin Niu, Hong Liu et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.