Long context inference with large language models becomes increasingly expensive as attention must operate over an ever growing KV cache. Page sparse attention reduces this cost by representing each KV page compactly and retrieving only a subset for each query. Existing retrieval methods are designed to estimate attent...
Zero Sum SVD (ZS-SVD) is proposed, a post-training method that performs global singular component selection using activation whitening and first-order calibration loss estimates in whitened coordinates, automatically yielding heterogeneous ranks without solving a rank allocation optimization.
Ali Abbasi, Chayne Thrash, Hao-Ran Qin et al.· arXiv.org· 9 citations· ⚡1
This work characterize the effect of a residual-stream perturbation through both its activation covariance and the local sensitivity of the model output, and derives a tractable spectral upper bound that reduces subspace selection to an eigendecomposition of a sensitivity-weighted covariance matrix, retaining the effic...
Chayne Thrash, Ke Chen, Soheil Kolouri· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.