Zero Sum SVD (ZS-SVD) is proposed, a post-training method that performs global singular component selection using activation whitening and first-order calibration loss estimates in whitened coordinates, automatically yielding heterogeneous ranks without solving a rank allocation optimization.
Ali Abbasi, Chayne Thrash, Hao-Ran Qin et al.· arXiv.org· 9 citations· ⚡1
Large language models are increasingly expensive to serve. In large-scale serving systems, autoregressive decoding is often bottlenecked by transferring model weights from accelerator high-bandwidth memory into on-chip SRAM. Mixture-of-experts (MoE) models reduce computation by activating only a small subset of experts...
Ali Abbasi, Justin Shi, Soheil Kolouri· 1 citation
CircuitSteer is introduced, a novel framework that leverages Sparse Autoencoders (SAEs) to identify and manipulate coherent semantic circuits distributed across multiple layers that yields strictly more robust and effective behavioral control than static single-point interventions.
Mehrshad Saadatinia, Parsa Razmara, Ardalan Aryashad et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.