Spectral Tail Interventions in Decoder-Only Language Models: Reasoning-Sensitive Weight Structure from Controlled Surgery
A finite-width conditional bound linking the inverse participation ratio of squared singular values to central pre-softmax logit kurtosis is derived and a pointwise query--key product-tail target is defined and compared with independent factor surgery with a product-targeted factorization that preserves native attentio...
Ibne Farabi Shihab, Sanjida Akhter, Md Najmus Swaqeeb et al.
· 0 citations