UniRank: Unified Rank Allocation for Low-Rank LLM Compression
It is shown, both geometrically and empirically, that high input--output cosine similarity implies low effective rank, and rank-preserving fine-tuning (RPFT) is proposed, which adapts only a small subset of retained singular components so that the allocated rank stays bounded without re-decomposition.
Chao Han, Yong-Jie Du, Jun-Jie Tan et al.
· 0 citations