FedSubMuon: Communication-Efficient Federated LLM Fine-Tuning via Structured Subspace Muon
FedSubMuon is proposed, a communication-efficient federated Muon fine-tuning method that optimizes compact coefficient matrices within shared structured subspaces that keeps Muon on a single matrix-valued trainable object, while reducing the client upload to compact coefficient matrices.
Shao-Long Chen, Youming Tao, Shu-Zhen Chen et al.
· 0 citations