Federated Compositional Muon Optimizer for Matrix-Wise Models
This work proposes an effective federated compositional Muon (FedCoMuon) optimizer to solve distributed matrix-wise compositional optimization problems and proposes a variance reduced variant of FedCoMuon (FedCoMuon-VR) based on a momentum-based variance reduced technique.
Wang Yan, Feihu Huang
· 0 citations