Mixture-of-Experts (MoE) inference on memory-constrained edge platforms is often dominated by expert-weight movement rather than computation. Although each token activates only a small subset of experts, routing is input-dependent, so all experts must remain accessible and memory movement dominates latency. Existing ap...
Cong-Wei Zhang, Youming Tao, Yi-Fei Zou et al.· Proceedings of the 32nd Annu...· 1 citation
A comprehensive study of DP-(finite-sum)-DRO with $\psi$-divergence and non-convex loss, and a novel DP Double-Spider optimization method, called DP Double-Spider, tailored to this structure, which achieves a utility bound matching the best-known result for non-convex DP-ERM.
Di-Fei Xu, Meng Ding, Zebin Ma et al.· arXiv.org· 2 citations
Two stable zeroth-order learning algorithms are designed and proved exact unlearning for both algorithms: the model and saved state have the same joint distribution as retraining with the corresponding learner on retained data.
Wei-Jun Yin, Yan-Qing Yao, Falko Dressler et al.· 0 citations
Mixture-of-Experts (MoE) inference on memory-constrained edge platforms is often dominated by expert-weight movement rather than computation. Although each token activates only a small subset of experts, routing is input-dependent, so all experts must remain accessible and memory movement dominates latency. Existing ap...
Cong-Wei Zhang, Youming Tao, Yi-Fei Zou et al.· Proceedings of the 32nd Annu...· 1 citation
FedSubMuon is proposed, a communication-efficient federated Muon fine-tuning method that optimizes compact coefficient matrices within shared structured subspaces that keeps Muon on a single matrix-valued trainable object, while reducing the client upload to compact coefficient matrices.
Shao-Long Chen, Youming Tao, Shu-Zhen Chen et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.