Wei Ni

Redundancy Meets Synergy: Dependency-aware Expert Selection for MoE via Submodular Optimization

While Mixture-of-Experts (MoE) models effectively scale model capacity through sparse activation, their deployment is often bottlenecked by prohibitive memory requirements. Extracting a compact subset of experts presents a promising solution. However, existing expert selection heuristics predominantly rely on Top-k ran...