Preprint
Oct 2026
SPARE: Sparsity Allocation via Routing Entropy for Efficient MoEs
Mixture-of-Experts (MoE) architectures scale model capacity efficiently, yet their substantial memory footprint poses a major barrier to deployment. Expert pruning reduces this footprint, but most methods focus on which experts to remove and prune every layer equally. We propose SPARE (Sparsity Allocation via Routing E...
Yi-Fan Yang, Han-Ling Yi, Dong-Quan Yang et al.
· 0 citations