SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving
Mixture-of-experts (MoE) models activate few experts per token, yet batched decoding can access nearly the entire expert pool, making expert-weight traffic a major bottleneck. Expert pruning reduces this traffic, but conventional approaches also prune compute-bound prefill, sacrificing model quality for little throughp...
Gunho Park, Kyoungho Jeun, Juntaek Oh et al.
· 0 citations