FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving
FluxMoE is presented, which decouples experts from physical GPU residency and adapts their footprint to available memory through a new \emph{expert paging} abstraction, and streams weights on demand while keeping expert computation on GPUs.
Qing-Xiu Liu, Yong-Chao He, Run-Han Jiang et al.
· 0 citations