TopoEP: Topology-Aware Load Balancing for Expert-Parallel MoE Training
A GPU-native, topology-aware load-balancing system for large-scale MoE training that converts the current routing result into hot-expert replication and token-rerouting decisions and executes the resulting plan without data-dependent host synchronization, reducing critical-path overhead.
Jia-Cheng Zhu, Xie Zhao, Gong-Ming Zhao et al.
· 0 citations