Author

Sheng Zha

h-index 15 View on Semantic Scholar

We have 1 of 30 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Preprint Aug 2026

Compute-Optimal Is Not Cluster-Optimal: Systems-Aware Scaling for Sparse Mixture-of-Experts

MOSAIC is developed, which formulates model architecture and systems co-design as an optimization problem that couples a predictive scaling law with a calibrated performance model that estimates Model FLOPs Utilization (MFU), communication cost, memory footprint, and the best parallel layout.

Soumajyoti Sarkar, Yu-Xin Tang, Sheng Zha · 0 citations