Preprint
Jul 2026
CoCoScale: Leveraging Layer-wise Scaling to Unlock the Potential of Online LLM Serving
CoScale is proposed, a layer-wise dynamic scaling mechanism that selectively expands the parallelism of hot layers onto idle resources reclaimed from underutilized devices, enabling elastic data parallelism without altering model architectures or adding hardware overhead.
Jingfeng Wu, Yiyuan He, Minxian Xu et al.
· 0 citations