Xiao-He Hu

Collective Communication for Distributed LLM Systems: Planning, Runtime Adaptation, and Computation Coordination

This paper presents a tutorial-style, collective-centric taxonomy for collective communication, which organizes recent advances into three layers: communication planning, which generates topology-aware collective schedules; communication execution and adaptation, which maps these schedules onto GPU runtimes and hardwar...