This paper describes two generations of production container networks deployed for over five years. Our first-generation network, Nimitz, used VxLAN for overlay-underlay mapping, providing flexibility and scalability from 2019-2022. However, with the rise of large-scale services (e.g., AI training and inference), Nimitz hit the C100K problem: beyond O(100K) containers, address mapping overhead and complex packet processing caused significant performance degradation. To overcome this, we built NetPila, a second-generation design that uses simple, practical IPv6 addressing to remove VxLAN encapsulation and overlay-underlay mapping tables. By embedding container addresses in the 128-bit IPv6 space, NetPila integrates the container network with the physical network, reducing packet-processing complexity and improving scalability. NetPila now supports millions of containers per tenant for production AI training and inference. We present the design, lessons, and deployment results of both generations.
Jiaqi Gao, Chao Qin, Sheng Cheng et al.· Proceedings of the ACM SIGCO...· 0 citations
Current GPU Collective Communication Libraries (CCLs) employ predefined schedules optimized for stable environments. Their supported schedules and selection logic are fixed at communicator initialization, which fails to account for evolving runtime conditions, such as workload characteristics and hardware health status. Consequently, long-running GPU jobs experience suboptimal performance after hours or days of execution, which translates into longer job completion times and wasted GPU cluster resources. To address this problem, we present Theseus, a novel CCL backend that provides schedule-level runtime adaptivity. It admits user-defined schedules and selection policies. As runtime conditions change, Theseus selects suitable schedules using cluster-wide runtime attributes beyond CCL-internal metrics. Moreover, it hot-swaps from the previous schedule consistently across GPUs with low overhead. Theseus acts as a drop-in replacement to facilitate integration. We evaluate Theseus extensively on various GPU workloads with intuitive policies. Compared with NCCL, Theseus achieves up to 1.61X speedup of communication time in stable environments and 2.46X in dynamic environments. It improves end-to-end job completion time by up to 1.84X while incurring comparable or lower overhead.
Rui Ding, Xiandong Lu, Jiajun Wang et al.· Proceedings of the ACM SIGCO...· 0 citations
EPIC mitigates imbalance via performance-aware expert migration and runtime expert activation, and then improves communication with topology-adaptive transport kernels and fine-grained computation-communication overlap.
Jiamin Cao, Qingxu Li, Yaozhong Liu et al.· 0 citations