StrataCL introduces registration-on-allocation to realize user-buffer direct communication, and designs communication operators with workload-balanced NPU-core partitioning and NPU-driven SDMA offloading to exploit supernode architecture features.
Abstract
Modern distributed AI workloads run across hundreds of accelerators, making communication a major bottleneck. Existing communication libraries remain largely buffer-centric because user and communication buffers are managed separately, causing redundant data copies or costly user-buffer registration. This paper presents StrataCL, a zero-redundancy and fabric-native communication library for production supernodes. StrataCL introduces registration-on-allocation to realize user-buffer direct communication, and designs communication operators with workload-balanced NPU-core partitioning and NPU-driven SDMA offloading to exploit supernode architecture features. On the Huawei CloudMatrix384, StrataCL improves collective bus bandwidth by up to 1.6x and improves MoE dispatch/combine bus bandwidth by up to 1.4x. Across three production workloads, StrataCL improves LLM inference throughput by 1.9x, reduces P99 TTFT by 2.2x, and reduces LLM and Recsys training iteration time by 1.4x and 1.3x, respectively.
This work presents CCSwitch, a modular switching fabric built from 4×4 non-blocking Collective Engines, a modular switching fabric built from 4×4 non-blocking Collective Engines (CEs) that combines spatial and temporal parallelism to perform reductions without accumulation buffers.
Sumukh Pinge, Hardik Soni, Bob Lantz et al.· Conference on Applications,...· 0 citations
A hardware-software co-design framework that unifies distributed compute, memory, and communication into a seamless processing-communication fabric is presented and a prefill-decode disaggregation approach that dynamically reconfigures PE organizations to maximize resource utilization is presented.
PReCCL is a drop-in NCCL replacement that combines software inband telemetry with cross-VT workload reallocation, and implements in-band monitoring within the CCL, and precisely measures the stall counts of each VT, and piggybacks the telemetry meta-data on existing collective traffic.
Zhiyong Chen, Kaihui Gao, Li Chen et al.· Conference on Applications,...· 0 citations
STORM is presented, a NIC-level scheduler for all types of RDMA workloads using NIC-only information: the known RDMA request size, and per-queue-pair backlog, and converts these signals into a small number of extra priority levels on the wire and prioritizes requests that are either near completion or blocking queued d...
Jichun Wu, Ran Shu, Gianni Antichi et al.· Conference on Applications,...· 0 citations
Multi-LoRA serving is how one base model becomes thousands of specialized variants, one adapter per user, task, or agent, and the deployments can hold 1000-plus adapters. Serving them is hard because the workload inverts what GPUs provide: terabytes of memory against only tens of TFLOPS, and because every published sys...
Zhong-Kai Yu, O. Venkatachalam, Zheng Wang et al.· 0 citations
This paper presents the first systematic study showing that memory contention, not network bandwidth, is the dominant bottleneck in message aggregation runtimes, and advocates for dedicated near-memory subsystems to improve the scalability and performance of message aggregation runtimes.
Shubhendra Pal Singhal, Aaron Welch, Oscar Hernandez et al.· IEEE International Symposium...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
