Due to the high operational complexity and limited deployment scale of lossless RDMA networks, the community has been exploring efficient RDMA communication over lossy fabrics. State-of-the-art (SOTA) lossy RDMA solutions implement a simplified selective repeat mechanism in RDMA NICs (RNICs) to enhance loss recovery efficiency. However, these solutions still face performance challenges, such as unavoidable ECMP hash collisions and excessive retransmission timeouts (RTOs). In this paper, we enhance RDMA reliability with the goals of being independent of PFC, compatible with packet-level load balancing, free from RTO, and friendly to hardware offloading. To this end, we propose DCP, a transport architecture that co-designs both the switch and RNICs, fully meeting the design goals. At its core, DCP-Switch introduces a simple yet effective lossless control plane, which is leveraged by DCP-RNIC to enhance reliability support for high-speed lossy fabrics, primarily including header-only-based retransmission and bitmap-free packet tracking. We prototype DCP-Switch using P4 switch and DCP-RNIC using FPGA. Extensive experiments demonstrate that DCP achieves $1.6\times $ and $2.1\times $ performance improvements, compared to SOTA lossless and lossy RDMA solutions, respectively.
Wenxue Li, Xiangzhou Liu, Yunxuan Zhang et al.· IEEE Transactions on Network...· 0 citations
Prefix caching has become a key technique for LLM serving, and nowadays the reusable KVCache contents are often hosted on distributed servers. For long-context LLM inferences with high cache hit ratio, cross-server KVCache transmission has become an emerging performance bottleneck; such network-intensive LLM inferences are increasingly prevalent in the coming era of agentic AI. However, existing LLM inference engines are essentially compute-centric; we find that they are highly inefficient when serving such workloads due to compute-stage service blocking and ignorance of KVCache-transfer cost. To efficiently serve network-intensive LLM inferences, in this paper, we design Sanic, an optimized LLM engine that treats KVCache transmission as a first-class citizen. Viewing KVCache loading and computation as equally-significant stages, Sanic decouples their service control and allows each stage to progress autonomously in an asynchronous manner, thereby improving the overall resource utilization. Moreover, when scheduling competing LLM inferences, Sanic treats the KVCache loading delay as an independent factor in service cost modeling, which is more accurate and can yield better scheduling decisions. Our testbed experiments with diverse benchmarks show that, Sanic can substantially enhance the service efficiency of network-intensive LLM inferences, improving the SLO-attainment by up to 61.67%.
Weiye Wang, Chen Chen, Junxue Zhang et al.· Asia-Pacific Workshop on Net...· 0 citations