Giganetes is a GPU-centric architecture for stateless LLM serving that externalizes KV caches to a disaggregated remote memory pool via GPUDirect RDMA. By treating remote memory as a GPU-addressable tier via GPUDirect RDMA, Giganetes eliminates session affinity constraints: any GPU can serve any request, enabling near-linear horizontal scaling in a Kubernetes-native deployment. A Scatter/Gather I/O interface bypasses the CPU and host memory entirely, achieving 52.4 GB/s application-level read throughput on our 4×200 Gbps RDMA testbed. A session-level metadata abstraction and proactive readahead mechanism reduce GPU bubbles by overlapping remote KV fetches with prefill computation and scheduling slack. On a 4-node H800 cluster, Giganetes delivers 33% higher throughput (QPS 2.4 vs. 1.8) and 1.75× lower P95 TPOT than PD-disaggregation with sticky sessions, with the gain driven by scheduling flexibility rather than faster transport alone.
Xiaoliang Wang, Zhen-Wei Pi, Rui Zhang et al.· Conference on Applications,...· 0 citations
Remote Direct Memory Access (RDMA) is widely adopted for high-performance datacenter workloads, but its strict ordering requirement conflicts with multipath transport in lossy networks. RDMA Network Interface Cards (RNICs) may interpret Out-of-Order (OOO) arrivals caused by diverse path latencies as packet loss. This triggers unnecessary retransmissions, severely degrading goodput collapse and inflating Flow Completion Time (FCT). We present PSN-PATH, a novel RDMA protocol to distinguish packet loss from OOO and avoid unnecessary retransmission, improving goodput and FCT of multi-path RDMA in lossy networks. PSN-PATH maps Packet Sequence Numbers (PSN) to network paths, enabling the receiver to verify packet continuity within each mapped path and distinguish real losses from OOO arrivals. PSN-PATH only retransmits lost packets and redirects traffic away from degraded paths. We implement PSN-PATH on Xilinx Alveo U50 FPGA using Vitis HLS and validate it via extensive simulation experiments. Experimental results demonstrate that PSN-PATH significantly outperforms existing solutions, reducing FCT by 95.81%, enhancing goodput by 5.11×, and reducing retransmissions by over 99% under high-load, lossy conditions.
Zhexiong Li, Shugui Wei, Puyu Zhao et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
