Aegis: Contract-Bounded Online Adaptation for Networked Accelerator Clusters

Abstract

Accelerator clusters generate bursty network traffic: distributed training exchanges gradients via all-reduce, inference pipelines fan out requests, and storage competes with latency-sensitive RPCs. Poor placement amplifies congestion: a mis-scheduled training job can raise ECN mark rates by 3× and inflate co-located service tail latencies by 40%. Yet schedulers ignore real-time fabric state. We present Aegis, a scheduler that adapts placement using live fabric telemetry under operator-defined contracts. Aegis ingests per-ToR queue depths and ECN counters at 1-second granularity, incorporates a congestion-pressure term into placement scoring, and evolves online via a conservative bandit with exposure control. Contracts on tail latency, fairness, and congestion compile into safety envelopes that gate rollouts and trigger automatic rollback. We evaluate Aegis on an 80-node cluster (160 A100-80GB GPUs, 160 TPU v4 chips, 2-tier Clos fabric). Under production-derived workloads, Aegis reduces service p99 RPC latency by 27.3% (152.4 ms vs. 209.6 ms), cuts SLO violations by 3.6×, and lowers ECN mark rate by 51% while improving utilization from 68.4% to 79.1%. These gains hold under workload shifts and telemetry delays up to 60 seconds. The design is scoped to shared clusters where placement can materially reshape fabric contention.