Eloise Stein

Workload-Static Optical Circuit Switching For Scalable LLM Inference Decode

LLM inference decode executes fine-grained collectives whose latency directly limits per-user interactivity, yet scale-up fabrics are still built from electronic packet switches designed for training. We observe that, for a given model deployment and parallelization, the communication structure of decode is fixed, and...