Preprint
Jul 2026
Talaria: Session-Aware Serverless Serving of Hundred-Billion-Parameter LLMs
Talaria is presented, a session-aware serverless multi-model serving system that makes session continuity a joint placement-and-admission decision, and its router ranks placements by model residency, KV locality, and instance pressure.
Utopia Meng, Unicornt Zhao, Derek Li et al.
· 0 citations