Preprint
Sep 2026
Semantics, Workflows, and Infrastructure: Understanding Agent Serving at Production Scale
Large language model (LLM) agents execute applications through a workflow of inference requests with tool calls and user interactions. Serving these applications at production scale requires understanding how application behavior shapes inference demand and for guiding efficient execution. Recent characterization studi...
Yi-Hao Zheng, Jing-Zhe Jiang, De-Jiang Zhu et al.
· 0 citations