Efficient serving of long-context reasoning-and-acting (ReAct) agents relies on KV cache reuse to reduce large language model (LLM) prefill latency and monetary cost. However, a semantic gap exists between agent harnesses and the underlying serving stack. Through context mutation, tool execution, and subagent coordinat...
Kai-Hua Fu, Yu-Kun Zhou, Chaokun Chang et al.· 0 citations
AgentPProf is a profiler that aggregates agent trajectories into pprof-compatible profiles, enabling flame graph visualization and analysis and introduces recursive operation segmentation, which recursively splits trajectories at task boundaries.
Yu-Sheng Zheng, Chaokun Chang, Yuan-Man Mao et al.· 0 citations
This work presents AgentSysBench, a benchmark suite and measurement toolkit with ten representative agentic applications and unified systems-level instrumentation, and identifies six properties that distinguish agentic workloads from conventional LLM serving.
Chaokun Chang, Yu-Kun Zhou, Kai-Hua Fu et al.· 11 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.