Prefix caching is critical for efficient large language model (LLM) serving, particularly for agentic workloads that repeatedly invoke the model with a growing conversation and tool-use history. By reusing the key-value (KV) states of previously processed prefixes, prefix caching avoids redundant prefill computation. I...
Lu-Chang Li, Shuai-Shuai Wang, Zhao Ruan et al.· 0 citations
The results suggest that the accuracy--cost trade-off of long-term agent memory depends not only on what information is retained, but also on the granularity at which it is consolidated, and that the accuracy--cost trade-off of long-term agent memory depends not only on what information is retained, but also on the gra...
Dongfang Li, Zi-Xuan Liu, Junmai Wang et al.· 3 citations
This work demonstrates a full-stack pathway from efficient trillion-parameter model post-training on Ascend infra to domain-specialized Flash models for solver-grounded mathematical modeling, advancing frontier-model systems for complex reasoning.
Dongfang Li, Xiaodong Luo, Ruoyu Sun et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.