LLM agents automate GPU kernel optimization by repeatedly composing candidates and measuring their duration on real GPUs. Existing systems preserve measurement fidelity by reserving a GPU for an entire agent session or benchmarking command. However, this results in poor utilization because only a small fraction of comm...
Tian-Yu Feng, Hao-Xuan Yu, Tian-Yuan Wu et al.· 0 citations
Efficient serving of long-context reasoning-and-acting (ReAct) agents relies on KV cache reuse to reduce large language model (LLM) prefill latency and monetary cost. However, a semantic gap exists between agent harnesses and the underlying serving stack. Through context mutation, tool execution, and subagent coordinat...
Kai-Hua Fu, Yu-Kun Zhou, Chaokun Chang et al.· 0 citations
Model-as-a-service platforms call for continuous optimization as complex serving conditions expose inefficiencies missed before deployment. Detailed always-on profiling can incur substantial overhead, while lightweight collection omits information needed for diagnosis. We present Herschel, a continuous optimization sys...
Lu-Ping Wang, Wei-Gao Chen, Yi-Fei Wu et al.· 0 citations
Weave is presented, to the authors' knowledge the first MoE overlap system that performs fine-grained dynamic SM scheduling - deciding per layer and per GPU by routing results at runtime, and achieves a 2.89x geometric-mean MoE-layer speedup and a 1.33x geometric-mean end-to-end speedup over five state-of-the-art basel...
Ziyu Huang, Yang-Jie Zhou, Chen-Hao Zhu et al.· 0 citations
Atrex-Bench is presented, a benchmark whose 30 operators and 440 shapes are sampled directly from full-cluster production inference traces of compute-limited, memory-rich GPUs, and a profile-driven kernel-optimization agent that combines iterative measure-revise search, optimization dropout for escaping stalled search...
By embedding container addresses in the 128-bit IPv6 space, NetPila integrates the container network with the physical network, reducing packet-processing complexity and improving scalability.
Jiaqi Gao, Chao Qin, Sheng Cheng et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
