Unified Memory Architecture (UMA) enables the CPU and GPU to share a single physical memory. However, its complexity and opacity often drive applications back to inefficient copying or to incorrect direct access that sacrifices performance or even correctness. This paper presents a comprehensive study of CPU-GPU memory...
Jin-Rong Yang, Zi-Meng Wang, Rong Chen et al.· 2 citations
Atomic profiling is presented, which lets agents share a GPU server while making each measured interval dedicated-equivalent and eliminates the five unrecovered out-of-memory failures seen under uncontrolled concurrency.
Ying-Yi Hao, Xing-Da Wei, Rong Chen et al.· 0 citations
This paper shows that learned search can support arbitrary K with the preprocessing cost of a single top-1 model, and reduces total serving and preprocessing computation by up to 28%.
Yifan Peng, Jia-Fei Fan, Xing-Da Wei et al.· 0 citations
SMetric addresses session-centric scheduling with differential scheduling based on two indicators derived from the request itself, the session turn and the local KV\$ hit: it schedules first-turn requests for load balance, and sticks follow-ups to the instance with the highest local hit for high KV\$ reuse.