Serving offline large language model (LLM) inference workloads (e.g., log summarization and bulk translation) can consume up to 30% of GPUs in production. Despite this significant share, the characteristics of offline inference remain largely understudied. In this paper, we start by analyzing 1.5 million tasks comprisi...
Le-Ping Yang, Xue Li, Kun Qian et al.· Proceedings of the ACM SIGOP...· 0 citations
By embedding container addresses in the 128-bit IPv6 space, NetPila integrates the container network with the physical network, reducing packet-processing complexity and improving scalability.
Jiaqi Gao, Chao Qin, Sheng Cheng et al.· Conference on Applications,...· 0 citations
EPIC mitigates imbalance via performance-aware expert migration and runtime expert activation, and then improves communication with topology-adaptive transport kernels and fine-grained computation-communication overlap.
Jia-Min Cao, Qingxu Li, Yaozhong Liu et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
