A data pipeline service that improves aggregate training throughput by dynamically allocating pipeline workers across jobs by using online profiling to identify jobs bottlenecked by the data pipeline and to estimate the throughput gain they would achieve from additional workers.
DeepShare is a scheduler that uses a continuous tenant-assurance signal to coordinate these decisions at runtime to achieve a more advantageous utilization-QoS trade-off than optimizing quotas, scheduling, and resource sharing independently.
Jing-Hao Wang, Yi-Hang Zhou, Xiao Zhou et al.· IEEE International Conferenc...· 0 citations
In this paper, we study a mixed-prompt scenario—where both short and long prompts coexist—in an LLM inference serving system that supports diverse applications with heterogeneous iteration-time SLOs. To improve throughput for long prompts, prior work divides them into chunks and batches requests or chunks to meet the t...
Hai-Ying Shen, Tanmoy Sen, Yuxiong He· International Conference on...· 0 citations
Modern deep learning workloads are increasingly co-located on a single GPU to improve hardware utilization, yet existing MPS-based sharing mechanisms expose concurrency without providing QoS-aware control. Deadline-constrained training jobs can suffer severe slowdown when co-executed with best-effort workloads, because...
De-Sheng Wang, Fu-Zhen Yong, Si-Chao Chen et al.· ACM Transactions on Architec...· 0 citations
SIF is introduced, a metric built on the notion of partial-nodes that is independent of historical workload knowledge that matches both theory and production and COMPASS-ABS is proposed, which employs the COMPact-ASSured (COMPASS) algorithm to confine the cluster state within a tight Anchor-Based Space (ABS).
The growing adoption of artificial intelligence in space communications has imposed increasingly demanding efficiency requirements on deep learning training for tasks such as intelligent signal processing and resource allocation. However, performance disparities and workload fluctuations across heterogeneous nodes caus...
Rui-Ming Wen, Wen-Jing Hou, Hong Wen et al.· Aerospace· 0 citations
Orthrus is a serving system that performs heterogeneous batching within a unified inference loop that addresses the primary challenge lies in unifying embedding and generation workloads with conflicting computational patterns while optimizing batch composition for high performance.
Dohyun Park, Hubertus Franke, Daniel G. Waddington et al.· Proceedings of the 2026 Conf...· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
