The conventional threading model multiplexes software threads onto hardware cores. This model inherently suffers from the overhead of 1) context switching, 2) scheduling, and 3) system event notifications (e.g., I/O interrupts). As computing enters μs-scale, such overheads become the key bottleneck of a wide range of d...
Yi-Ming Yao, Xiao-He Qin, Yi Fan et al.· Proceedings of the ACM SIGOP...· 0 citations
RedKnot establishes a new foundation for AI infrastructure by transforming the KV cache from a monolithic, passive runtime artifact into a dynamic, model-aware runtime substrate for scalable LLM serving.
Yang Liu, Zhao-Kai Luo, Hua-Yi Jin et al.· arXiv.org· 7 citations· ⚡1
Tessera is presented, the first kernel disaggregation system to improve performance and cost efficiency on heterogeneous GPUs for large model inference, and shows that a heterogeneous GPU pair under Tessera can even exceed the throughput of two homogeneous high-end GPUs at a lower cost.
Tian-Cheng Hu, Jin Qin, Zheng Wang et al.· arXiv.org· 0 citations
Evaluating twelve open and proprietary models, it is found that abstraction failure is lawful, and the central finding is a gap between recognition and cross-voice mapping: models identify a structure within one voice far better than they carry it across voices; every model pays this tax, and mapping strong enough to n...
Tong Zhang, Zhiyuan Shi, Yun Peng et al.· 0 citations
This work presents a DeepSeek-V4 realization of RedKnot's head-aware reuse principle, a DeepSeek-V4 realization of RedKnot's head-aware reuse principle, and describes the factorization, position repair, token-row closure, sparse-MoE support, TP8 integration, and the measurement boundaries needed to interpret these resu...
Yang Liu, Zhao-Kai Luo, Hua-Yi Jin et al.· 0 citations
StrataCL introduces registration-on-allocation to realize user-buffer direct communication, and designs communication operators with workload-balanced NPU-core partitioning and NPU-driven SDMA offloading to exploit supernode architecture features.
Tian-Cheng Hu, Jin Qin, Yu-Zheng Wang et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.