Large Mixture-of-Experts (MoE) models are promising for local inference because sparse routing allows the routed-expert pool to reside on SSD while DRAM caches recently used experts. However, under practical DRAM cache capacities, substantial SSD I/O remains on the critical path despite I/O-computation overlap. Token-l...
Yu-Tong Lin, Yingdi Shan, Shao-Yuan Chen et al.· 0 citations
This work proposes to decouple low latency from fault tolerance, and design a hybrid solution that seamlessly integrates client-coordinated two-phase commit (2PC) with consensus, respectively to reduce latency in normal situations and tolerate faults in faulty situations, and designs xRaft, which can adaptively switch...
Yan Chen, Xinyi Yu, Shengyun Liu et al.· Conference on Applications,...· 0 citations
This work develops a non-clairvoyant scheduling algorithm that does not know task work in advance, yet loses only a small additive term relative to the clairvoyant lower bound, and proves a performance upper bound under uniform random data placement.
H. Kang, Yiwei Zhao, Kunal Agrawal et al.· ACM Symposium on Parallelism...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
