Production LLM serving multiplexes hundreds of heterogeneous models on shared clusters, exposing three challenges that existing systems fail to address simultaneously: unpredictable bursts, power-law application popularity, and heterogeneous yet complementary resource demands. We present Janus, a Service-Engine co-desi...
Tian-Bao Zhou, Yi Wang, Yu Zhou et al.· Proceedings of the ACM SIGOP...· 0 citations
The first fully automated framework that synthesizes high-quality, proof-centric benchmarks from natural language mathematical corpora and a new type of hybrid-formatted questions, named ``$m$-out-of-$n$ multiple judge questions'', specifically designed to enable robust, automatic evaluation while being resilient to gu...
Ye-Bo Peng, Zixiang Liu, Yao-Ming Li et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.