We have 3 of 48 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Ofan, a switch-based destination-aware LB scheme that can reach $O(1)$ queueing, is suggested, a switch-based destination-aware LB scheme that can reach $O(1)$ queueing.
FairInference provides the novelelta-token fairness guarantee: for a well-behaved client, if a token is generated in d time units in isolation, it will be generated within d + {\delta} time units in multi-tenant execution, providing strong latency isolation guarantees for LLM serving.
Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granularity of kernels, on separate streams, reduces only part of this communication cost. Fused kernels often have better performance by transmitting each output tile as soon a...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
