ROCm InferaSim: Cut LLM Serving Trial Costs and Deploy the Best Configuration
A five-candidate short list contains a deployment within 12% of the best feasible pairing of model and latency target, and a five-candidate short list contains a deployment within 12% of the best feasible pairing of model and latency target.
Anshu Raina, Yao Fu, Jie-Jing Zhang et al.
· 0 citations