BlockLock: Reproducible LLM Inference with Fixed Batch Geometry
BlockLock is presented, which keeps shared batched prefill and pins only the numerical path of the protected request, and needs no custom kernel implementations and constrains only the protected requests.
Xiu-Yuan Qi, Pedro Reviriego, Javier Conde et al.
· 0 citations