Shared KV Caching for Replicated 27B Inference: Correctness Failures and Performance Boundaries
This engineering case study identifies practical validation steps and the locality conditions in which shared caching pays off and isolates a raw-pointer fallback that omits the dependency on the current CUDA stream.