Seeing What Matters, Verifying What Pays: Load-Adaptive Speculative Decoding for Vision-Language Models
Speculative decoding accelerates vision-language model (VLM) inference without altering the output distribution, yet two factors limit its gains. Existing drafters either lack the image evidence required for visually grounded tokens or pay to condition on every visual token, although a short draft block typically depen...