Approximate Speculative Decoding
Approximate Speculative Decoding (ASD) is introduced, a training-free verifier that replaces binary first-mismatch truncation with budgeted longest-prefix selection and reuses the contiguous target-greedy suffix without additional approximate decisions or target-model forward passes.
Yuannuo Feng, Zegang Peng, Yuxin Xie et al.
· 0 citations