Fang-Hua Ye

Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution

Speculative decoding accelerates autoregressive generation by using a lightweight draft model to propose multiple tokens that are verified by a target model in parallel. However, the standard acceptance rule focuses on exact distribution correction and rejects tokens that remain highly plausible under the target model...