Towards Optimal Multi-draft Speculative Decoding
The results show that the draft sampling method strongly influences the optimal acceptance rate, with sampling without replacement outperforming sampling with replacement, and the theoretical upper bound of MDSD efficiency is measured for the first time.