BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration
Speculative decoding accelerates autoregressive generation by using a lightweight draft to propose multiple tokens for parallel verification. However, existing methods often require an additional draft model or weight representation, introducing non-negligible memory overhead on resource-constrained devices. Self-specu...
Chen-Ce Yang, Ning-Xi Cheng, Arash Akbari et al.
· 0 citations