Author

Runheng Liu

h-index 2 View on Semantic Scholar

We have 2 of 11 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#natural language process... Conference Open access May 2024

FlashBack: Efficient Retrieval-Augmented Language Modeling for Fast Inference

FlashBack is proposed, a modular RALM designed to improve the inference efficiency of RALM with appending context pattern while maintaining decent performance after fine-tuning by Low-Rank Adaption and introduces Marking Token as two special prompt tokens for marking the boundary of the appending context during fine-tu...

Runheng Liu, Xing-Chen Xiao, Heyan Huang et al. · Annual Meeting of the Associ... · 2 citations

Preprint Jul 2026

Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts

Results show that accounting for expert activation cost is important for efficient speculative decoding in large-scale MoE models, and a cost-aware speculative decoding framework that incorporates predicted marginal expert activation cost into draft selection is proposed.

Jincheng Xie, Runheng Liu, Heyan Huang et al. · 1 citation