EStream: Fast and Memory-Efficient MoE Prefill through Expert Virtualization on Mobile NPUs
Mobile vendors and application developers increasingly deploy LLMs on smartphones for diverse prefill-only services. Yet current systems rely mainly on dense models whose regular computation maps efficiently to mobile NPUs, leaving more capable MoEs underused. MoE prefill does not fit mobile NPUs: NPU graphs are fixed...
Jun-Ming Zhang, Zhen-Zhe Zheng, Fan Wu et al.
· 0 citations