Aug 2026
DR-EFT: Exploring and reloading domain-representative experts for the memory-constrained fine-tuning of MoE large models.
An algorithm framework named DR-EFT (Domain-Representative Experts for Fine-Tuning), which explores and loads the domain-representative experts for subsequent retraining and reincorporation and demonstrates robustness through validations on popular MoE LLMs, including Qwen, DeepSeek, and Ernie.
Zhaomeng Cheng, Zhong Ji, Yan Zhang et al.
· Neural Networks · 0 citations