One Model, Multiple Compute Budgets: Multi-Ratio ConceptMoE for Flexible Compute Across Turns
Large language models typically support different inference budgets through separate model sizes or fixed compute configurations. In multi-turn interactions, however, compute demand can vary substantially across turns. Using a high-compute model for every turn wastes computation, while routing among independent models...
Yi-Yang Fu, Zi-Hao Huang, Dong-Chen Han et al.
· 0 citations