From Collaboration to Capability: Internalizing Routed LLM Experts into Compact Reasoners
This work studies whether learning from both the controller's decisions and the experts'reasoning and code improves its generation after expert removal, and introduces Rivet for collaboration internalization, supporting the effectiveness of training on the content and structure of verified collaborations.