Vincent W. S. Wong

CORTEX: Learning to Share and Specialize in Dense Language Models

Large language models are trained on heterogeneous data mixtures, where different knowledge domains require both shared knowledge and specialization. Existing modular approaches typically impose explicit components or discover modules through interpretability analysis after training. In this work, we propose CORTEX, a...