Scaling Laws for Multimodal Data Mixtures
This work derives data mixing scaling laws for three modalities: text, vision, and audio from Mixture-of-Experts (MoE) architectures, and provides a principled foundation for understanding the multimodal data mixtures needed to train frontier MLLMs.
Aditi Khandelwal, A. Tarun, Yi-Xuan Xu et al.
· 0 citations