OLMo2 with Hybrid MLM-CLM Training for the BabyLM 2026 Multilingual Challenge
We present our submission to the BabyLM 2026 Multilingual Shared Task. Our system uses OLMo2, a modern decoder-only architecture with grouped-query attention, RoPE, and SwiGLU, trained from scratch on the official BabyLM trilingual corpus (English, Dutch, Chinese). We employ a hybrid training ob-jective following (Char...