A. H. Ahmed

Cross-Lingual Alignment for Decoder-Only Models using MoE Routers

Cross-lingual contrastive learning has been a core component of multilingual encoder training, but the ability to explicitly align representations is not possible in decoder-only LLMs because of varying multilingual tokenization. However, a growing amount of research suggests that even in LLMs, higher cross-lingual rep...