Adapting English Quality Classifiers for Multilingual LLM Pretraining Data Selection
Recent advances in large language model (LLM) pretraining highlight the role of high-quality training data in improving performance. While model-based filtering has proven effective in selecting high-quality subsets from web-scale corpora, especially for high-resource languages, low-resource languages face challenges d...
Vinko Sabolcec, Bettina Messmer, Yassine Turki et al.
· 0 citations