Model Collapse in Bangla and English: A Pre-Registered Study of Iterative Self-Training with Gemma-3-270m

Abstract

Pre-registration of a computational experiment on model collapse in a low-resource language setting, frozen before any model is trained. We fine-tune the open-weight model google/gemma-3-270m (bfloat16) in Bangla and in English over six generations of self-training, where each generation is trained on text generated by the previous one. Three regimes are compared per language: REPLACE (synthetic text only), ACCUMULATE (real seed text plus all synthetic generations) and ACCUMULATE-FILTER (the same with a pre-specified quality filter). Corpora are built from public Bangla and English Wikipedia and openly licensed web text, matched across languages in words per document and in token budget (10 million generated tokens per generation). Outcomes are held-out log-perplexity and bits per byte, lexical diversity of fixed-prefix samples, and recall on a paired factual-probe set, all computed automatically. Hypotheses: H0, both languages degrade under REPLACE (anchor); H1, Bangla degrades faster than English under REPLACE on the pre-specified slopes; H2, the relative benefit of accumulating real data is smaller for Bangla. The decision rule, the robustness phases in a fixed priority order with a stopping rule under the approved compute cap (one rented RTX 4090, US$30 / 70 GPU-hours), the scoped-out conditions and the analysis plan are given in the attached 01_PREREGISTRATION.md; exact chain and corpus settings are in chains.yaml and corpora.yaml; prereg_freeze_2026-10-13.json lists the SHA-256 of each attached file. No human participants are involved; the study uses public text and model outputs only. Authors: Md Tahidul Islam (ORCID 0009-0004-6798-892X, corresponding) and Maksuda Akter (ORCID 0009-0004-5808-518X), Department of Computer Science and Engineering, CCN University of Science & Technology, Cumilla, Bangladesh. Registered before any training run.