Silver-Corpus Transfer for Kazakh Morphosyntactic Analysis: A Cross-Validated Benchmark of Compact and Pretrained Models

Abstract

The limited amount of manually annotated data remains a major constraint for supervised Kazakh morphosyntactic analysis: UD_Kazakh-KTB contains 1078 sentences and 10,536 syntactic words and is too small for a conventional train–development–test split. We therefore benchmark five systems—a compact 2.5 M-parameter hybrid tagger (CAGF-CBT+CRF), a classical Stanza pipeline trained from scratch (5.5 M), and fine-tuned KazRoBERTa (83.7 M) and XLM-R (278.6 M) encoders—under identical cross-validation folds, prediction targets, and official CoNLL-2018 scoring, with pooled out-of-fold evaluation over all 1078 sentences. Pretraining the compact model on a 210 K-token automatically annotated corpus improves Lemmas F1 from 74.93 to 82.81 and UPOS from 82.99 to 87.83; the effect is stable across three training seeds and the lemma gain replicates on all three neural architectures tested (+7.9, +3.5, and +3.1 for CAGF-CBT+CRF, KazRoBERTa, and XLM-R, respectively). Under the stricter protocols added in this revision—source-grouped five-fold cross-validation and leave-one-source-out evaluation on fully held-out documents—the silver effect on the compact model does not shrink but grows, reaching +13.30 Lemmas and +17.07 AllTags on a held-out Wikipedia document. Fine-tuning large pretrained encoders on this treebank is intermittently unstable: three of 18 KazRoBERTa and three of 10 XLM-R runs collapse on morphological features under early stopping, whereas none of 60 compact-model runs does. Under the stratified protocol, the from-scratch Stanza pipeline is the strongest system on Lemmas, UFeats, and AllTags (92.07/85.90/82.85), and XLM-R the strongest on UPOS (93.39); we report these results as obtained and frame the contribution accordingly—a reproducible cross-validated benchmark for UD_Kazakh-KTB, a quantification of silver-corpus transfer and its robustness to domain shift, and a documented instability of encoder fine-tuning at this corpus scale—rather than state-of-the-art accuracy. A scaling experiment with 100 K, 210 K, 500 K, and 1 M silver tokens shows continued improvement with diminishing returns beyond 500 K tokens. The source code, fold definitions, predictions, and analysis materials are publicly available.