Detecting AI-Generated Kazakh text with a specialised small encoder

Abstract

Fluent AI-written Kazakh threatens the academic integrity of universities where requirements to study in Kazakh are rising. To the best of our knowledge, there exists no public automatic detector of Kazakh AI-written text. In order to evaluate detectors’ quality, we compile a balanced benchmark with 24,678 documents by mixing Til-Qazyna and new texts generated by two open-weight LLMs (Qwen2.5-32B-Instruct-AWQ and Qwen2.5-7B-Instruct). We compare baselines with classical TF-IDF models, frozen multilingual encoders (mBERT, XLM-R, LaBSE), finetuned XLM-RoBERTa-base, and a BERT-Small encoder of only 29.5M parameters specially trained for Kazakh. Fine-tuning XLM-R achieves macro-F1 0.978; specialisation of BERT encoder to Kazakh brings the same score with 9$\times$ fewer parameters and outperforms all frozen-encoder and TF-IDF baselines. Both finetuned detectors achieve almost perfect recall when evaluated on new Qwen generations (0.994-1.000); however, the encoder demonstrates better robustness under challenging conditions: 0.88 against 0.84 AI detection recall for shortest samples (200–359 chars) and 0.835 vs 0.794 after attack based on back-translation KZ$\rightarrow$EN$\rightarrow$KZ. Our results demonstrate that a compact Kazakhspecialised detector is competitive and much more robust compared to all other models and therefore suitable for CPU-based university deployment.