Automatic transformation of Kazakh text into sign language glosses using multilingual transformer-based models

Abstract

This study investigates the automatic transformation of Kazakh text into sign language glosses (Text-to-Gloss) using multilingual transformer-based models with emphasis on preserving morphological structure in a low-resource agglutinative language framework. Given the scarcity of high-quality intermediate representations for Kazakh Sign Language, a methodology for corpus formation was developed, resulting in a specialized dataset of 11 190 unique text−gloss pairs sourced from educational materials. To assess the effectiveness of transfer learning under low-resource constraints, we fine-tuned mT5-small and mT5-base models on this dataset. Experimental results indicate that the mT5-base architecture consistently outperforms the smaller variant across all metrics, achieving a 2.61-point improvement in BLEU (87.51 → 90.12) and a 2.47-point increase in ROUGE-1 F1 (89.61% → 92.08%), alongside gains in exact match (74.96% → 80.68%) and chrF (91.6 → 94.1). Statistical significance testing (p < 0.001) confirms the stability of this improvement. Error analysis reveals that larger model capacity particularly improves morphological handling and token coverage, which is critical for agglutinative languages, while maintaining semantic accuracy. These findings demonstrate that pretrained multilingual models effectively address TTG transformation for low-resource agglutinative languages through transfer learning, with practical applications for Kazakh Sign Language assistive systems and education.