Low-Resource Kichwa Mispronunciation Detection Using CNN and Wav2Vec2 Transformer-Based Deep Learning Models

Abstract

Low-resource indigenous languages remain underrepresented in speech technologies, particularly in pronunciation assessment and computer-assisted language learning. This paper addresses automatic mispronunciation detection for Kichwa, an indigenous language spoken in Ecuador and other regions of Latin America. To support this task, a dedicated Kichwa speech corpus comprising 6,180 word-level recordings from 30 volunteer participants was constructed. The corpus was processed through manual segmentation, transcription assignment, expert-based annotation by native Kichwa-speaking evaluators, majority-vote labeling, metadata construction, and privacy-preserving organization for model development. Several deep learning models were evaluated for the proposed detection task, including fully connected, convolutional, transfer-learning, and Wav2Vec2.0 transformer-based architectures. The models were evaluated using speaker-disjoint partitions across five independent runs initialized with different random seeds, considering training conditions with and without targeted augmentation of the minority mispronounced class. Validation-based decision-threshold tuning was additionally examined to characterize different operating trade-offs under class imbalance. Among the evaluated configurations, Wav2Vec2.0 LayerFusion without data augmentation and with validation-based threshold tuning provided the strongest overall balance across the considered performance measures. The results further showed that data augmentation and threshold adjustment produced distinct precision–recall–specificity trade-offs across model configurations. Overall, the findings support the feasibility of automatic Kichwa mispronunciation detection and highlight the potential of speech-oriented transformer representations for pronunciation-assistance technologies within the linguistic and experimental setting considered in this study.