A Telugu-Adapted Low-Resource Speech-to-Speech Translation System using IndicWhisper, IndicTrans2, and Language-Aware Processing

Abstract

Speech-to-Speech Translation (S2ST) is a vital technology for facilitating communication across languages, especially in multilingual and resource-limited environments. However, it is still hard to make reliable S2ST systems for low-resource languages like Telugu because there aren’t many parallel speech resources, the morphology is complex, the speakers are different, and the real world is noisy. This paper introduces a Telugu-adapted cascaded S2ST framework that incorporates IndicWhisper for automatic speech recognition, IndicTrans2 for machine translation, and text-to-speech synthesis for generating spoken output. The primary contribution of this work is the integration of Telugu-aware preprocessing and post-processing techniques, encompassing audio normalization, noise reduction, sentence-level segmentation, Telugu script normalization, numerical normalization, and named-entity preservation. These language-aware parts are meant to make transcription more reliable and translation more accurate when there aren’t many low resources.We test the proposed system with benchmark and custom datasets, such as TeluguST-46, the BhashaAnuvaad subset, and recordings from the real world that are noisy. The full system gets a BLEU score of 0.62, a WER of 0.21, and a chrF score of 0.89, which is better than a plain cascade baseline. Ablation analysis further substantiates that both preprocessing and post-processing significantly influence the final performance. The research illustrates that language-aware adaptation enhances cascaded S2ST systems for Telugu and other low-resource languages.