CTC score-based transcription quality annotation for stable text-to-speech synthesis training on noisy transcriptions

Abstract

Automatic speech recognition (ASR) has been increasingly adopted for generating transcriptions to train text-to-speech (TTS) models, substantially reducing manual annotation costs. However, transcription errors introduced by ASR systems inevitably degrade TTS performance, and existing approaches lack explicit quality awareness, leaving it unclear whether TTS models can learn to distinguish and respond to varying transcription quality. This study investigates whether explicit quality supervision can enable TTS models to develop quality-aware representations and achieve controllable stable synthesis. This study proposes transcription quality labels (TQL), automatically derived from connectionist temporal classification scores. During training, TQL provides explicit quality supervision, enabling the model to associate transcription quality with acoustic characteristics. During inference, setting TQL to high values guides the model toward stable synthesis that faithfully follows the input text. Under controlled conditions isolating the effect of transcription quality, our best TQL-based model achieves 8.4% word error rate compared to 15.9% for the baseline, a 47% relative reduction. While synthesis accuracy gains are significant, naturalness improvements are modest (mean opinion score of 3.66 vs. 3.55), indicating that TQL primarily enhances synthesis stability rather than perceptual quality. Further analysis confirms that synthesis quality varies systematically with TQL values at inference, demonstrating learned quality-conditioned behavior.