Skip to content

Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoring

Sep 2026 · arXiv.org · Vol abs/2609.30924 · 0 citations · 28 references
Computer Science Engineering

TL;DR

A training-free ST2P pipeline that integrates both lexical and acoustic information at inference time is proposed that outperforms all baselines, including a trained ST2P model and commercial multimodal LLMs.

Abstract

Accurate and efficient pronunciation transcription is essential for preparing text-to-speech training data at scale. Existing approaches have different limitations: grapheme-to-pronunciation (G2P) and speech-to-pronunciation (S2P) methods each capture only partial information, using only text or only speech, while speech-and-text-to-pronunciation (ST2P) methods use both but require costly pronunciation-annotated data. To address this problem, we propose a training-free ST2P pipeline that integrates both lexical and acoustic information at inference time. Lexical resources and G2P tools generate text-constrained candidates, and a left-to-right greedy search selects the best one using whole-sequence negative log-likelihoods from frozen pretrained S2P models. On three Japanese corpora, our method reduces Character Error Rate (CER) from 0.60--1.40\% (text-only baseline) to 0.04--0.17\% with reference transcripts, and 0.64--1.58\% with ASR transcripts. It outperforms all baselines, including a trained ST2P model and commercial multimodal LLMs. Our greedy search method is 3--3.5$\times$ faster than beam search at similar CER, and the cascade is 2$\times$ faster than direct decoding ensuring the efficiency and accuracy. In Spanish, French, and preliminary English, it also surpasses four open multimodal LLMs and the best traditional methods.

View source

Similar papers

#natural language process... Preprint Oct 2026

Phoneme-Guided Initialization for LLM-based Speech Recognition

Speech large language models (speech LLMs) perform well on automatic speech recognition (ASR) when sufficient paired speech-text data is available, but their performance degrades in low-resource settings. A cascaded pipeline that performs speech-to-phoneme (S2P) conversion followed by phoneme-to-grapheme (P2G) conversi...

Ryotaro Magoshi, Shinsuke Sakai, Tatsuya Kawahara · 0 citations
Preprint Oct 2026

Pronunciation-Supervised Text-to-Speech for Irish (Gaeilge)

Speech technology for Irish must be built from very little transcribed speech, yet the language's pronunciation is thoroughly documented in dictionaries and dialect descriptions. We ask whether that documentation can supervise the adaptation of a large pretrained text-to-speech model to a language it was never trained...

William Lane · 0 citations
Preprint Sep 2026

PersianVox: A Prosody-Aware Approach for Speech Dataset Generation from In-the-Wild Data

This paper introduces PersianVox, a fully automated pipeline designed to generate high-quality speech corpora from unlabeled web data, using a novel prosody-aware segmentation strategy that utilizes acoustic turn-detection to preserve linguistic completeness and optimize utterance duration for long-context modeling.

Saeedreza Zouashkiani, Soheil Khalesi, Saman Soleimani Roudi et al. · 0 citations
Preprint Oct 2026

Pronouncing Hard Words: Phoneme Adaptation of a Pretrained Conversational Text to Speech Model

Voice agents built as cascaded automatic speech recognition (ASR)->large language model (LLM)->text-to-speech (TTS) pipelines increasingly rely on conversational TTS: streaming, context conditioned models that adapt delivery to dialogue history. Such models are natural and expressive but offer little explicit control o...

Yashish M. Siriwardena, Dinesh K. Gnanasekaran, Tanmay Laud et al. · 0 citations

Dictionary-Constrained Grapheme-to-Phoneme for Unsegmented Languages from LLM-Annotated Data

A context-aware, segmentation-agnostic neural G2P framework that models the joint segmentation-and-reading hypothesis space, scoring paths of a discriminative conditional random field (CRF) over a dictionary-derived word lattice over a dictionary-derived word lattice is presented.

Rui Hu, Zhen Zhan, Xiao-Long Lin · 1 citation
Open access Oct 2026

TABR: Tone-Auxiliary Base-Syllable Rescoring for Low-Resource Tujia Speech Recognition

Tujia is a low-resource endangered language, and its digital documentation and preservation are of considerable importance. The scarcity of transcribed speech data leads to low recognition accuracy in Tujia automatic speech recognition (ASR). We propose Tone-Auxiliary Base-syllable Rescoring (TABR), a two-stage framewo...

Jian-Feng Li, Ding Jiang, Zi-Yang Hu et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.