Across systems, WER has little rank agreement with CTEM (Spearman $\rho=-0.28$) or TSR (Spearman $\rho=-0.28$), and even the strongest system leaves nearly one-third of recordings with an unrecovered critical value.
Abstract
Automatic speech recognition is usually evaluated with word error rate (WER), although voice workflows often require exact written values. VoiceCodeBench measures whether transcripts preserve identifiers, paths, commands, and other structured tokens needed by downstream software. It contains 300 human-recorded English workplace segments (5.59 hours, 85 speakers) and 1,482 audited entities across 26 types and eight domains. Under a raw-audio-only protocol, we evaluate 19 batch and streaming systems using WER, Canonical Token/Entity Match (CTEM), and strict segment-level Task Success Rate (TSR). Across systems, WER has little rank agreement with CTEM (Spearman $\rho=-0.28$) or TSR ($\rho=-0.22$). The best CTEM and TSR are 91.8% and 68.7%. Even the strongest system therefore leaves nearly one-third of recordings with an unrecovered critical value. Symbol-, separator-, and boundary-sensitive entities account for most errors.
StreamAlign performs online speech-text alignment by combining character-level RNN-Transducer alignment with word-level ASR guidance, mitigating ASR-LLM vocabulary mismatch while preserving recognition accuracy, and achieves the lowest WER and highest UTMOS among evaluated tokenizers.
Kang-wook Kim, Jinyoung Park, Jinsoo Kim et al.· 0 citations
The resulting 82M-parameter model, Wayu-Paxa-TTS-Edge, enables on-device Thai TTS without reference audio and achieves the lowest pause-placement error and intra-word pause rates among the three systems.
Kunat Pipatanakul, Potsawee Manakul, Warit Sirichotedumrong et al.· 1 citation
Forced alignment aligns speech audio with a text transcript to generate word and phone timestamps. Published comparisons normalize transcripts, split the data and match boundaries differently, so their numbers cannot be read together. We present FA-Bench, an open framework that fixes those choices once and releases the...
Wei Chu, Yuan-Zhe Dong, Ke Tan et al.· 0 citations
Automatic speech recognition is typically trained assuming that the reference transcript is the only valid labeling of an utterance, yet even nominally verbatim transcripts contain localized differences in pronunciation, spelling, or lexical realization that the acoustics do not uniquely determine. Omni-temporal Classi...
Saurabh Kumar, Diptiman Mohanta, P. Ghosh· 0 citations
This study presents a reproducible synthetic benchmark for evaluating error propagation in a speech-to-text pipeline that links acoustic condition, transcription-error type, deterministic normalization, phrase-aware mapping, sentence correctness, and semantic preservation on the same records.
U. A. D. Priyanka D. Vaidya· Natural Resources for Human...· 0 citations
FuseAlign, a transformer-based aligner trained on large-scale pseudo-labeled speech with online label correction, is introduced, a transformer-based aligner trained on large-scale pseudo-labeled speech with online label correction that substantially outperforms all baselines and remains robust under real ASR transcript...
Mithilesh Vaidya, Stephen W. Bailey, Sumukh Badam et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 15, 2026
A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.
Microsoft Research Blog· microsoft.comOct 15, 2026
Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduSep 24, 2026
