A CS-ASR synthetic benchmark and multidimensional evaluation framework tailored to enterprise domains, systematic evaluation of frontier ASR systems across 5 language pairs, and diagnostic analysis of the additional transcription errors that code-switching introduces across language pairs and models are proposed.
Shama Gupta, Hoang Nguyen, Chelsea Huang et al.· 0 citations
EVA-Bench is presented, an end-to-end evaluation framework that addresses realistic conversation simulation and comprehensive voice-specific evaluation and introduces two composite metrics: EVA-A (Accuracy) and EVA-X (Experience).
Results from benchmarking four MOS predictors and four Audio-LLM judges reveal that MOS predictors collapse onto acoustic signal quality, while Audio-LLM judges show selective, prompt-dependent detection that does not generalise across all dimensions.
Oluwanifemi Bamgbose, Simon Rosen, J. Shah et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.