Deterministic target-speaker extraction (TSE) tends to preserve observed linguistic content but may extract the interfering speaker. In contrast, autoregressive generative TSE improves perceptual quality, but is prone to content hallucination and speaker identity drift, which can be mitigated by model grounding with the evidence from the deterministic TSE. However, grounding such a generative TSE model with speaker-confused evidence may therefore reinforce the error. To address this gap, we proposed a repair-before-grounding framework with a training-free evidence-repair stage. Conditioning-diverse candidate selection (CDCS) selects target speaker-consistent evidence from complementary frozen TSE outputs. The selected waveform can then be used by code-space grounding (CSG) to constrain LLM-based TSE decoding. On Libri2Mix, CDCS reduces wrong-speaker extraction errors from 6.87% to 0.75% on clean mixtures and from 18.95% to 10.46% under noise. With the LLM-based TSE checkpoint unchanged, CSG further reduces noisy content- and acoustic-switch rates from 12.79% and 1.68% to 9.10% and 0.51%, respectively. Despite this stronger grounding, generative TSE retains its predicted perceptual-quality advantage over direct deterministic extraction. These results show that repairing evidence identity before grounding improves the reliability of generative TSE. Audio demos and code are available at https://hanyu-meng.github.io/LLM-TSE-Grounding-Demo/.
Target speaker extraction (TSE) is a key frontend for enabling audio large language models (LLMs) to recognize and understand a designated speaker in overlapping speech. However, multilingual mixtures exhibit heterogeneous extraction demands, causing fixed-depth systems to either under-process challenging inputs or was...
Yan-Xiang Chen, Hao-Yuan Deng, Jin-Yu Wang et al.· 0 citations
Oracle analysis shows that substantially larger WER reductions are possible through utterance-level scale selection and identifies how beneficial adjustments vary with domain shifts.
Yi-Wen Guan, Jacob Whitehill· arXiv.org· 0 citations
MuseFlow-TSE, a discriminative deletion, generative insertion framework for music-bearing scenes, is presented, a music-aware cross-source band-split extractor that estimates target, interfering speakers, music, and noise under conservation and music-leakage constraints.
Lu Ma, Yong-Liang Wang, Zhe-Wen Su et al.· 0 citations
We propose structured speaker conditioning (SSC) for diarization-guided target-speaker recognition. SSC combines local activity structure with session-level acoustic identity to construct identity-consistent target conditions that control the acoustic encoder. Target-centric relational encoding (TCRE) retains per-speak...
Ruo-Yu Wang, Shu-Tong Niu, Jun Du et al.· 0 citations
Speech recognition in real time in far-field conversational settings still incurs significant errors. Identifying potential error spans during streaming ASR permits a recourse mechanism where the speaker can be asked to repeat the affected content under cleaner acoustic conditions. We propose PARDON, a new framework fo...
Sankalp Parashar, P. Jyothi, Sunita Sarawagi· 0 citations
Recent audio deepfake detectors separate bona fide speech from synthetic speech, yet it remains unclear which stage of a text-to-speech system supplies the detection evidence. We address this with controlled resynthesis and detector adaptation in an F5-TTS-BigVGAN pipeline. Since vocoder reconstruction of a real mel ca...
Eunji Shin, Kyudan Jung, Jihwan Kim et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.