Skip to content
Preprint

Speaker-Consistent Grounding for Confusion-Resilient LLM-Based Target Speech Extraction

Nov 2026 · 0 citations · 33 references
Engineering Computer Science

Abstract

Deterministic target-speaker extraction (TSE) tends to preserve observed linguistic content but may extract the interfering speaker. In contrast, autoregressive generative TSE improves perceptual quality, but is prone to content hallucination and speaker identity drift, which can be mitigated by model grounding with the evidence from the deterministic TSE. However, grounding such a generative TSE model with speaker-confused evidence may therefore reinforce the error. To address this gap, we proposed a repair-before-grounding framework with a training-free evidence-repair stage. Conditioning-diverse candidate selection (CDCS) selects target speaker-consistent evidence from complementary frozen TSE outputs. The selected waveform can then be used by code-space grounding (CSG) to constrain LLM-based TSE decoding. On Libri2Mix, CDCS reduces wrong-speaker extraction errors from 6.87% to 0.75% on clean mixtures and from 18.95% to 10.46% under noise. With the LLM-based TSE checkpoint unchanged, CSG further reduces noisy content- and acoustic-switch rates from 12.79% and 1.68% to 9.10% and 0.51%, respectively. Despite this stronger grounding, generative TSE retains its predicted perceptual-quality advantage over direct deterministic extraction. These results show that repairing evidence identity before grounding improves the reliability of generative TSE. Audio demos and code are available at https://hanyu-meng.github.io/LLM-TSE-Grounding-Demo/.

View source

Similar papers

Preprint Oct 2026

Phonology-Aware Speaker Extraction Frontend for Audio LLMs

Target speaker extraction (TSE) is a key frontend for enabling audio large language models (LLMs) to recognize and understand a designated speaker in overlapping speech. However, multilingual mixtures exhibit heterogeneous extraction demands, causing fixed-depth systems to either under-process challenging inputs or was...

Yan-Xiang Chen, Hao-Yuan Deng, Jin-Yu Wang et al. · 0 citations
Preprint Oct 2026

MuseFlow-TSE: Music-Aware Target Speech Extraction via Cross-Source Band-Splitting and Data-to-Data Flow Matching

MuseFlow-TSE, a discriminative deletion, generative insertion framework for music-bearing scenes, is presented, a music-aware cross-source band-split extractor that estimates target, interfering speakers, music, and noise under conservation and music-leakage constraints.

Lu Ma, Yong-Liang Wang, Zhe-Wen Su et al. · 0 citations
Preprint Nov 2026

Structured Speaker Conditioning for Diarization-Guided Target-Speaker ASR

We propose structured speaker conditioning (SSC) for diarization-guided target-speaker recognition. SSC combines local activity structure with session-level acoustic identity to construct identity-consistent target conditions that control the acoustic encoder. Target-centric relational encoding (TCRE) retains per-speak...

Ruo-Yu Wang, Shu-Tong Niu, Jun Du et al. · 0 citations
Preprint Oct 2026

Ask, Don't Guess: Targeted Clarification for Speech Recognition Errors

Speech recognition in real time in far-field conversational settings still incurs significant errors. Identifying potential error spans during streaming ASR permits a recourse mechanism where the speaker can be asked to repeat the affected content under cleaner acoustic conditions. We propose PARDON, a new framework fo...

Sankalp Parashar, P. Jyothi, Sunita Sarawagi · 0 citations
Sep 2026

Tracing and Relearning Detection Evidence in Text-to-Speech Systems

Recent audio deepfake detectors separate bona fide speech from synthetic speech, yet it remains unclear which stage of a text-to-speech system supplies the detection evidence. We address this with controlled resynthesis and detector adaptation in an F5-TTS-BigVGAN pipeline. Since vocoder reconstruction of a real mel ca...

Eunji Shin, Kyudan Jung, Jihwan Kim et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.