Skip to content

HINTT Submission to the 2nd MLC-SLM Challenge: Comparing Cascaded and Unified Approaches to Diarization and ASR

Oct 2026 · 2nd Workshop on Multilingual Conversational Speech Language Model (MLC-SLM 2026) · 0 citations · 29 references
Engineering Computer Science

TL;DR

Experimental results demonstrate that the cascaded system remains more reliable under the MLC-SLM Task 1 conditions, while unified speech LLMs offer a promising direction for future speaker-attributed ASR.

Abstract

This paper presents the HINTT system submitted to the 2nd Challenge and Workshop on Multilingual Conversational Speech Language Model (MLC-SLM). We address multilingual speaker-attributed ASR, where systems must determine who spoke when and what was spoken. We investigate two modeling strategies for this problem: a cascaded pipeline that combines speaker diarization with speech-LLM-based ASR, and a unified speech LLM that directly generates speaker labels, timestamps, and transcriptions. Our final submission is based on the cascaded pipeline, consisting of a fine-tuned DiariZen diarization model, a fine-tuned Qwen3-ASR model, and LLM-based generative error correction. For comparison, we also fine-tune VibeVoice-ASR as a unified model using the same official training data. All task-specific fine-tuning and model selection are performed using only the official MLC-SLM data, without external data or pseudo-labels. Experimental results demonstrate that the cascaded system remains more reliable under the MLC-SLM Task 1 conditions, while unified speech LLMs offer a promising direction for future speaker-attributed ASR.

Read PDF

Similar papers

Open access Oct 2026

The SMIIP System for Multi-Speaker Automatic Speech Recognition in the 2nd MLC-SLM Challenge

We present the SMIIP system for Task 1 of the 2nd Multi-lingual Conversational Speech Language Model (MLC-SLM) Challenge, which requires jointly performing speaker diarization and speech recognition on raw conversational recordings without oracle speaker labels or time boundaries. Our system couples a w2v-S2SND diariza...

Li Li, Zeng-Yao Li, Jin-Chen Wang et al. · 0 citations
#natural language process... Open access Sep 2026

SEAR: Segment-Evidence-Aware Routing for Weak-to-Strong Multilingual Speech MCQ

This paper describes the system for Task~2 of the second Multilingual Conversational Speech Language Model (MLC-SLM) Challenge, which adapt Qwen3-Omni-30B-A3B-Instruct with a segment-evidence-aware data and post-training pipeline and obtains 90.92% accuracy on the final official evaluation set.

H. Le, L. Nguyen, Minh Tri Dao · 1 citation
#natural language process... Open access Sep 2026

The Bairong System for MLC-SLM 2026: Dynamic Question-Aware Evidence Routing for Multilingual Conversational Speech Understanding

Long multilingual conversational spoken question answering requires systems to balance long-range transcript semantics with sparse acoustic and speaker-sensitive cues. We present the Bairong system for the MLC-SLM 2026 Challenge, where a diarization-ASR front-end produces speaker-attributed transcripts and a dynamic ev...

Shang-Kun Huang, Jun-Chao Hu, Hua Shen et al. · 1 citation
#natural language process... Open access Sep 2026

The Eloquence submission for Task 2 of the Interspeech 2026 MLC-SLM challenge

This paper details the Eloquence team's approach to Task 2 of the 2nd MLC-SLM challenge at Interspeech 2026, which involves multilingual Multiple-Choice Question Answering (MCQA) across 21 languages. Three approaches are explored. First, we fine-tune Voxtral-Mini-3B via LoRA with cross-lingual data augmentation, ASR tr...

Jordi Luque, Lorenzo Concina, M. Matassoni et al. · 1 citation
#natural language process... Preprint Oct 2026

Phoneme-Guided Initialization for LLM-based Speech Recognition

Speech large language models (speech LLMs) perform well on automatic speech recognition (ASR) when sufficient paired speech-text data is available, but their performance degrades in low-resource settings. A cascaded pipeline that performs speech-to-phoneme (S2P) conversion followed by phoneme-to-grapheme (P2G) conversi...

Ryotaro Magoshi, Shinsuke Sakai, Tatsuya Kawahara · 0 citations
Conference Aug 2026

Multilingual Speech-to-Text and Automated Meeting Summarization Using NLP and Machine Learning

Manually preparing minutes of a meeting is time-consuming and error-prone, especially in the multilingual case, such as that in India, where the same meeting can use Tamil, Telugu, Hindi, Kannada and English languages. In this paper we propose a pipeline to automate the preparation of meeting minutes by utilizing multi...

P. G. Krishna, T. Thilagam · 0 citations

Related blog posts

Microsoft Research Blog Oct 22, 2026

Flint-interactive: A language for creating interactive data visualizations

Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.