Skip to content
Preprint

Cached LLM Probability Retrieval for Speech Recognition

Aug 2026 · 0 citations · 19 references
Engineering

TL;DR

Context length analysis indicates that benefits peak at a context length of 8, suggesting that cached probability retrieval is an effective and lightweight ASR adaptation method, in contrast to the heavy training required for Generative Error Correction (GER) or knowledge distillation (KD).

Abstract

Large language models (LLMs) enhance automatic speech recognition (ASR) by providing linguistic priors; however, their direct rescoring is costly because it requires evaluating every N-best hypothesis. This paper introduces"cached LLM probability retrieval,"which involves querying a local teacher LLM offline to obtain next-token probabilities for ASR-relevant context-target pairs. These probabilities are then utilized during recognition via cache lookups, backoff strategies, and optional scoring for significant misses. The method is training-free and can integrate with existing recognizers without requiring modifications to acoustic models. Evaluations across various ASR models reveal that cached retrieval outperforms 1-pass ASR in 28 of 39 settings and achieves lower non-oracle errors. Context length analysis indicates that benefits peak at a context length of 8, suggesting that cached probability retrieval is an effective and lightweight ASR adaptation method, in contrast to the heavy training required for Generative Error Correction (GER) or knowledge distillation (KD).

View source

Similar papers

#natural language process... Preprint Sep 2026

Merging the Knowledge of LLMs for Automatic Speech Recognition

This method integrates the LMs directly into the parameters of an LLM-based ASR model, requiring no additional computational cost at inference, and consistently improved the ASR performance in the target domains, without degrading inference speed or memory footprint.

Hayato Futami, Tatsuya Kawahara · 0 citations
Preprint Sep 2026

Training-Free Contextual ASR via SpeechLLM-Based Error-Aware Selective Retrieval

Recognition of domain-specific and low-frequency terms remains challenging for automatic speech recognition (ASR). Although contextual biasing can improve their recognition, directly providing a large terminology dictionary introduces many irrelevant biasing terms. Retrieval-based contextual biasing addresses this issu...

Natsuo Yamashita, Ai Nemoto, Ryosuke Koichi et al. · 0 citations
#artificial intelligence Preprint Aug 2026

Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactions

This work refine warm-initialized LLM-based ASR models by leveraging their own pre-adaptation base LLMs by leveraging their own pre-adaptation base LLMs, focusing on LoRA-adapted settings where the base LLM is preserved.

Chan-Jan Hsu, Jaeyeon Kim, Chao-Han Huck Yang et al. · 0 citations
#natural language process... Preprint Oct 2026

Phoneme-Guided Initialization for LLM-based Speech Recognition

Speech large language models (speech LLMs) perform well on automatic speech recognition (ASR) when sufficient paired speech-text data is available, but their performance degrades in low-resource settings. A cascaded pipeline that performs speech-to-phoneme (S2P) conversion followed by phoneme-to-grapheme (P2G) conversi...

Ryotaro Magoshi, Shinsuke Sakai, Tatsuya Kawahara · 0 citations
Preprint Sep 2026

DUPAR: Dual-Path Conversational Retrieval via Speech Retriever with Cross-Turn Evidence Caching

Voice assistants grounded in external knowledge typically use automatic speech recognition (ASR) to transcribe speech queries before retrieving evidence from textual knowledge bases. This cascade adds latency and propagates recognition errors, whereas direct speech retrieval is vulnerable to cross-modal misalignment. T...

Yuan-Jun Li, Yi-Wen Liu, Da-Peng Li et al. · 0 citations
Preprint Aug 2026

INSPIRE: A Benchmark for Instruction-Aware Speech Retrieval

Existing speech retrieval systems rely on fixed similarity matching and cannot adapt to diverse user intents. We introduce INSPIRE, the first benchmark for instruction-aware speech retrieval, in which natural-language instructions dynamically specify relevance criteria, including semantic content, speaker identity, spe...

Chen-An Li, Hung-Yi Lee · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.