Real-Time Correction of Named Entities in Speech Transcriptions: A Lightweight Approach

Abstract

End-to-end on-device Automatic Speech Recognition (ASR) systems have demonstrated remarkable accuracy and efficiency in recent years. However, challenges persist in correctly transcribing infrequent named entities (e.g., geographical locations, business entities, person names, etc.) and handling diverse user accents, which remain underrepresented in training datasets. While information retrieval augmentation or Retrieval Augmentation Generation (RAG) for correction of named entities has shown promise in knowledge-grounded NLP tasks when paired with large language models (LLMs), its integration into real-time on-device systems is non-trivial due to computational constraints. We introduce a novel lightweight method combining phonetic-aware retrieval, vector-based semantic search and generative correction. The system leverages a lightweight phonetic index for rapid candidate entity retrieval and a dense vectorembedding module to refine predictions as well as model the ASR error output distribution in generative space. Additionally, we introduce a novel approach to model ASR errors in natural language. Experiments on test sets emphasizing place names, monuments, airports, and landscapes yielded an increase in correct Named Entity (NE) recognition accuracy by 9.6% compared to baseline. These gains underscore the efficacy of hybrid retrieval-generation paradigms in resource-constrained environments.