This work introduces LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and speech-based interfaces, and quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in speech-driven coding environments.
Abstract
AI-assisted programming environments have accelerated software development, giving rise to new paradigms like vibe coding. However, their benefits remain largely inaccessible to visually impaired programmers, as existing screen readers and assistive tools offer limited support for these emerging workflows. We introduce LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and speech-based interfaces. LipCoder offers features comprising speech feedback and earcon cues for comprehension and validation, as well as natural language input for navigation and modification. In an exploratory evaluation, 5 visually impaired programmers performed a series of coding tasks comparing LipCoder with a baseline of VSCode, Copilot, and VoiceOver. Quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in speech-driven coding environments.
As multimodal interaction becomes central to digital learning, technical interfaces, such as eBooks and conversational agents, present unique accessibility challenges for users relying on the auditory modality. While modern Text-to-Speech (TTS) systems excel at rendering natural language, they often fail to convey the...
Marco Cardia, Letizia Angileri, M. Buzzi et al.· 0 citations
Audiodescription is essential for making audiovisual content accessible to visually impaired individuals, converting visual information into audio descriptions. Typically done by human narrators, audiodescription can be complemented by virtual technologies with synthesized voice. This study evaluated the accessibility...
A. D. de Oliveira· Revista Brasileira Multidisc...· 0 citations
Speech editing is gradually making its way into everyday applications. This paper presents Speech Editor, a practical software system built upon SpeechTripleNet for disentangled speech representation learning. The system enables real-time end-to-end speech editing, including local modification of pitch and energy as we...
Jiayi Wang, Qifeng Han, Danyi Huang et al.· International Conference on...· 0 citations
An interpretable audio-visual synchronization framework that explicitly models speech-lip temporal alignment through feature encoding, cross-attention fusion, lag estimation, stability quantification, and visualization is proposed.
Bo-Wen Yu, Ming-Yue Huang, Yi-Shen Liu et al.· 0 citations
Software debugging is a crucial but lengthy process in software development that may require developers to decode complicated error messages and stack traces. The growing complexity of Python applications today underscores the need for more intuitive and effective debugging methods. This paper focuses on enhancing debu...
P. L., Rakshitha S, M. P et al.· Engineering & Technology· 0 citations
Environmental noise obscures the content cues needed for whisper-to-normal conversion. We propose WhisperVC-AV, which restores acoustic content features while leaving the original WhisperVC conversion module unchanged. Its context-guided restoration module combines temporal acoustic context with synchronized lip featur...
Zi-Yue Yin, Dong Liu, Ming Li· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 20, 2026
Senior Research Scientist Jim Glass discusses how his MIT-IBM Computing Research Lab collaboration connects perception and language, and addresses the shortcoming of current models.