A Semantic Multimodal Architecture for Accessible Navigation in Immersive Virtual Environments

Abstract

Accessible interaction in immersive virtual environments remains largely constrained by visually driven design paradigms, limiting meaningful engagement for users with visual impairments. Existing approaches to VR accessibility often rely on application-specific adaptations, low-level sensory cues, or vision-centered augmentations, resulting in fragmented and non-scalable solutions. This paper introduces a conceptual architecture for a semantic multimodal interface that mediates access to immersive virtual environments through non-visual representations. The architecture operates as a reusable interaction layer that translates visual information into coherent auditory and haptic feedback grounded in semantic understanding, supporting users across the full spectrum of visual impairment, from total blindness to low vision. Central to the architecture is the use of Vision Large Language Models as a semantic mediation layer, enabling dynamic interpretation of visual scenes and context-aware multimodal feedback independent of application logic. The interface is designed as a modular architecture that can be integrated into diverse VR applications, decoupling accessibility mechanisms from scenario-specific implementations. The architecture is instantiated in heterogeneous immersive environments to illustrate its generality and architectural feasibility. By treating accessibility as a foundational interface concern rather than an add-on feature, this work contributes a design-oriented perspective for scalable, inclusive interaction in immersive virtual environments.