A Theory-Informed Roadmap Towards Multimodal Ocular-based Attention Metrics for Conversational VR

Abstract

Eye‑tracking in consumer head-mounted displays (HMDs) already provides reliable data for visual attention tracking. We now strive to extend this capability to conversational attention, which also involves auditory processing, cognitive load, mind‑wandering, and fatigue. To this end, we propose a taxonomy that maps gaze, blink dynamics, and pupillometry onto these five complementary dimensions and briefly summarize current eye‑tracking‑based evaluation methods. From the identified gaps, we outline a three‑stage research roadmap — (i) robust pupillometry calibration, (ii) multimodal signal disambiguation, and (iii) composite attention‑score fusion — aimed at achieving reliable, real‑time inference of attentional engagement in Virtual-Reality (VR)‑based dialogues.