Pupillometry in the Wild: A Window into Human Vision and Cognition in Dynamic Environments

Abstract

Our pupils do more than regulate how much light enters the eyes. Pupil size also changes with cognitive effort, arousal, and attention, which makes it a promising physiological indicator of how people see and think. However, pupil signals are difficult to interpret in everyday situations. While someone is driving, watching a film, or under any complex environments, light levels, eye movements, and cognitive processes all change at the same time. Visually driven pupil responses can easily mask the smaller changes associated with cognition. This thesis addresses that problem by developing Open Dynamic Pupil Size Modelling (Open-DPSM), an open-source method for predicting pupil size changes to dynamic visual input. The method models pupil responses to luminance and contrast changes and takes gaze position into account. It also estimates how strongly visual events in different parts of the visual field contribute to pupil size changes. This makes it possible either to remove visually driven changes from raw pupil signal or to use the spatial pattern as meaningful information. The method was applied to three main research questions. First, it was tested as the basis for a less demanding form of vision testing. Participants watched films containing simulated visual-field defects, while Open-DPSM estimated the relative contribution of visual events across the visual field. The resulting spatial patterns successfully revealed the simulated defects. This proof of concept suggests that movie-viewing pupil perimetry can potentially offer a more engaging and objective complement to conventional visual-field testing, particularly for special populations such as . Second, the thesis examined whether pupil size could reliably track cognitive effort during dynamic viewing. Participants performed memory tasks with different levels of difficulty while viewing either constant visual input or driving videos. Under dynamic visual conditions, luminance changes reduced the ability of pupil size to distinguish levels of effort. Correcting for luminance substantially improved this distinction, and combining pupil size with behavioural performance and subjective workload produced the strongest measure of effort. Third, the thesis revealed that pupil responses could track covert attention (i.e., attentional shift without moving the eyes) in dynamic environments. Spatial patterns in pupil responses reflected the influence of visual salience, task goals, and leftward attentional biases. The method was then used to study spatial biases in covert attention with dynamic gaze positions. The results showed both a general preference for the screen center and a gaze-dependent bias towards the screen center, only when the gaze was on the left. Overall, this thesis shows that pupil size is not a simple readout of a single mental process. It reflects several visual and cognitive influences at once. When these influences are explicitly modelled, however, pupillometry can provide valuable information about vision, effort, and attention in realistic and continuously changing environments.