Skip to content
Review

Working memory in a multisensory world: flexible representation of relevant events across multiple modalities

Abstract

Working memory allows us to temporarily maintain information in support of goal-directed behavior. Most research on working memory has focused on vision, yet everyday events often provide information through several senses. Drawing on functional magnetic resonance imaging (fMRI), electroencephalography (EEG), behavioral experiments, and a theoretical review, this thesis examines how sensory information is encoded, maintained, and used in a multisensory world. The first part comprises two studies examining how auditory information is encoded and maintained. The first study (Chapter 2) examined how the brain represents interaural level differences (ILD), a cue to sound location. Early auditory cortex responded preferentially to sound coming from the contralateral side of space, whereas auditory association regions showed a wider range of preferences, with sounds coming from the center yielding the sharpest neural tuning. The second study (Chapter 3) investigated neural markers of the amount of auditory information held in working memory. Auditory working memory load could be decoded from patterns of alpha-band activity that changed over the retention interval. In contrast with visual working memory, lateralized neural responses (e.g., the contralateral delay activity; CDA) did not reliably track auditory load. These findings suggest that the auditory system follows idiosyncratic principles for spatial encoding and working memory maintenance, distinct from the visual system. The second part turns to working memory in a multisensory context. A theoretical review (Chapter 4) proposes a conceptual framework that distinguishes between different possible representational formats for storing multimodal stimuli in working memory, ranging from strictly modality-specific to fully supramodal codes. This chapter also presents experimental and analytical strategies that may help differentiate between these candidate mechanisms. Building on this, a behavioral study (Chapter 5) examined the fundamental organizational principle of working memory storage: are memories segregated by sensory modality (auditory vs visual) or by information type (spatial vs temporal)? The results showed that temporal information benefited from auditory presentation, whereas spatial information benefited from visual presentation. Interference during retention impaired performance, but was not selective to either the sensory modality or the type of information being remembered, suggesting more abstract storage. The final behavioral study (Chapter 6) examined how observers use multisensory information to localize moving objects, and how this depends on expected sensory interference. The results showed that observers can flexibly prioritize one sense over the other in anticipation of modality-specific interference. Together, this thesis highlights the importance of studying the distinctive properties of individual senses, and supports a view of working memory as a multisensory system that is capable of flexibly transforming, prioritizing, and switching between representational formats to meet the demands of a dynamic environment.

View source

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.