Skip to content
Book Open access

Belief-Based World Models for Partially Observable Reinforcement Learning

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 56 references

Abstract

Embodied agents operating in immersive environments often rely on incomplete egocentric observations, where the current view is insufficient to infer the underlying scene dynamics and make reliable decisions. World models learn compact latent representations to predict future dynamics and rewards, enabling sample efficient policy optimization. However, in partially observable embodied environments, the same observation-action pair may correspond to different hidden states, leading to ambiguous transition and reward dynamics from the agent's perspective. We present BRAVE (Belief Representation for pArtially obserVable Environments), which constructs latent belief representations by integrating the current observation with informative visual cues retrieved from historical memory. Specifically, BRAVE first selects representative key frames from past experience to preserve both relevance and temporal diversity, and then fuses them to construct a more informative belief state for dynamics prediction and downstream decision making. By explicitly modeling belief construction as a retrieval-and-fusion process over egocentric visual memory, BRAVE improves long-horizon prediction and policy learning in partially observable settings. We evaluate BRAVE on diverse embodied decision-making benchmarks, including immersive navigation, open-ended interaction, and multi-agent coordination tasks. Experimental results demonstrate that BRAVE consistently outperforms strong baselines in both sample efficiency and final performance.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.