Ask the World Before Acting: Environment Probing for Calibrating Agent World Models

Abstract

Long-horizon language agents enable autonomous decision-making in complex environments by planning and executing sequences of interdependent actions. To act reliably, these agents must maintain an accurate world model that tracks the environment's evolving state and supports subsequent decisions. However, as interactions accumulate, stored beliefs can become outdated or incorrect, causing agents to plan and act on assumptions that no longer reflect the environment. Such errors are difficult to resolve through additional reasoning or self-reflection alone, because the information needed for correction must be obtained from the environment. Actively probing the environment therefore offers a direct way to verify and correct these beliefs. To this end, we introduce ENVPROBE, a framework that enables agents to actively explore their environments and correct inaccurate beliefs during long-horizon execution. Specifically, ENVPROBE prioritizes belief fields for verification based on task relevance and potential errors. It then selectively probes the environment to retrieve the actual value of a chosen field and incorporates the observation into the agent's world model. With the updated beliefs, the agent revises its plan and continues execution based on the newly acquired environmental information. Extensive experiments across three controlled environments demonstrate that ENVPROBE substantially improves terminal world-state accuracy compared with agents operating without environment probing, leading to more reliable long-horizon decision-making. These results demonstrate the importance of actively verifying and correcting beliefs through environmental interactions for maintaining accurate world models in long-horizon language agents.