Step Into the Unknown: Online Visual-Physical Bayesian Inference for Open-World Terrain Perception

Abstract

Reliable terrain perception is fundamental for the embodied navigation of legged robots in open-world environments. However, dynamically discovering unbounded terrain categories from streaming visual and proprioceptive data remains challenging. Existing methods suffer from catastrophic forgetting, rely on computationally expensive experience replay, or use heuristic rules to resolve cross-modal conflicts. To address these limitations, we propose an online multimodal Bayesian inference framework for open-world latent terrain perception. We employ a Dirichlet Process (DP) prior that adaptively expands the terrain space for unknown categories. To retain historical knowledge without experience replay overhead, we formulate a Dynamic Belief State based on conjugate priors, which compresses visual-physical observations into fixed-size sufficient statistics to enable efficient recursive updates. Furthermore, we derive an analytical posterior log-odds hypothesis test to evaluate whether an observation is known or novel by weighting multimodal evidence. Experiments on a quadruped robot in both simulation and real-world environments demonstrate its effectiveness, yielding a severe error rate of 0.013, an average retention rate of 97.1%, and a Bayesian backend inference time of approximately 1 ms.