Rationality Randomization With Maximum Entropy for Robust Dynamic Obstacle Avoidance on Legged Robots

Abstract

Reinforcement learning (RL) for legged robots has recently gained significant attention due to its ability to enable traversal of complex terrains and navigation in cluttered environments. However, most existing approaches focus on static or predictably moving obstacles and rely on handcrafted curricula or manual task design, which may limit robustness to diverse obstacle behaviors. In this letter, we propose a method to automatically generate rich and dynamic obstacle-avoidance scenarios for learning robust navigation policies. Inspired by adversarial RL, we model obstacles as agents that induce challenging interactions for a protagonist policy. To improve robustness, we vary obstacle behavior by modulating their rationality, defined as the level of stochasticity in their actions. This is controlled via a temperature parameter that interpolates between deterministic, goal-directed motion and random behavior. We implement this through a domain randomization framework that samples rationality coefficients from a probability distribution, exposing the policy to a diverse range of obstacle behaviors during training. Crucially, this distribution is adapted online via a maximum entropy objective, which expands behavioral diversity while maintaining task performance. We demonstrate that the proposed method, RAtionality randomization with Maximum ENtropy (RAMEN), enables the learning of navigation policies that generalize across a wide range of simulated dynamic environments. Furthermore, we show successful zero-shot deployment on a legged robot operating in dynamic real-world scenarios using only onboard sensory inputs.