Pareto frontier-guided differentiated skill reinforcement learning for quadrupedal wheeled-legged robots

Abstract

Wheeled-legged robots possess both extreme maneuverability and strong terrain adaptability, but there is an inherent physical contradiction between high-speed locomotion and disturbance-rejection robustness. Traditional end-to-end reinforcement learning (RL) methods typically rely on black-box trial and error, lacking explicit constraints on the physical performance limits of the system, which causes policies to easily become conservative or highly unstable. To address this issue, this paper proposes a Pareto frontier-guided differentiated skill reinforcement learning framework for quadrupedal wheeled-legged robots. First, a wheeled-legged composite single rigid body dynamics model is established, and quantitative indicators representing system agility and robustness are defined based on the dynamic Zero Moment Point (ZMP) theory. The Non-dominated Sorting Genetic Algorithm II (NSGA-II) is utilized to calculate the Pareto optimal frontier curve of velocity-stability margin within a specific physical domain. Subsequently, this theoretical boundary is transformed into a parameterized reward function in RL to achieve policy optimization under mechanistic constraints. In a physical simulation environment, an ”agility expert” and a ”robustness expert” policy, approaching the two ends of the Pareto frontier respectively, are successfully trained. Comparative validation experiments demonstrate that the mechanism-guided training paradigm effectively improves the physical consistency of the policies. The agility expert achieves extreme velocity tracking on flat terrain, while the robustness expert exhibits outstanding posture recovery capabilities on unstructured terrain and under external disturbances. This study provides a new perspective combining physical interpretability and learning flexibility for the high-performance locomotion control of wheeled-legged robots.