Contact-History-Based Velocity-Tracking Reward Derived from Center-of-Mass Controllability for Quadruped Locomotion

Abstract

Velocity tracking is a fundamental task in reinforcement learning for quadruped locomotion. Conventional velocity-tracking rewards are typically formulated in terms of the squared error between commanded and actual velocities. In this study, we propose a velocity-tracking reward from the controllability Gramian of center-of-mass (CoM) dynamics, with ground reaction forces (GRFs) as control inputs. The proposed reward dynamically scales the velocity error according to recent contact history. It is defined as a quadratic form of the translational velocity-error vector and the inverse controllability Gramian, thereby embedding CoM controllability through GRFs into the velocity-error metric. We compared policies trained using the proposed reward with those trained using a conventional squared-error-based velocity-tracking reward. When used as the translational velocity-tracking term, the proposed reward reduced the cost of transport (CoT) by approximately 29% and the time-averaged squared vertical velocity by approximately 93.4%. In addition, increasing the weight of the proposed reward tended to shift the learned gait from bound to trot. These results suggest that incorporating CoM controllability into velocity tracking suppresses unnecessary vertical motion and promotes the learning of body states suitable for forward locomotion.