FROM TRAJECTORY OPTIMIZATION TO LEARNING: DEVELOPING CONTROLLERS FOR DYNAMIC LEGGED ROBOTS

Abstract

Legged robots are well suited for cluttered, uneven, and human-centered environments that remain challenging for wheeled systems. However, achieving agile, robust, and efficient locomotion comparable to that of humans and animals in such settings remains difficult due to underactuation, nonlinear hybrid dynamics, and the need for robustness to model uncertainty, complex interactions with uncertain environments, and external disturbances, all under strict real-time control requirements. To address these challenges, this thesis develops control frameworks for legged robots that bridge model-based trajectory optimization and data-driven learning, enabling safer and more robust locomotion, scalable generation of whole-body dynamic motions, and successful deployment of learned behaviors on robots whose hardware dynamics are difficult to capture accurately in simulation. First, this thesis integrates a reduced-order model predictive control (MPC) locomotion pipeline with Riemannian Motion Policy (RMP)-based reactive control for self-collision avoidance. This framework expands the set of safely reachable footholds, improves disturbance rejection, and is validated in large-scale simulation and on a custom point-foot bipedal platform. Beyond reactive locomotion, this thesis next considers the generation of highly dynamic whole-body motions. To this end, it presents a biomechanics-inspired framework that combines offline kinodynamic trajectory optimization with imitation-based reinforcement learning to generate and track human-like dynamic motions. Applied to a humanoid soccer-kicking task, this method produces powerful kicks and improves training sample efficiency by using dynamically consistent reference trajectories rather than purely kinematic ones. While effective, generating each new motion with this framework requires substantial manual effort, which limits its scalability to large motion libraries. To address this limitation, this thesis introduces a fast, physically informed motion-retargeting pipeline for large-scale humanoid learning from motion-capture data. A custom SQP-based inverse-kinematics solver with parallel processing achieves up to 10,000 frames per second, enabling large datasets such as AMASS to be processed in minutes rather than days without sacrificing retargeting fidelity. A contact-implicit MPC based dynamics validation stage further improves physical feasibility and helps filter out motions that lie outside the hardware capabilities of the robot before policy training, thereby avoiding wasted computation on infeasible behaviors. Finally, this thesis addresses a complementary challenge in transferring behaviors learned in simulation to real robots, particularly for systems with coupled, cooperatively actuated joints and actuated toes, where standard open-chain simulation often fails to capture important hardware-specific dynamics. To bridge this gap, the thesis develops a reinforcement-learning framework that incorporates actuator-space action parameterization, coupling-aware torque constraints, critical physical effects such as reflected rotor inertia, and system-identification-informed priors. This framework enables successful hardware demonstration of dynamic motions on StaccaToe, an anthropomorphic single-leg hopping robot. Together, these contributions advance scalable, transferable, and physically grounded control for dynamic legged robots.