A Sim-to-Real Approach towards Optimal Assistance Torque in Hip Exoskeleton with a Coupled Human-Exoskeleton Model

Abstract

This study presents a comprehensive sim-to-real approach aimed at determining the optimal assistance torque for hip exoskeletons, based on reinforcement learning and musculoskeletal simulation. A coupled human-exoskeleton modeling framework was developed, in which a bilateral hip exoskeleton with dynamic properties was explicitly integrated into an anatomically-accurate human musculoskeletal model. Reinforcement learning (RL) algorithm was employed with a phase-aware policy trained utilizing stage-wise learning, in order to generate a bilateral hip assistive torque profile. The learned torque profiles exhibited a stable and consistent biphasic pattern over the gait cycle. The musculoskeletal simulations showed that the gluteus maximus and the flexor digitorum longus muscles had reduced activation at their functional phases with exoskeleton assistance. The learned torque profile was then directly deployed to a customized hip exoskeleton system for real-word applications. In the preliminary metabolic experiments, the overall metabolic power decreased by 3.8%, and baseline-subtracted power decreased by 6.1% relative to the walking without exoskeleton condition. These findings offer compelling evidence that underscores the pivotal role of integrating sim-to-real techniques with musculoskeletal modeling in enhancing walking-assistive exoskeletons, paving the way for more advanced control strategies and bridging the gap between simulation and real-world deployment.