A computational framework for Kármán gaiting in robotic fish: spatio-temporal perception and CPG-based reinforcement learning

Abstract

Navigating in unsteady wake flows, such as Kármán vortex streets, presents a formidable challenge for biomimetic autonomous underwater vehicles. Biological fish achieve this by utilizing their lateral line sensory systems to perceive local flow gradients and adopting an energy-efficient swimming pattern known as the Kármán gait. To translate this biological phenomenon into a practical robotics engineering solution, this paper proposes a fully computational framework focusing on the modeling and simulation of a spatio-temporal sensory system to autonomously generate the Kármán gait. To overcome the unrealistic assumption of full-state observability common in existing reinforcement learning studies, we model a multi-point lateral line array coupled with a frame-stacking mechanism. This allows the simulated agent to reconstruct the spatio-temporal topology of the surrounding unsteady flow relying exclusively on local pressure and velocity gradients. The sensory model is integrated with a spatio-temporal perceptual twin delayed deep deterministic policy gradient (STP-TD3) algorithm, which drives a Hopf-oscillator-based central pattern generator. Through rigorous high-fidelity computational fluid dynamics simulations, we quantitatively evaluate the autonomous emergence of the Kármán gait by assessing the agent’s kinematic energy proxy-mapped from joint actuation effort. Results reveal that the agent expends significantly less mechanical effort navigating through the turbulent vortex street compared to swimming in steady water, suggesting the active exploitation of the local wake dynamics. The results theoretically underscore the necessity of distributed STP for biomimetic robots, providing a robust algorithmic blueprint for future physical deployments in complex aquatic environments.