Neuromorphic control of a simulated shape-memory-alloy-driven multi-legged robot using a spiking neural network

Abstract

Abstract We present spiking-neural-network (SNN) reinforcement learning for autonomous locomotion control of a starfish-inspired multi-legged soft robot driven by shape memory alloy (SMA) actuators. Such robots are usually driven open loop by pre-designed periodic signals, because coordinated motion is hard to hand-design at high degrees of freedom. We replace the actor of Twin Delayed Deep Deterministic Policy Gradient (TD3) with an SNN: a Spiking Actor Network. The membrane potential, an internal SNN state, is reconstructed at update time from a short observation history through the current actor, rather than stored at collection time. To reduce the controller’s inference cost, we add a regularization term on the squared membrane potentials, encouraging actions from fewer spikes. In simulation the agent acquired the leg coordination needed to approach a target, and its firing patterns developed a periodicity matched to the SMA response. The resulting gait gives continuous propulsion without excessive firing. Goal error matched an ANN-actor TD3 baseline with smaller spread across seeds, and reconstruction gave the lowest goal error of the membrane-state treatments. The regularization lowered the firing rate until the estimated inference energy fell below the ANN’s.