Recurrent Actor-Critic Navigation of Unmanned Aerial Vehicles in Arbitrary-Shape Obstacle-Dense Environments via Temporally-Augmented Observation

Abstract

Reactive navigation of Unmanned Aerial Vehicles (UAVs) in cluttered environments using only local sensor feedback is severely hampered by partial observability: irregularly-shaped obstacles can disappear from the sensor’s instantaneous field of view, causing wrap-around collisions and local-minima traps. However, existing Reinforcement Learning approaches only partially address these challenges, as they commonly assume regularly shaped, often circular or convex, obstacles. This work proposes a lightweight recurrent extension of the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm that tackles navigation in cluttered environments populated with arbitrary-shape obstacles. By augmenting the agent’s observation with a sliding window of recent states, the resulting Temporally-Augmented Observation TD3 (TAO-TD3) feeds stacked observations through a recurrent feature backbone, enabling the policy to better infer obstacle structure from temporal information. Benchmark evaluations across five environments with increasing obstacle density and count demonstrate that TAO-TD3 consistently reduces collisions and improves goal-reaching rates relative to memory-less Reinforcement Learning.