Skip to content
Preprint

AquaOrbit: Sim-to-Real Reinforcement Learning for Underwater Target Orbiting under Intermittent Visual Feedback

Sep 2026 · 0 citations · 24 references
Computer Science

TL;DR

AquaOrbit, a reinforcement-learning controller with a recovery module for underwater target orbiting under interrupted visual feedback, is presented and zero-shot physical deployment with fully onboard perception and control demonstrates elliptical, figure-eight, and variable-depth circular trajectories, including the latter two trajectory types absent from training.

Abstract

Intermittent visual loss disrupts target-relative feedback during underwater orbiting, making it difficult to maintain coordinated motion and reacquire a moving target. We present AquaOrbit, a reinforcement-learning controller with a recovery module for underwater target orbiting under interrupted visual feedback. During detection loss, the recovery module uses latched line-of-sight, roll, and depth references to support stabilization and target reacquisition. We train the controller in Isaac Sim with dynamics, observation, and vision-loss randomization. Evaluated without retraining in Gazebo/ROS2 under a different physics engine and perception perturbations, AquaOrbit completes 20/20 orbiting trials in each of the static- and moving-target conditions on an unseen variable-depth 3-D trajectory. In the moving-target condition, it reduces mean line-of-sight error by approximately 46% relative to a PID-based visual servoing controller with recovery while maintaining comparable path-tracking accuracy; removing the recovery module reduces completion to 9/20. Zero-shot physical deployment with fully onboard perception and control demonstrates elliptical, figure-eight, and variable-depth circular trajectories, including the latter two trajectory types absent from training. The robot maintains attitude stability during manual occlusions lasting up to 8s and reacquires the target within 2.5s in the reported attitude-induced field-of-view loss events.

View source

Similar papers

Open access Oct 2026

Confidence-Gated Finite-Horizon Residual SAC–LSTM Control for Visual Target Tracking by a Rudderless Six-Thruster AUV

Autonomous visual tracking of a moving target by a rudderless underwater vehicle is complicated by nonlinear dynamics, intermittent visual observations, environmental currents, and constrained distributed thrust. This paper combines a recurrent SAC–LSTM policy with a finite-horizon model-based residual correction. The...

Vadim Kramar, Aleksandr Liashko, Evgenii Norenko · 0 citations
#machine learning Preprint Sep 2026

Learning to Fly: Stable Vision-Guided UAV Servoing with Compact Target-Centric Cues and Reinforcement Learning

The results demonstrate that compact target-centric representations can support robust long-horizon aerial visual servoing and that visual curriculum training can improve robustness to dynamic distribution shifts despite limited gains in nominal performance.

Saurbh Singh Jamwal, Nived Chebrolu · 0 citations
Preprint Oct 2026

Barrier-Shaped Recurrent Reinforcement Learning for Autonomous Landing on a Heaving Ship Deck

This paper addresses autonomous landing of unmanned aerial vehicle (UAV) rotorcrafts on a heaving ship deck using a recurrent policy trained via asymmetric actor-critic reinforcement learning, validated using a parallel-manipulator-platform-based deck emulator and a quadcopter UAV with an onboard camera.

Ritwik Shankar, Chiranjeev Prachand, Abhishek et al. · 0 citations
#reinforcement learning Open access Sep 2026

Action-Conditioned Mamba with Conformal Recovery for PTZ-Based UAV Tracking

Pan–tilt–zoom (PTZ) cameras are central to visual counter-UAV surveillance: they steer the optical axis to keep a small, agile target centered and adequately resolved, which tightly couples perception with camera control in a closed loop. Learned PTZ controllers face three obstacles: temporal models that treat the obse...

Zi-Liang Sang, Wei Han, Hong-Wei Liu · 0 citations
Preprint Oct 2026

ISB-NMPC: Interception-Oriented Spherical-Bearing Nonlinear Model Predictive Control for Quadrotors

Quadrotor interceptors equipped with strapdown cameras offer a low-cost solution for physically countering unauthorized UAVs. However, this configuration inherently couples vehicle maneuvering and camera pointing, creating a conflict between rapid closing and maintaining target visibility. We present interception-orien...

Li Hong, Lin-Kai Liu, Xiang-Yu Ren et al. · 0 citations
Preprint Oct 2026

Learning Event-based Obstacle Avoidance via Differentiable Simulation

Reliable obstacle avoidance is essential for safe quadrotor flight in unknown environments, yet conventional frame-based cameras remain challenged by motion blur, limited dynamic range, and latency under fast motion. Event cameras, with sub-millisecond latency, high dynamic range, and negligible motion blur, offer a pr...

Chen-Lu Shou, Yun-Fan Ren, Jiaxu Xing et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.