Action-Conditioned Mamba with Conformal Recovery for PTZ-Based UAV Tracking

Abstract

Pan–tilt–zoom (PTZ) cameras are central to visual counter-UAV surveillance: they steer the optical axis to keep a small, agile target centered and adequately resolved, which tightly couples perception with camera control in a closed loop. Learned PTZ controllers face three obstacles: temporal models that treat the observation stream as exogenous and therefore ignore the agent’s own influence on it; loss detection and recovery driven by hand-tuned thresholds with no measurable notion of reliability; and a costly reliance on real flight data and physical hardware for training. We present CMW-Track, which combines an action-conditioned hierarchical Mamba policy that modulates the state-transition operator with the executed PTZ command; an ensemble state predictor that is deliberately lightweight—it predicts only the low-dimensional target state required for PTZ control rather than reconstructing future frames—and is calibrated by split conformal prediction; and Active Uncertainty-Gated Exploration for Recovery (AUGER), a tracking–uncertain–recovery machine gated by conformal-interval violations instead of a tuned confidence threshold. The controller is trained only on procedurally generated trajectories under domain randomization, with no physical PTZ platform in the loop. Evaluated zero-shot in an unseen high-fidelity Unreal Engine 5 environment, CMW-Track attains the highest tracking-success (81.7%) and loss-recovery (85.0%) rates among all evaluated controllers and matches the best completion rate, in real time (median: 2.5 ms per control step). Against five baselines spanning classical, filtered and recurrent-reinforcement-learning control, it improves tracking success by 5.7 pp over the strongest of them in simulation (95% CI: [+4.4, +6.9]), and the margin persists when that baseline is widened to matched policy capacity. The split-conformal bound holds at calibration time, but closed-loop coverage falls 15–21 percentage points below nominal—a direct measurement of exchangeability breakdown that the deliberately redundant trigger absorbs. We therefore report the trigger as calibrated and auditable, not as a deployment-time guarantee.