Closing the Loop in Scene Reconstruction: Automated Evaluation and Guided Data Collection

Abstract

Recent advances in Neural Radiance Fields and 3D Gaussian Splatting have enabled photo-realistic rendering of complex, view-dependent scenes. However, rendering fidelity heavily depends on strategic viewpoint selection, which currently relies on manual intervention or inefficient uniform sampling. This paper introduces an automated, closed-loop framework that pairs a neural quality estimator with a multi-agent route optimization strategy for targeted, scene-dependent data collection.Our approach integrates two core components: a reference-free perceptual quality network that predicts reconstruction defects without ground-truth images, and a multi-agent path planner formulated as a Vehicle Routing Problem to optimize autonomous resampling trajectories. In an iterative setup, the system reconstructs an initial scene, evaluates local quality to pinpoint artifacts, and dispatches an autonomous fleet to capture targeted supplementary views.Experimental validations in simulated urban environments show a distinct reduction in Learned Perceptual Image Patch Similarity (LPIPS) error after a single refinement loop. While the route optimisation successfully minimizes operational flight costs, our analysis reveals that naive greedy viewpoint selection suffers from severe spatial redundancy. Conversely, introducing a spatial Non-Maximum Suppression (NMS) layer over predicted error distributions breaks up viewpoint clustering, decisively outperforming standard geometric coverage baselines. This framework establishes a scalable, data-efficient pathway toward autonomous, self-improving scene capture systems.