A Projector-Based Augmented-Reality Localization Device for Breast-Conserving Surgery: Phantom Pipeline and End-to-End Error Budget

Abstract

Breast-conserving surgery (BCS) requires accurate intraoperative localization of the tumor, which moves between the prone diagnostic and supine surgical positions. We present a projector-based augmented-reality (AR) localization system that projects a crosshair directly onto the patient’s skin at the predicted tumor location. The pipeline combines rigid registration of ceramic fiducials between prone and supine CT (tumor error 0.30 mm on a real phantom), a SOFA finite-element simulation of gravity-driven prone-to-supine soft-tissue deformation (mean bead displacement 24.7 mm on a synthetic mesh; not validated on a deformable phantom), two-stage ICP surface alignment (RMSE 0.92 mm on a simulated camera; not yet validated on real D415 acquisitions), and projector–camera (ProCam) extrinsic calibration driving the crosshair projection. Because the tumor and sub-surface fiducials are not visible in the live RGB-D scene, a direct non-circular in vivo target registration error (TRE) on the tumor cannot be measured in the present phantom configuration. We instead report a component-based end-to-end error budget obtained as the root-sum-square of three independently measured component errors: 2.7 mm (RSS planning estimate, not a directly measured end-to-end accuracy) —rigid CT 0.30 mm, ProCam physical projection 2.50 mm, ICP 0.92 mm. The ProCam term is largest but sub-3 mm. The Gray-code reprojection residual (~11 mm in 15-pose recalibration; 8.62 mm in the deployed 4-pose calibration) overestimates the actual projection error by ~4× (resp. ~3.4×): least-squares calibration averages the consumer-DLP vertical quantization (16-level Gray-code decode on XPR-DLP) into an accurate rigid transform, so the per-observation residual reflects observation quantization, not projection accuracy. The quantization’s measurable cost is projection variance (max 5.5 mm), which motivates phase-shifting structured light. Boundary conditions on phantom rigidity, sub-surface fiducials, and simulated-camera ICP are stated explicitly.