Skip to content

DART: Depth-as-Target Pretraining for Surgical Vision Foundation Models

Sep 2026 · arXiv.org · Vol abs/2609.04555 · 0 citations · 53 references
Computer Science

TL;DR

DART shows that freely available geometric pseudo-labels can strengthen foundation model pretraining without extra labels or added inference cost, pointing toward stronger backbones for surgery.

Abstract

Vision foundation models (VFMs) are valuable in data-scarce domains such as surgery, where a single pretrained backbone can provide rich representations for many downstream tasks. Yet the dominant self-supervised pretraining paradigm uses only RGB images, leaving readily available complementary signals, such as depth maps, unused. This is a particular missed opportunity in surgery, where natural-image VFMs transfer poorly while the scene geometry is rich and informative. With strong off-the-shelf models now able to produce pseudo-labeled dense depth for any image corpus, we hypothesize that such signals can be folded into pretraining to learn better representations. We present DART, an RGB-D pretraining recipe that builds on DINOv2 with a simple modification: a pixel-space depth reconstruction objective applied to masked iBOT patches, supervised by pseudo-labeled depth. Depth is used only during pretraining, so fine-tuning and inference remain RGB-only. We find that this pixel-level reconstruction head improves representation quality rather than disrupting it. We further show that depth, which encodes scene geometry, is more effective as a target than alternative dense signals such as Canny edges, confirming that the gains stem from depth rather than added supervision alone. Across eight surgical benchmarks spanning segmentation, depth estimation, and image-level recognition, DART outperforms both natural-image and in-domain baselines, including a vanilla DINOv2 trained on identical data, improving dense prediction while also strengthening image-level understanding. More broadly, DART shows that freely available geometric pseudo-labels can strengthen foundation model pretraining without extra labels or added inference cost, pointing toward stronger backbones for surgery.

View source

Similar papers

#artificial intelligence Preprint Nov 2026

InnerLogic-Surg: An Open Surgical Foundation Model via Continual Pretraining

Vision foundation models are the backbone of modern surgical image analysis, yet most surgical systems still rely on encoders pretrained on natural images or on specialist models trained from scratch on comparatively small datasets. Pretraining encoders on large amounts of surgery-specific data has the potential to gen...

Kun Yuan, Hao Ding, Yi-Qing Shen et al. · 0 citations
Open access Sep 2026

SurgDepth: Test-Time Adaptation of Depth Foundation Models for Surgical Scene Understanding.

Through the first systematic study of TTA for dense depth regression (eight baselines), it is shown that existing TTA methods consistently fail in this setting: entropy-based methods degrade regardless of the number of adapted parameters, and temporal-photometric adaptation diverges on non-rigid tissue.

Sheng-Yao Wang, Jun-Pei Xue, Xin Zhong et al. · 1 citation
Open access Oct 2026

Masked Position Modeling as a Pre-Training Strategy for Foundational Deformable Image Registration Models.

The core objective of deformable image registration (DIR), estimating a displacement vector field that aligns two images, is fundamentally the same across imaging modalities and anatomical regions. Although this shared objective should imply the use of foundation models, most deep learning approaches for DIR operate wi...

Wenke Karbole, Vasiliki Sideri-Lampretsa, Hua-Qi Qiu et al. · 0 citations
Book Open access Nov 2026

IC-DVLoRA: Input-Conditioned Dynamic Adaptation for Endoscopic Depth Foundation Models

Analysis of routing utilization, top-k selection, trainable parameters, and inference latency demonstrate that IC-DVLoRA provides an effective and efficient adaptation strategy for endoscopic depth foundation models.

Ze-Yun Liu, Ke Niu, Xue Feng et al. · 0 citations
Preprint Oct 2026

Dyna3: VLM-Guided Training-Free 4D Reconstruction via Depth Foundation Models

The key insight is that DA3's cross-view features, though trained only for depth consistency, implicitly encode motion-discriminative signals when combined with best-match feature search across frames, which could enable much denser temporal sampling that prior methods cannot support.

Xin-Hao Xiang, Wei-Yang Li, Zhi-Jie Zheng et al. · 0 citations
Preprint Nov 2026

Do Pretrained Backbones Transfer to Physical Sensors? A Systematic Study of Multimodal Initialization Across the Sensor Stack

It is found that MultiMAE is a surprisingly strong out-of-the-box baseline, transferring effectively even to modalities it has never seen, while targeted modifications yield incremental but consistent improvements on multi-sensor transfer and overall robustness.

Fabio Montello, Ronja Güldenring, Lucas M. Sandby et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.