SGP-Net: semantically-guided multi-task refinement for monocular depth estimation in scenes with potentially movable object categories
TL;DR
Semantically-guided progressive network (SGP-Net) is proposed, a semantically guided progressive refinement framework for MDE based on multi-task learning that improves key relative-error and accuracy metrics over the DCDepth baseline and remains competitive with recent methods.
Abstract
Monocular depth estimation (MDE) is a fundamental task for 3D scene understanding and provides an efficient perception solution for autonomous driving, robotic navigation, augmented reality, and other intelligent systems. Despite recent progress in supervised models and large-scale depth foundation models, accurate MDE in scenes containing potentially movable object categories remains challenging, since such foreground categories often introduce feature entanglement with static background regions and blurred object boundaries. In this paper, ‘dynamic’ refers to potentially dynamic semantic-category regions, such as vehicles and pedestrians, derived from semantic labels or pseudo-labels; it does not denote explicit frame-level motion estimated from optical flow, tracking, or multi-frame input. To address these challenges, we propose semantic-guided progressive network (SGP-Net), a semantically guided progressive refinement framework for MDE based on multi-task learning. SGP-Net is built upon DCDepth and extends it with semantic-aware branches and dynamic-category constraints. SGP-Net consists of two key components. First, a semantically-guided pyramid feature fusion module uses pixel-level semantic priors to reweight multi-scale depth features, thereby preserving object contours and enhancing geometric perception in dynamic-category regions. Second, a dynamically weighted multi-constraint loss with a phased weight scheduling strategy is designed to balance scale-invariant and edge-aware supervision during training while alleviating negative transfer between tasks. Experiments on KITTI, NYU-Depth-v2, and TOFDC show that SGP-Net improves key relative-error and accuracy metrics over the DCDepth baseline and remains competitive with recent methods. On KITTI dynamic-category pixels, SGP-Net reduces AbsRel from 0.0477 to 0.0448 and improves δ1 from 0.9786 to 0.9862. Qualitative results and ablation studies further verify the effectiveness of semantic-guided refinement and dynamic-category supervision in regions belonging to potentially movable categories.