Jul 2026· Measurement science and technology· Vol 37, pp. 325405· 0 citations· 38 references
Physics
TL;DR
Semantically-guided progressive network (SGP-Net) is proposed, a semantically guided progressive refinement framework for MDE based on multi-task learning that improves key relative-error and accuracy metrics over the DCDepth baseline and remains competitive with recent methods.
Abstract
Monocular depth estimation (MDE) is a fundamental task for 3D scene understanding and provides an efficient perception solution for autonomous driving, robotic navigation, augmented reality, and other intelligent systems. Despite recent progress in supervised models and large-scale depth foundation models, accurate MDE in scenes containing potentially movable object categories remains challenging, since such foreground categories often introduce feature entanglement with static background regions and blurred object boundaries. In this paper, ‘dynamic’ refers to potentially dynamic semantic-category regions, such as vehicles and pedestrians, derived from semantic labels or pseudo-labels; it does not denote explicit frame-level motion estimated from optical flow, tracking, or multi-frame input. To address these challenges, we propose semantic-guided progressive network (SGP-Net), a semantically guided progressive refinement framework for MDE based on multi-task learning. SGP-Net is built upon DCDepth and extends it with semantic-aware branches and dynamic-category constraints. SGP-Net consists of two key components. First, a semantically-guided pyramid feature fusion module uses pixel-level semantic priors to reweight multi-scale depth features, thereby preserving object contours and enhancing geometric perception in dynamic-category regions. Second, a dynamically weighted multi-constraint loss with a phased weight scheduling strategy is designed to balance scale-invariant and edge-aware supervision during training while alleviating negative transfer between tasks. Experiments on KITTI, NYU-Depth-v2, and TOFDC show that SGP-Net improves key relative-error and accuracy metrics over the DCDepth baseline and remains competitive with recent methods. On KITTI dynamic-category pixels, SGP-Net reduces AbsRel from 0.0477 to 0.0448 and improves δ1 from 0.9786 to 0.9862. Qualitative results and ablation studies further verify the effectiveness of semantic-guided refinement and dynamic-category supervision in regions belonging to potentially movable categories.
3D Semantic Scene Completion (SSC), a cornerstone task in computer vision, aims to simultaneously predict the geometric structure and semantic content of 3D scenes from sparse inputs. This capability is crucial for advancing applications in robotics, autonomous driving, and virtual reality. However, SSC faces significant challenges, including the high computational cost of capturing long-range contextual information and the scarcity of 3D semantic labels leading to overfitting. To address these limitations, we propose an enhanced network for semantic scene completion. Firstly, we devise a 3D Local- Global Linear Attention Mechanism (LG-LAM) that efficiently captures long-range contextual information with linear complexity, enabling a comprehensive understanding of the 3D scene without heavy computational burdens. Secondly, a 3D Feature Augmentation Module (FAM) is integrated to enrich feature diversity through rotation-invariant learning, mitigating overfitting and enhancing the model’s robustness given limited annotations. Extensive experiments on the NYUCAD dataset demonstrate that our method achieves state-of-the-art performance among non-iterative methods while introducing negligible computational overhead.
Jie Li, Jiaheng Xu, Laiyan Ding et al.· International Conference on...· 0 citations
Multi-Task Learning (MTL) in robotics perception systems supports comprehensive 3D spatial scene understanding by integrating semantic segmentation and depth estimation. While Vision Foundation Models (VFMs) are increasingly adopted as robust feature encoders, existing decoding strategies present a critical bottleneck. To address this, we propose DPNeXt, a streamlined multi-scale feature fusion decoder and efficient alternative to the standard Dense Prediction Transformer (DPT). DPNeXt uses dual depthwise separable inverted bottlenecks to improve frozen VFM utilization through fusion-centric decoding and independent task modularization. To further mitigate negative inductive transfer between tasks, we introduce the Multi-Task Boundary Guidance (MTBG) strategy. Unlike prior boundary-aware methods that add fusion modules or gating, MTBG applies symmetric boundary-focused supervision to encourage geometric consistency without extra annotation or inference cost. Experiments on Cityscapes show that DPNeXt-S outperforms prior state-of-the-art (SOTA) MTL models, while DPNeXt-B further improves the overall performance and achieves the best results among the compared methods. On NYUv2, DPNeXt-B also achieves the best semantic segmentation and depth estimation results among the compared methods while requiring substantially fewer trainable parameters than prior large-scale MTL models. Compared with the standard DPT, DPNeXt-S reduces trainable parameters by 78.6% and achieves the fastest inference speed among the compared models on resource-constrained laptop hardware. The source code, model checkpoints, and a demo video will be made available at https://github.com/kangjehun/DPNeXt.
Jehun Kang, Jungha Wang, Young-Joon Hwang et al.· 0 citations
Multi-object removal in 3D scenes is challenging due to severe occlusions, semantic entanglement, and the difficulty of maintaining geometric and multi-view consistency. Existing 3D Gaussian Splatting (3DGS) methods perform well for single-object editing but scale poorly to multi-object scenarios, often requiring repetitive optimization and yielding unstable geometry in removed regions. We propose CoGeo-GS, a concept-driven framework for controllable multi-object removal in 3D scenes. CoGeo-GS assigns concept-aware semantic tags to Gaussians, enabling flexible object selection and reducing interference between foreground objects and background structures within a single optimization stage. To recover plausible geometry, we introduce a geometry-aware completion pipeline that combines monocular depth priors with diffusion-based refinement and boundary-aligned blending. A geometry-regularized refinement strategy further stabilizes reconstruction and preserves multi-view consistency. Experiments demonstrate that CoGeo-GS outperforms existing methods in visual quality and reconstruction fidelity.
Yuanxiang Ni, Xianliang Huang, Chen-Hang Ma et al.· 0 citations
For embodied agents to navigate and reason indoor spaces, they need object-level 3D representations that stay consistent over time as new frames arrive from a monocular camera. Current online 3D instance segmentation methods either depend on posed RGB-D input with ground-truth depth or couple tightly to the internal representations of specific foundation models, sacrificing modularity. We observe that appearance-based and geometry-based object matching exhibit complementary failure modes: appearance is ambiguous among spatially separated duplicates, while geometry is unreliable for visually distinct objects at similar locations. This motivates SAM3R, a training-free pipeline that fuses spatial overlap, 3D centroid displacement, and visual-semantic similarity into a single assignment cost solved via bipartite matching. The cost is constructed entirely from the outputs of frozen foundation models without accessing internal representations. Object tracks are classified through a cascaded decision tree that detects scene changes via field-of-view gated temporal voting. On ScanNet200 and Replica, SAM3R performs competitively with methods that require architecture-specific features or additional training, despite operating in a fully online, monocular setting. Qualitative evaluation on the Aria Digital Twin dataset further demonstrates that the pipeline maintains correct object identities through physical object manipulation, including hand occlusion and large spatial displacement.
M. Mohrat, E. Derevyanka, I. Obrubov et al.· Journal of Instrument Engine...· 0 citations
Camera-reconstructed 3D scenes often require offline visual cleanup before inspection, presentation, or reuse as renderable virtual-scene assets. Representative applications include removing temporary furniture, parked vehicles, equipment, signage, and other distracting or obsolete objects from reconstructed indoor and outdoor environments. Such editing requires not only accurate target localization across viewpoints but also plausible recovery of the previously occluded background. Existing methods often depend on manually specified masks or category-restricted detectors, while projection-based pipelines independently inpaint multiple views and subsequently refine the 3D representation, potentially introducing cross-view appearance and geometry inconsistencies. We present OVR-GS (Open-Vocabulary Removal in Gaussian Splatting), an instruction-driven object-removal framework for pre-trained 3D Gaussian Splatting (3DGS) scenes. Given a free-form instruction, a language parser generates target-oriented queries and a textual background-completion condition. Grounding DINO and the Segment Anything Model (SAM) produce multi-view candidate masks, which are filtered using Contrastive Language–Image Pre-training (CLIP). The proposed Semantic-Aware Gaussian Selector (SAGS) aggregates rendering-contribution-weighted mask evidence, groups spatially coherent candidates, and identifies the target Gaussian subset through rendered-cluster semantic verification. After removal, new Gaussians are initialized from boundary-adjacent primitives and interior samples and optimized locally using Score Distillation Sampling (SDS), while the original background remains fixed. On IMFine, SPIn-NeRF, and Inpaint360GS, OVR-GS achieves peak signal-to-noise ratio (PSNR) values of 19.78, 17.82, and 24.62 dB and Fréchet inception distance (FID) values of 142.30, 148.60, and 34.80, respectively. The results demonstrate the effectiveness of localized Gaussian optimization for instruction-driven cleanup of reconstructed environments before visual inspection, presentation, or reuse as renderable virtual-scene assets.
Yongpeng Ding, Feng Ouyang, Jiawei Fan et al.· Italian National Conference...· 0 citations
Semantic scene completion (SSC) requires a model to infer both the geometry and semantic labels of a complete 3D scene from partial point cloud observations. Recent point cloud SSC methods improve efficiency over dense volumetric formulations, yet progressive point cloud decoders must still resolve three coupled sources of ambiguity: local spatial context around each generated point, information exchange between geometric refinement and semantic prediction, and semantic relations among visually or structurally similar scene classes. We propose Bi-SGL, a Bidirectional, Spatially Grounded, and Language-informed framework for point cloud semantic scene completion. Bi-SGL integrates three complementary signals within a stagewise semantic-geometric decoder. Spatially-Aware Hierarchical Geometric Grounding (SAHGG) aggregates multi-resolution encoder features into a pointwise spatial grounding signal. Bidirectional Branch Coupling (BBC) enables geometry and semantics to exchange information during progressive refinement. Class-Similarity Adaptive Bias (CSAB) introduces language-derived class priors from frozen OpenCLIP text embeddings: a text-seed prior preserves task-specific class identities while injecting class-level semantics, and a class-relation prior biases semantic decoding with inter-class similarities. Experiments on SSC-PC and NYUCAD-PC show that Bi-SGL improves semantic completion accuracy among evaluated point cloud SSC baselines while using substantially fewer parameters than cascaded dense-fusion models. On SSC-PC, Bi-SGL improves mIoU from 90.71% to 92.92% over ProtoFormer while reducing CD from 8.917 to 8.672. On NYUCAD-PC, it achieves 51.46% mIoU, outperforming ProtoFormer by 2.27 percentage points and CasFusionNet by 2.13 percentage points. Ablation and class-level analyses indicate that spatial grounding, branch coupling, and language-derived class priors make complementary contributions to point cloud SSC.
Houda Saffi, N. Otberdout, A. E. Seghrouchni· IEEE Access· 0 citations