Across a range of detection transformer models, the results show a surprisingly strong and previously unknown ability of 2D DETR models to represent useful information about the 3D properties of objects, despite the complete lack of 3D supervision during model pre-training.
Abstract
Detection transformer models, including DETR and its extensions, learn to output a set of object-level embeddings that can be simultaneously decoded into 2D bounding boxes and class distributions. In this paper, we investigate what pre-trained 2D detection transformers understand about the 3D properties of objects. Specifically, we investigate the extent to which properties including the depth of objects from the camera and the 3D location of objects relative to the camera can be recovered from object-level embeddings using linear and non-linear probes. Across a range of detection transformer models, our results show a surprisingly strong and previously unknown ability of 2D DETR models to represent useful information about the 3D properties of objects, despite the complete lack of 3D supervision during model pre-training.
This work proposes a framework for detailed object-level reasoning across multiple objects with three components: MO3D, an instruction dataset requiring fine-grained multi-object comparison; Multi-3DLLM, using a minimal Patch-Interaction Transformer (PIT) that models inter-/intra-object relationships while preserving l...
K. Ide, Ryousuke Yamada, Yue Qiu et al.· 2026 IEEE/CVF Conference on...· 1 citation· ⚡1
This work investigates whether a frozen, self-supervised point transformer already contains the structural information required to isolate object instances without any handcrafted geometric prior, and develops a training-free segmenter that groups points via connected components on a key-similarity graph, using neither...
Ted Lentsch, Santiago Montiel-Mar'in, Holger Caesar et al.· 0 citations
Large-scale pre-training has transformed representation learning in 2D vision, yet its transferability to 3D single object tracking (SOT) remains insufficiently understood. Directly fine-tuning self-supervised 3D encoders, such as masked autoencoders (MAE), often leads to sub-optimal adaptation because the reconstructi...
Si-Fan Zhou, Qi-Wei Wang, Lin-Yue Tan et al.· 0 citations
Results from an adaptation of DINOv3 to the instance-level fusion pipeline of IIFNet3D are interpreted as evidence for a controlled transfer recipe rather than as a causal estimate of the individual contributions of VAID or depth filtering.
Lin-Man Wang, Zi-Fei Zhang, Chun-Ran Zheng et al.· 0 citations
Category-level object pose estimation (COPE), capable of generalizing to intra-class unknown objects, has become a core technique for robotic 3D scene understanding. However, existing COPE methods still require labor-intensive recollection of real-world training data for novel object categories, which limits their scal...
Jian Liu, Wei Sun, Zhen-Qi Dai et al.· 0 citations
B2TFPose is presented, a training-free zero-shot method for 6DoF pose estimation of unseen objects from RGB images, establishing state-of-the-art performance among training-free RGB methods and outperforming trained counterparts including GigaPose and GenFlow, at competitive inference speed.
Ali Rafiaei, Michael A. Greenspan· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
