Bi-SGL: Bidirectional, Spatially Grounded, and Language-Informed Framework for Semantic Scene Completion
TL;DR
Bi-SGL is proposed, a Bidirectional, Spatially Grounded, and Language-informed framework for point cloud semantic scene completion that improves semantic completion accuracy among evaluated point cloud SSC baselines while using substantially fewer parameters than cascaded dense-fusion models.
Abstract
Semantic scene completion (SSC) requires a model to infer both the geometry and semantic labels of a complete 3D scene from partial point cloud observations. Recent point cloud SSC methods improve efficiency over dense volumetric formulations, yet progressive point cloud decoders must still resolve three coupled sources of ambiguity: local spatial context around each generated point, information exchange between geometric refinement and semantic prediction, and semantic relations among visually or structurally similar scene classes. We propose Bi-SGL, a Bidirectional, Spatially Grounded, and Language-informed framework for point cloud semantic scene completion. Bi-SGL integrates three complementary signals within a stagewise semantic-geometric decoder. Spatially-Aware Hierarchical Geometric Grounding (SAHGG) aggregates multi-resolution encoder features into a pointwise spatial grounding signal. Bidirectional Branch Coupling (BBC) enables geometry and semantics to exchange information during progressive refinement. Class-Similarity Adaptive Bias (CSAB) introduces language-derived class priors from frozen OpenCLIP text embeddings: a text-seed prior preserves task-specific class identities while injecting class-level semantics, and a class-relation prior biases semantic decoding with inter-class similarities. Experiments on SSC-PC and NYUCAD-PC show that Bi-SGL improves semantic completion accuracy among evaluated point cloud SSC baselines while using substantially fewer parameters than cascaded dense-fusion models. On SSC-PC, Bi-SGL improves mIoU from 90.71% to 92.92% over ProtoFormer while reducing CD from 8.917 to 8.672. On NYUCAD-PC, it achieves 51.46% mIoU, outperforming ProtoFormer by 2.27 percentage points and CasFusionNet by 2.13 percentage points. Ablation and class-level analyses indicate that spatial grounding, branch coupling, and language-derived class priors make complementary contributions to point cloud SSC.