3D vision-language question answering with explicit scene graphs and local topology priors

Abstract

The development of 3D large multimodal models (3D-LMMs) has advanced research on 3D visual question answering. Yet most existing methods rely on implicit feature mapping, where point clouds or scene features are directly projected into the latent space of a language model, without explicitly modeling local spatial structure. In complex indoor environments, this design can lead to spatial judgments that are inconsistent with the actual physical layout during 3D visual question answering, a phenomenon referred to as spatial hallucination. To address this issue, this paper proposes TA-LMM, a 3D visual question answering method built on explicit scene graphs and local topological priors. The method begins by parsing a raw 3D scene into an explicit spatial-semantic scene graph and extracting instance-level representations that encode both semantic features and geometric location information. It then constructs a local physical neighborhood around the target object, serializes neighboring objects together with their distance information into structured priors, and injects them into the multimodal reasoning sequence as conditional context. Under the current experimental setting on the Replica dataset, the results show that the proposed method achieves strong performance on spatial-relation question answering while also alleviating spatial hallucination to a certain extent. These findings suggest that explicit local topological priors can improve scene consistency in 3D visual question answering.