TL;DR
This work extends its previous work in scene graph generation to integrate image captioning, incorporating large language models to interpret scene graph structures and produce high-quality captions encompassing complex actions and relationships.
TA-LMM, a 3D visual question answering method built on explicit scene graphs and local topological priors, is proposed, suggesting that explicit local topological priors can improve scene consistency in 3D visual question answering.
The Cross-Modal Dual-Path Generator (CM-DPG), a model for robust open-vocabulary SGG that enhances object-level semantic representations through joint visual-textual encoding and improves relational reasoning using complementary visual and geometric cues, is proposed.
SPOT is introduced, a structured prompting framework that augments open-source VLMs with spatial reasoning abilities for scene graph generation with minimal training, and achieves competitive or superior relation prediction compared to large proprietary models.
Image captioning models can produce rapid Sentences, without visual relationships, or insert non-existing plausible objects. A common cause is to compress image evidence into visual symbols that carry a weak neighbourhood context. The multimodal context-enhanced visual representation learning framework (MCVRL) addresse...
This work introduces three tasks/objectives for reverse localization of text as an instruction-tuning mechanism, where the model is guided to extract textual content based on spatial localization cues, thereby enhancing its spatial grounding ability.
Despite the recent advances in Transformer-based image captioning models, reliance on implicit semantic representations and the lack of integration of topic-level and word-level semantic information with appearance and geometric features remain challenging. To address these limitations, we propose a semantic modeling f...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
