Nov 2026· Proceedings of the 34th ACM International Conference on Multimedia· pp. 9628-9637· 0 citations
Abstract
Existing weakly supervised referring expression comprehension (WREC) and segmentation (WRES) methods perform anchor-text matching using flat sentence-level representations, thereby discarding the multi-granularity hierarchical semantics inherent in both visual and linguistic modalities. Because referring expressions naturally form hierarchical clauses of increasing specificity and flat Euclidean space distorts such tree-like structures, explicit hierarchical modeling is necessary yet overlooked. We propose HCNet (Hyperbolic hierarchical Clause Network), which first decomposes each expression into four hierarchical clauses via dependency parsing, then refines sentence representations through a hierarchical text enhancer that attends to coarser clause levels. Subsequently, both visual and textual features are projected into a dual Euclidean-Lorentzian hyperbolic space, where an entailment cone loss enforces hierarchical nesting on the text side and a structure alignment loss transfers this hierarchy to the visual modality. Experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate that HCNet achieves state-of-the-art on both WREC and WRES, with improvements of up to +3.54% over the previous best. The code is available at https://github.com/VILAN-Lab/HCNet.
Hyperbolic vision-language models are well suited to modeling the hierarchical structure of multimodal data, because hyperbolic volume grows exponentially with radius, enabling low-distortion representations of tree-like taxonomies from general to specific concepts. However, existing pretraining objectives for hyperbol...
Ming-Yang Gao, Yu-Wei Wu, Shi-Yu Luo et al.· 0 citations
Six hierarchical metrics and embedding analyses show that the hierarchical prompt learning plug-in for frozen hyperbolic VLMs produces more taxonomically consistent predictions and induces a hierarchy-consistent organization of parent, class, and image embeddings in hyperbolic space.
Andro Erdelez, P. Mettes, B. Bozorgtabar· arXiv.org· 0 citations
LIRSeg is proposed, which fully replaces explicit CoT with a compact set of learnable latent tokens for reasoning segmentation, and introduces three complementary mechanisms from an information perspective: extreme-advantage sampling for selecting informative training signals, decoupled exploration-stability updates fo...
Referring Expression Comprehension (REC) localizes the image region described by a natural language expression, underpinning applications such as autonomous navigation, human-robot interaction, and medical image analysis. Under weak supervision, REC is challenged by the visual-linguistic semantic gap and the difficulty...
Yue Chang, Tzu-Hsuan Weng, Y. Amin et al.· 0 citations
While large vision-language models excel at optical character recognition, it remains unclear how their underlying vision encoders process text and its meaning, and whether they support anything resembling reading comprehension. We systematically study transformer-based vision encoders, using cross-modal alignment betw...
Sugam Karki, Raúl Vázquez, Yova Kementchedjhieva· Findings of the Association...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.