Skip to content
Book Open access

HCNet: Hyperbolic Hierarchical Clause Network for Weakly Supervised Multi-task Visual Grounding

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 50 references

TL;DR

This work proposes HCNet (Hyperbolic hierarchical Clause Network), which first decomposes each expression into four hierarchical clauses via dependency parsing, then refines sentence representations through a hierarchical text enhancer that attends to coarser clause levels.

Abstract

Existing weakly supervised referring expression comprehension (WREC) and segmentation (WRES) methods perform anchor-text matching using flat sentence-level representations, thereby discarding the multi-granularity hierarchical semantics inherent in both visual and linguistic modalities. Because referring expressions naturally form hierarchical clauses of increasing specificity and flat Euclidean space distorts such tree-like structures, explicit hierarchical modeling is necessary yet overlooked. We propose HCNet (Hyperbolic hierarchical Clause Network), which first decomposes each expression into four hierarchical clauses via dependency parsing, then refines sentence representations through a hierarchical text enhancer that attends to coarser clause levels. Subsequently, both visual and textual features are projected into a dual Euclidean-Lorentzian hyperbolic space, where an entailment cone loss enforces hierarchical nesting on the text side and a structure alignment loss transfers this hierarchy to the visual modality. Experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate that HCNet achieves state-of-the-art on both WREC and WRES, with improvements of up to +3.54% over the previous best. The code is available at https://github.com/VILAN-Lab/HCNet.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.