MELON is introduced, the first large-scale dataset designed to extend text-video retrieval to long-form videos featuring complex, multi-event structures, and a multi-event aware loss is proposed that encourages models to differentiate between full-event and partial-event matches, yielding substantial improvements in retrieval accuracy.
Abstract
Existing text-video retrieval datasets primarily consist of short-form clips containing a single dominant event. While suitable for measuring basic vision-language alignment, they are limited in capturing real-world retrieval scenarios, where long-form videos naturally contain multiple semantically distinct events and a single text query may correspond to several non-contiguous temporal segments. To bridge this gap, we introduce MELON, the first large-scale dataset designed to extend text-video retrieval to long-form videos featuring complex, multi-event structures. MELON explicitly annotates multiple event intervals per video along with their corresponding textual descriptions, enabling both training and evaluation of multi-event understanding in long, untrimmed videos. In addition, we propose a multi-event aware loss that encourages models to differentiate between full-event and partial-event matches, yielding substantial improvements in retrieval accuracy. Together, the MELON dataset and our proposed loss establish a robust foundation for expanding text-to-video retrieval to complex long-form scenarios and provide a more realistic evaluation setting for future research in the field.
Current large video-language models (LVLMs) still face challenges when dealing with long videos, mainly because frames are often processed independently, making it difficult to capture temporal dependencies across events. Although retrieval-augmented approaches have been introduced to provide additional context, most o...
Yu-Hao Qin, Jun-Bo Wang, Yu-Ke Li et al.· 0 citations
Experiments on four text-video retrieval benchmarks show that MARS achieves state-of-the-art results in both direct similarity-based retrieval and reranking settings, and a hard-negative-aware slot specialization objective that encourages the slots to capture discriminative matching cues is introduced.
Uicheol Jung, Juyoung Hong, Geuntaek Lim et al.· 0 citations
MultiVENT-Raw is released, a multilingual collection of nearly 120,000 primarily raw videos paired with 130 events and 222 event-centric queries, along with human-annotated video relevance judgments and human-extracted key facts for relevant videos.
Reno Kriz, David Etter, Alexander Martin et al.· 0 citations
Finding the right video often requires distinguishing similar scenes in which different events occur. Joint matching improves retrieval, but processing rich video representations for each query is costly. VEDJE compresses features within sampled frames while keeping their representations separate in a reusable cache. F...
S. Wagner, Gabriele Serussi, D. Ben-Ami et al.· 0 citations
ClearText-Video (CTVid), a large-scale, scene-text-aware benchmark for studying text-centric video understanding under controlled quality variation, is introduced, providing a rigorous foundation for restoration-aware, quality-robust text-centric video systems.
Jinlong Li, Jiaming Ding, Ding-Fu Lu et al.· 0 citations
Multimodal large language models (MLLMs) process multimodal inputs by converting text, images, and videos into token sequences that are subsequently processed by a backbone language model. While MLLMs have achieved excellent performance in understanding the content of individual images, video understanding remains sign...