Skip to content

Beyond Scalar IoU: Structured Verification from Rollout Groups for Video Temporal Grounding

Oct 2026 · 0 citations · 18 references
Computer Science

TL;DR

This work introduces SUTURE, which conditions verification on the rollout group and exploits its structure at two complementary scales: disagreement across rollouts controls how strongly the target is reweighted, while coverage at each position determines where reward mass is redistributed.

Abstract

Reinforcement learning with verifiable rewards (RLVR) provides a natural framework for adapting pretrained models to video temporal grounding, where generated temporal intervals can be scored directly against ground truth intervals. Yet existing overlap verifiers typically score each rollout independently, leaving the joint structure of the rollout group unused. We introduce SUTURE, which conditions verification on the rollout group and exploits its structure at two complementary scales: disagreement across rollouts controls how strongly the target is reweighted, while coverage at each position determines where reward mass is redistributed. We show that the resulting verifier admits an exact decomposition into the standard IoU term and a covariance correction determined by the rollout group. A local gradient diagnostic finds a preference for responses covering relatively less supported target regions in the analyzed groups. Across five temporal grounding benchmarks, SUTURE improves grounding performance at every reported IoU threshold. Its trained policy also shows less video-start anchoring in reasoning traces: for later events, the first temporal mention more often overlaps the annotated target. Together, these results show that the joint structure of a rollout group can support a more informative temporal verifier.

View source

Similar papers

Review Sep 2026

Grounding with Confidence: Controllable Generative Video Temporal Grounding

Video temporal grounding supports applications such as video search, content review, and automated editing by localizing events described in natural language. Yet existing generative models typically output timestamps without explicit interval-level confidence scores to guide candidate selection. We separate candidate...

Jin-Hao Chen, Ben-Lei Cui, Rui-Jian Jia et al. · 2 citations
Preprint Oct 2026

Learning When to Trust and Reobserve Sparse-Frame VideoQA from Temporal Sampling Perturbations

Sparse-frame video-language models can change their answers when only the sampling timestamps shift. Raw agreement exposes this problem, but it need not rank errors better than token confidence or identify which additional observation will repair an answer. We argue that answer risk and action value must be estimated s...

Shu-Jia Liu · 0 citations
#artificial intelligence Preprint Oct 2026

IsoVRBench: Video Reward Modeling Without Human Feedback

IsoVRBench is proposed, a video generation reward modeling benchmark of 8,000 preference pairs across eight interventions, and it is shown that state-of-the-art models separate broad quality differences between generators but miss important features such as temporal order.

S. Li, Jacqueline He, Jia-Hui Chen et al. · 1 citation
Sep 2026

Frame Differential On-Policy Self-Distillation for Video Reasoning

Reinforcement learning (RL) has substantially improved the reasoning ability of multimodal language models through verifiable rewards and increasingly fine-grainedvisual or temporal credit assignment. In video reasoning, however, current RL methods typically train with a fixed sparse frame budget: increasing the number...

Haiying He, Xin Zheng, Shao-Li Hu et al. · 0 citations
Preprint Oct 2026

SAPO: Self-Supervised Attention Policy Optimization for Improving Temporal Video Grounding in Vision-Language Models

Video vision-language models (VLMs) have strong semantic understanding, yet remain unreliable at precise temporal grounding. Existing gains from explicitly aligning video frames with absolute time suggest that VLMs may already encode accurate temporal locations internally but lose precision when expressing them as time...

Zi-Xuan Chen, De-Peng Wang, Wei Chen et al. · 0 citations
#small language model Preprint Oct 2026

Behavior Pack Optimization for Video MLLM Post-Training

Video multimodal large language models (MLLMs) keep climbing video question answering benchmarks, yet shuffling the frames, masking the segment that supports the answer, or occluding the target object barely changes their predictions. The accuracy rests on appearance and language priors, not on the temporal evidence th...

Zhaolu Kang, Shi-Yu Liu, Tai-Long Luo et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.