Video* Bench: Benchmarking Fine-grained Video Perception with "Think with Videos"

Abstract

Interactive agents need fine-grained long-video understanding. Current Video-LLMs remain brittle when relevant evidence is subtle or sparse. Fixed frame budgets force rich streams into a few sampled frames and often discard decisive cues. We introduce Video* Bench, a benchmark with 2,212 real-world videos and ten tasks spanning content, semantic, and spatio-temporal perception. Its low Detail Index emphasizes sparse, localized evidence. We evaluate 33 systems under multiple-choice and open-ended settings. Models recognize appearance and text better but remain weak at temporal localization, duration, and counting; performance also falls as the required detail becomes finer. We further evaluate ''Think-with-Videos'' (TwV) agents that resample frames and perform multi-turn reasoning at fixed budgets. They improve fine-grained and temporal perception without more visual tokens and remain robust when frame budgets shrink. The benchmark data and annotations are publicly available on ModelScope: https://modelscope.cn/datasets/RuizheChen/VideoStarBench.