AI Networking Cookbook: Practical recipes for AI-assisted network automation and development
TL;DR
A lightweight frame selection method to identify keyframes and train it via a two-stage strategy that improves performance of diverse down-stream MLLMs across benchmarks spanning medium to long videos.
CSES is a training-free semantic keyframe selector that adaptively determines the numbers of frames to score and keyframes to select that formulates keyframe selection as a coverage problem that jointly accounts for semantic relevance, temporal redundancy, and visual redundancy.
MarKey is a training-free framework that formulates keyframe selection as subset-aware greedy optimization and consistently outperforms existing methods on six benchmarks spanning holistic video understanding, human-centric video understanding, and open-ended video understanding.
The sample efficiency and scalability of RL post-training for video MLLMs and introduces OraRL, a decoupled advantage estimator that scales with model size and data, surpassing its backbone from 0.8B to 9B and GRPO up to 100k prompts.
EcoFrame is proposed, a training-free framework for low-overhead query-adaptive visual evidence scheduling that leverages the VLM's inference feedback to determine when to increase the frame budget and where to search for additional candidate evidence.
VideoRouter (VR) is proposed that rethinks long-video understanding as coordinating complementary evidence views rather than selecting a single subset of frames, and introduces a verification-guided router to determine which view is better supported by the selected evidence and select the final answer.
Experimental results demonstrate that S2V can consistently improve LVU performance across multiple LVU benchmarks, outperforming both general MLLMs and reasoning-based methods not only in LVU accuracy but also in training and inference efficiency.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.