Skip to content
Preprint

VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes

Sep 2026 · 0 citations · 122 references
Computer Science

TL;DR

VidaForge is presented, an open research infrastructure that represents a video data recipe as an executable five-stage workflow from raw videos to training datasets, and demonstrates how VidaForge connects data-recipe choices to downstream model performance.

Abstract

Video foundation models increasingly rely on large-scale pretraining data, yet the end-to-end data pipelines behind them remain largely closed and difficult to inspect or reuse. Researchers seeking to understand how video data recipes affect model pretraining often need to build substantial infrastructure before testing even a focused hypothesis. We present VIDAFORGE, an open research infrastructure that represents a video data recipe as an executable five-stage workflow from raw videos to training datasets. A decision in this workflow can be varied to construct alternative datasets while preserving how every sample was produced. To demon strate this research workflow, we compare data recipes with different coverage and quality in early from-scratch pretraining of Wan 2.1 and V-JEPA 2.1. Across both learning objectives, the broader-coverage recipe achieves the highest downstream benchmark scores, while loss-based evaluation favors different recipes. This study demonstrates how VidaForge connects data-recipe choices to downstream model performance. We further release VIDAFORGE-3M, containing 3.14 million scene level clips totaling 6,475 hours, with fine-grained annotations and curation signals for video data-recipe research.

View source

Similar papers

Preprint Sep 2026

SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models

SolarWM, a fully open foundation for building interactive video world models from data preparation through long-horizon inference, is introduced, providing a reproducible and extensible foundation for interactive world-model research.

Jun-Chao Huang, Guian Fang, Sheng-Ju Qian et al. · 12 citations
Preprint Sep 2026

LongLive-Plug: Once-for-All Distillation for Video Generation

Video diffusion models are increasingly developed into specialized models for diverse downstream tasks, and this development often includes a distillation stage, for example to accelerate sampling or to improve long-video generation. This stage is typically repeated for every specialized model. We introduce LongLive-Pl...

Shuai Yang, Luo-Zhou Wang, Wei Huang et al. · 0 citations

Advancing Video-Text Pretraining with Multi-View Captions

This work proposes a large-scale multimodal large language model-based supervision generation framework that improves supervision diversity, fidelity, and semantic coverage, and introduces a granularity-aware text representation with separate CLS tokens for summary and detailed views.

Fida Mohammad Thoker, Renaud Vandeghen, Karen Sánchez et al. · 0 citations
Preprint Sep 2026

VI-Bench: Benchmarking Prompt Inversion from AIGC Videos

VI-Bench shows that video prompt inversion is a distinct and under-evaluated capability requiring models to transform visual understanding into replay-stable generative control, and evaluates 18 representative VLMs on this benchmark.

Wu-Lin Xie, Rui Zhao, Ke-Cen Li et al. · 0 citations

UvA-DARE (Digital Academic Repository) Elastic ViTs from Pretrained Models without Retraining

SnapViT: single-shot network approximation for pruned Vision Transformers is introduced, a new post-pretraining structured pruning method that enables elastic inference across a continuum of compute budgets, and a self-supervised importance scoring mechanism that maintains strong performance without requiring retrainin...

Walter Simoncini, Michael Dorkenwald, Tijmen Blankevoort et al. · 0 citations
Preprint Sep 2026

I Have a Stream: Making Self-Supervised Learning Work on Continuous Video

Self-supervised learning draws inspiration from infant visual development, yet standard training pipelines bear little resemblance to it: images are independently sampled and globally shuffled across epochs. We study self-supervised learning from continuous video streams, where frames are consumed in temporal order usi...

Ivan Martinovic, Lukas Knobel, Yuki M. Asano · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.