The data pipeline, architecture, training methodology, and runtime system behind Waypoint 1.5, a real-time diffusion world model for interactive video generation on consumer-grade hardware, are described and the safety and ethics considerations unique to iWMs are discussed.
Abstract
We present Waypoint 1.5, a real-time diffusion world model for interactive video generation on consumer-grade hardware. Unlike general video diffusion models, interactive world models (iWMs) must respond to dense user controls under strict latency and throughput constraints. Waypoint 1.5 is pre-trained on 100,000 hours of diverse, control-aligned video game data across hundreds of games, and generates playable video conditioned on full keyboard and mouse input. The model includes two resolution variants that run across a wide spectrum of consumer hardware. To characterize this unique setting, we distinguish rendered FPS, latent FPS, and control rate. We describe the data pipeline, architecture, training methodology, and runtime system behind Waypoint 1.5. We evaluate interactivity through latency and throughput. Finally, we discuss the safety and ethics considerations unique to iWMs.
Talking-head and dyadic models now achieve real-time inference, yet fast motion generation alone does not produce an interactive conversation. A live system must synchronize the model's output with speech from an external voice agent, schedule video frames for playback, and handle interruptions. We introduce AVTR-1, an...
A. Kravtsov, Dmitrii Ziganshin, V. Poletaev et al.· 0 citations
Astronex-World 1.0 is presented, an open controllable video world-model foundation that provides a bidirectional model for full-context generation and a causal model with block-causal attention and cross-block KV caching for persistent generation, both built on the Wan2.2-TI2V-5B prior.
Streaming Video Large Language Models (VLMs) enable continuous video understanding, yet existing benchmarks focus on low-dynamic scenarios. Under bounded context budgets, models must balance temporal history, spatial resolution, and temporal granularity; sparse sampling at 1--2 FPS misses fast events. We introduce Fast...
Yuxuan Hu, Wei-Kang Shi, Yang Bo et al.· 0 citations
Hanni, a networked testbed for real-time human-GenAI interaction, is presented and the usability of Huni is demonstrated through two case studies: comparing call setup time across different WebRTC and QUIC-based schemes, and evaluating the performance of multiple congestion control algorithms under diverse network cond...
Rui-Zhi Cheng, Mohamed Mohsin Shaik, Guo-Wu Xie et al.· Proceedings of the 4th ACM W...· 1 citation
MotionBlind, a contrastive benchmark of self-recorded video for physically grounded motion, the variables a world model must predict, is introduced, a pair of near-identical clips that differ only in motion.
Dhairya Bhatia, Bishoy M. Galoaa, Oliver Fritsche et al.· 0 citations
Vision-language models are extending video understanding from offline clip analysis to continuous interactive streaming, but most research still emphasizes model capability rather than deployable low-latency interaction. This paper presents a unified edge-cloud system for real-time video VLM applications. Lightweight p...
Pu-Nan Dai, Jun Xu, Bing-Cong Lu et al.· 0 citations
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.
