We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises large-scale, high-quality datasets of executable 3D code, a coding harness for spatial reasoning and automated verification, and practical guidance for 3D coding. First,...
Yipeng Gao, Ziyao Zeng, Jinfa Huang et al.· 0 citations
Image editing models produce impressive results for individual edits, but their quality often deteriorates over multiple editing turns. Each editing turn introduces small distribution shifts that accumulate over successive edits, creating a gap between the input distributions encountered during the editor's original tr...
Sara Dorfman, Nir Goren, Gal Barak et al.· 0 citations
An event-centric evaluation framework that disentangles audio and visual events at the event level, analyzes each event independently, cross-validates relations across modalities, and aggregates the judgments into a coarse-to-fine, three-level consistency metric, providing a general formulation for audio-video consiste...
Yu-Ruo Wu, Kai-Hang Pan, Kai Shen et al.· 0 citations
Real desktop automation is neither purely visual nor purely programmatic. Agents must understand user-facing interfaces while efficiently manipulating files, scripts, and system state. However, native GUI/code coordination cannot be obtained by prompt-level tool composition. In our preliminary experiments, naively addi...
Bowen Yang, K. Jin, Zichen Ding et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Across eight multimodal benchmarks, SCOPED consistently outperforms the backbone and mixed-capability RL, demonstrating the effectiveness of skill-decoupled on-policy expert distillation for integrating independently acquired capabilities.
Yu-Qi Xiong, Chun-Yi Peng, Zhi-Peng Xu et al.· 0 citations
TRACE, a topology-aware, prediction-adaptive objective that uses a target skeleton to define required local connections, and introduces a topology-aware skeleton carrier that preserves both foreground and class-specific topology, enabling reliable definition of these connections.
This work introduces a motion- and gravity-aware framework under the Learning Using Privileged Information (LUPI) paradigm, and suggests that LUPI offers a practical way to inject motion and physical priors into feed-forward 3D reconstruction without increasing test-time complexity.
Streaming vision-language models (VLMs) must continuously integrate information from the past while remaining responsive to current observations and user queries. Existing memory mechanisms typically compress, retrieve, or cache historical observations and reintroduce them into the input context, forcing historical mem...
Zhuoming Liu, Jinhong Lin, Lin Zhang et al.· 0 citations
As multimodal AI systems are becoming our everyday thought partners, the shift towards studying how users transfer control to these systems becomes especially urgent. In this paper, we report an overview of findings from an online study with n=56 AI-literate participants and 392 Video Question Answering trials. After i...
Cross-object alignment among geometrically similar interaction regions during training enables structures on unseen objects that resemble previously observed ones to produce representations similar to those learned during training, facilitating target-region recognition.
Embodied spatial intelligence requires more than visual recognition: models must localize entities, reason about their relations, and express this knowledge in action-relevant forms. Yet joint training can conflate the knowledge needed at different stages because supervision is fragmented across incompatible geometric...
Wei-Lin Ruan, Bo-Han Zhou, Jing-Hao Li et al.· 0 citations
We propose DINO-FIL, a transformer-based framework for pixel-level localization of generative image forgeries. The method leverages multi-layer representations from a pretrained DINOv3 ViT-H+/16 encoder combined with a lightweight fusion decoder to capture both semantic structure and fine-grained texture inconsistencie...
Simon Freitter, David Fischinger, Silvia Poletti et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.