Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision-language models (VLMs) can overlook sparse evidence and struggle with cross-modal reasoning in a single...
Zhuo-Wen Liang, Xiao-Tian Lin, Yu-Yu Luo et al.· 0 citations
Code world models represent world dynamics through explicit state and executable rules. We investigate how to coordinate object representations, behavioral revisions, and visual memory within this formulation. We present Mallea, a system for building, steering, and memorizing worlds from text. A coding agent infers obj...
Large multimodal models have achieved remarkable progress by scaling model capacity and training on massive paired multimodal data. However, this paradigm introduces a fundamental bottleneck: aligned multimodal datasets are expensive to curate, difficult to scale across diverse modality combinations, and increasingly c...
R. Moseley, Md Kaykobad Reza, Ameya Patil et al.· 0 citations
Text-to-video models often generate the visual concepts requested in natural language prompts but fail to satisfy their intended ordering, timing, and persistence requirements. We propose representing these requirements as specifications in a formal Temporal Logic, and introduce Temporal Specification Guidance (TSG), a...
Avishree Khare, Hideki Okamoto, Bardh Hoxha et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Video deep research aims to answer questions by jointly understanding video content and retrieving external knowledge from the open Web. However, these tasks often require diverse and multi-step tool-use trajectories, which may lead to unnecessary video tool calls, incurring additional costs and failure risk. In this p...
Xintong Zhang, Xiaomeng Fan, Shilin Yan et al.· 0 citations
Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents, with an emerging strategy of scaling test-time compute to improve capability. However, we observe that doing so increases latency, token usage, and FLOPs while yielding uneven, often diminishing gains in downstream succe...
Jadelynn Dao, Milan Ganai, Yasmina Abukhadra et al.· 0 citations
Vision-language-action (VLA) models with flow-matching action heads, such as {\pi}0 and {\pi}0.5, have become a standard recipe for generalist robot manipulation, but every action chunk is produced by integrating a learned velocity field over about ten sequential action-expert evaluations, so action generation dominate...
Wuyang Luan, Junhui Li, Weiguang Zhao et al.· 0 citations
Zero-shot object navigation (ZSON) requires robots to find target objects in unseen environments without task-specific fine-tuning or pre-built maps, a key capability for general-purpose service robots. Yet methods that perform well in simulation often degrade in cluttered real-world scenes with severe occlusion and la...
Diffusion models show promise for 3D molecular generation, but face a fundamental trade-off between sampling efficiency and conformational accuracy. While flow-based models are fast, they often produce geometrically inaccurate structures, as they have difficulty capturing the multimodal distributions of molecular confo...
We propose a novel cognitively-inspired prompting method to stabilize and interpret the physical simulation produced by autoregressive image generation models, called Chain-of-Time. Chain-of-Time involves generating a series of intermediate images during a simulation, and it is motivated by mental simulation in humans,...
YingQiao Wang, Eric Bigelow, Boyi Li et al.· 0 citations
A compute-aware Bengali captioning pipeline trained on LaBSE-verified EN-BN pairs and 110k bilingual-prompted synthetic images, with a core novelty that tightens real-synthetic patch alignment, reduces spurious matches, and drives strong gains on Flickr.
Riad Ahmed Anonto, Sardar Md. Saffat Zabin, M. S. Rahman· arXiv.org· 2 citations
Depth Anything has achieved remarkable success in monocular depth estimation with strong generalization ability. However, it suffers from temporal inconsistency in videos, hindering its practical applications. Various methods have been proposed to alleviate this issue by leveraging video generation models or introducin...
Sili Chen, Hengkai Guo, Shengnan Zhu et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.
