Multimodal language models (MLLMs) exhibit promising spatial reasoning capabilities, yet the internal mechanisms supporting them remain poorly understood. We uncover S-Space, a linear representation subspace in which object-token representations encode continuous coordinates along horizontal, vertical, and distance axe...
Zhi-Yuan Shi, Xiang Li, Xing-Zhuo Guo et al.· 0 citations
Modeling human reactions to spatial audio requires understanding both what is heard and where it comes from. Prior work generates motion offline from complete acoustic and spatial input sequences. Humans, however, react to sounds without knowing what they will hear next. We present Sonic-AMG, to our knowledge the first...
Pretrained vision transformers learn structured image representations spanning spatial patch tokens and non-spatial tokens. Recent analyses suggest that non-spatial register tokens encode global scene properties such as illumination, blur, and style. Yet patch-based image decoders discard these outputs, leaving the dec...
David Jacob Drexlin, Julius Hense, Jonas Dippel· 0 citations
On-policy distillation (OPD) reveals versatility and convenience during VLM post-training, providing dense supervision for student from teacher. However, we observed an anomaly that the distilled VLM could still answer the questions accurately even when the related visual regions were masked or replaced with noise. Bui...
Haowei Wang, Hao Lin, Qi Zhu et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
TAP-JEPA is introduced, a ViT-B/3D trained on 113,611 CT volumes that is competitive with state-of-the-art encoders in scan-level classification and shows that contextual predictability is not a sufficient proxy for clinical relevance in medical representation learning.
F. Drexel, B. Kainz, J. Kather et al.· 0 citations
Video generation supports digital content creation, with applications in virtual avatars, AI-driven filmmaking, interactive game engines, and world models. State-of-the-art systems achieve controllability primarily through image-to-video (I2V) generation, in which an injected anchor image determines the identity, compo...
Jinyuan Xu, Shitong Shao, Lichen Bai et al.· 0 citations
A controlled swap keeps the test, the labels and the metric fixed, change only the feature space, and judge each defense by retraining the victim on what it keeps, not by its detection score.
Federated vision-language models (FedVLMs) represent one of the emerging frontiers of machine learning, aiming to integrate federated learning within the fine-tuning pipelines of vision-language models (VLMs). In this work, we introduce BLEND, a new framework designed to jointly enhance the personalization and generali...
Payam Abdisarabshali, Fardis Nadimi, Kasra Borazjani et al.· 0 citations
Fewer latent tokens promise cheaper diffusion. Yet aggressive latent compression creates a reconstruction-generation dilemma: adding channels improves reconstruction but makes generation harder. We introduce APEX, short for Asynchronous Prefix denoising for EXtreme compression, which pairs a hierarchical latent space w...
David Dinkevich, Nisan Chiprut, Yoav HaCohen et al.· 0 citations
Stereo vision is increasingly used for robotic manipulation, with a growing body of work incorporating it into Vision-Language-Action (VLA) models. However, the scarcity of large-scale, real-world stereo datasets remains a fundamental bottleneck for scaling such policies. Recent generative monocular-to-stereo methods t...
Jason Chen, I-Chun Arthur Liu, Gaurav Sukhatme et al.· 0 citations
Zebra3D, a layerwise sparse-linear hybrid attention framework that improves the efficiency of pretrained 3D foundation models without training from scratch, achieves strong reconstruction and camera-pose accuracy at high attention sparsity, with training-free variants remaining close to their dense counterparts.
Utkarsh Saxena, Rui-Jing Yang, Mehdi Rezagholizadeh et al.· 0 citations
High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation across Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and Vision Mamba (ViM). Unlike predominantly qualitative studies, we quantify the alignment betwee...
Marco Stefani, Edoardo Parpaiola, Matteo Bino et al.· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.