BGM2Pose is proposed, a non-invasive 3D human pose estimation method using pre-selected everyday music as active sensing signals and a Frequency-wise Attention Module that enables the model to focus on subtle acoustic variations attributable to human movement by dynamically computing attention across frequency bands.
Yuto Shibata, Yusuke Oumi, Go Irie et al.· IEEE Open Journal of Signal...· 1 citation
The event-based Vision-Language Model (VLM) recently has made good progress for practical vision tasks. However, most of these works just utilize CLIP for focusing on traditional perception tasks, which obstruct model understanding explicitly the sufficient semantics and context from event streams. To address the defic...
Pengteng Li, Yunfan Lu, Pinghao Song et al.· 0 citations
SportD opens a new direction for rigorously evaluating physical strategic decision-making in VLMs, showing that careful decomposition of their choices can reveal the mechanisms underlying systematic biases such as risk aversion.
Jasin Cekinmez, Addison J. Wu, Hao-Tian Xia et al.· 0 citations
Echocardiography interpretation requires integrating multi-view temporal evidence with quantitative measurements and guideline-grounded reasoning, yet existing foundation-model pipelines largely solve isolated subtasks and fail when tool outputs are noisy or values fall near clinical cutoffs. We propose Echo-CoPilot, a...
Moein Heidari, Ali Mehrabian, M. A. Roohi et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Deep networks trained on structural MRI for Alzheimer's disease (AD) staging often reach reasonable accuracy while attending to anatomically irrelevant regions, and multimodal models that add clinical tables frequently rely on variables that were used to assign the diagnostic label in the first place. We study both iss...
Bench2Dex is meant as a platform for studying visuo-tactile learning across dexterous hands and offers a shared setting for algorithm development while tactile hardware and simulation models are still evolving.
Zhen-Jie Yang, Yi-Deng Zhang, Dong-Jie Zhang et al.· 1 citation
Visual Question Answering (VQA) with Vision-Language Models (VLMs) is increasingly used in privacy-sensitive and bandwidth-constrained settings. Federated Learning (FL), Split Learning (SL), and U-Shaped Split Learning (USL) keep raw data local, but transmitting all visual tokens across a model partition remains costly...
Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures often exhibit substantially different image intensities across modalities. In this work, we present a comprehensive benchmark of intra-patient 3D multimodal deformable regi...
Matteo Barbieri, G. La Barbera, J. De la Plata et al.· 0 citations
Through pre-training on extensive text and image datasets, current multi-modal large language models (MLLMs) achieve strong performance on general tasks. However, circuit schematics present a unique challenge for MLLMs due to their dense component layouts and distinct topological logic, demanding fine-grained structura...
Jinyuan Deng, Yuqi Jiang, Wenjing Huang et al.· 0 citations
KaiNinja, a part-level extension of TRELLIS.2 built on a dual-volume form of its O-Voxel representation, is the first 3D generative model trained on agent-authored part data and finds that whole-object fidelity improves over the same backbone fine-tuned on the same dataset.
Rui-Han Yu, Lian Fu, Mu-Yao Niu et al.· 0 citations
ModaLens, a paired image-swap audit, measures how report availability changes image sensitivity and limits conclusions about visual correctness; the direction replicates in two further model lineages.
S. Ordóñez, Maximin Lange, Quang Bui et al.· 1 citation
The VANGUARD team proposes a human-centered multimodal framework that analyses and incorporates the psychological and demographic characteristics of human annotators into the detection pipeline and releases the full pipeline and analysis to support reproducible human-centered modeling.
A. Mocanu, Sebastian Mocanu, Ciprian-Octavian Truica et al.· 0 citations
Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.