Mobile gaze tracking on handheld devices commonly relies on \emph{appearance-based 2D gaze estimation} from the device's \emph{front-facing RGB camera}. Although this approach makes eye tracking more accessible by avoiding dedicated infrared hardware, maintaining accuracy remains difficult because the spatial relations...
Yaxiong Lei, Yuheng Wang, Fergus Buchanan et al.· 0 citations
Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their visually rich worlds and diverse mechanics, remain challenging and offer a demanding next step toward general digital control. Current game-agent learning pipelines ofte...
Mingyu Ouyang, Yiran Wang, Siyuan Hu et al.· 0 citations
Interactive video generation commonly relies on small bidirectional chunks to reduce response latency. Yet our preliminary experiments show that larger chunks improve video quality and better utilize GPU parallelism, allowing more latents to be sampled with only a modest increase in sampling time. Motivated by these ob...
Yubo Huang, Yuxuan Zhang, Jiaming Liu et al.· 0 citations
We propose DeltaStreamer, a novel agentic framework for streaming video understanding inspired by how humans observe, remember, and investigate. Streaming video understanding requires a system to retain evidence before knowing which details future queries will need. Existing approaches often obscure event transitions a...
Boyuan Sun, Deshui Miao, Wenzhao Gao et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
It is found that external rewards drive most of the accuracy gains, while preserving more coherent reasoning when rewarded for reasoning tokens, while preserving more coherent reasoning when rewarded for reasoning tokens through internal attention rewards.
Video large language models (Video LLMs) often generate responses unsupported by the input video. Contrastive decoding (CD) can mitigate these hallucinations, but its effectiveness depends on how the negative branch weakens visual evidence. Existing methods rely on predefined perturbations or masking guided by question...
Zhi-Xi Cai, Ruo-Qi Yuan, Xin-Cheng Yang et al.· 0 citations
Multimodal in-context learning (ICL) is an important capability of multimodal large language models (MLLMs). Its performance is closely tied to the demonstrations it uses, making demonstration retrieval a core problem for ICL. Common methods use fixed scoring methods to retrieve all the demonstrations at once, which ma...
Yanshu Li, Rui-Xiang Tang, Junyi Jessy Li et al.· 0 citations
Embodied visual tracking requires a robot to identify a designated target from visual observations and control its own motion to follow that target through an environment. Following an isolated person is a comparatively simple demonstration of embodied visual tracking, but following the intended person among distractor...
Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitations in spatial reasoning and perception. Post-training techniques have shown promise for improving these capabilities; yet, there are limited data-driven surveys of such...
Mahdieh Mallahnezhad, Masoud Jafaripour, Mohamadali Tofigh et al.· 0 citations
The Distillation-to-Reinforcement Learning (D2RL) method is proposed to address the limited generalization of supervised fine-tuning and the sparse reward problem in reinforcement learning, and produces substantial gains over models trained on open-source datasets, with performance comparable or even superior to models...
Qing-Song Xie, Jia-Hao Tian, Jia-Cheng Tang et al.· 0 citations
SemCER is introduced, which keeps CER's alignment and denominator and charges each grapheme edit a severity from an LLM judge, with no word segmentation.
Interpreting an ambiguous utterance in real-world interactions often requires context beyond a single utterance. When the utterance refers to objects in a visual scene, it is necessary to maintain and update the set of plausible referents as context accumulates, without assuming that the available information identifie...
Shun Inadumi, Lee Sangmyeong, Koichiro Yoshino· 0 citations
Exploring how generative AI could make machine vision more accessible to businesses. The post GenEye in a Box: Making Machine Vision Something You Can Just Ask For appeared first on GPT-Lab.
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity. The post What AI gets wrong and what failure teaches us appeared first on Microsoft Research.
Computer scientist, entrepreneur, and philanthropist will collaborate with the MIT Schwarzman College of Computing to advance AI and scientific discovery.
