Peng-Fei Zhang

The Platonic brain bridge hypothesis: human brain networks as an architectural prior for omni models

Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models

Audio-visual large language models (AVLLMs) have made remarkable progress in multimodal understanding and reasoning through interactions among visual, auditory, and linguistic information. However, recent studies show that AVLLMs face a critical challenge: $\textbf{source-confused grounding hallucination}$, where cues...

SEmoEdit: Probing and Harnessing the Editability of Pre-trained Speech Flows

SEmoEdit is the first training-free framework that formulates emotion editing as dynamic velocity transport between source and target emotions, enabling robust, flow-based speech emotion editing directly within pretrained TTS models, and is highly effective and broadly applicable.

Listen, Do Not Copy: Internalizing Audio-Grounded Scaffold Context for Robust Omni-Model Speech Understanding

A joint GDPO task in which the model learns when to use a clue and how to produce a speaker-attributed transcript from separately normalized format, gate, and transcript rewards is formulated.

The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models

The Platonic brain bridge hypothesis is proposed: omni models, multimodal large language models that process video, audio and text jointly, converge on brain-like representations usable in both directions, and human brain organization is a usable architectural prior for multimodal large language models.

DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation

We present \textbf{DreamX-Phi 1.0}, an action-conditioned video world model for robotic manipulation that, given an observed frame, a language instruction, and a prescribed action sequence comprising end-effector poses and gripper states, predicts the resulting future observations. Yet realism alone does not guarantee...

Evaluation-Verification Reward for Consistent Multi-Reference Image Editing

A Multi-dimensional Evaluation-Verification Reward (EVR) decomposes evaluation into distinct visual criteria; for each criterion, an MLLM Evaluator generates multiple candidate hypotheses, and a Verifier grounds each claim in concrete visual evidence to accept or reject it, producing reliable and fine-grained reward si...