Skip to content
Preprint

BiMoGen: Bidirectional Motion-Text Generation via Unified Masked Discrete Diffusion

Sep 2026 · 0 citations · 66 references
Computer Science

Abstract

Text-to-motion generation and motion-to-text captioning are two fundamental tasks in human motion modeling, both grounded in the same underlying motion-text correspondence. Existing unified approaches mostly rely on autoregressive modeling, which imposes a fixed generation order and is therefore poorly suited to the bidirectional dependencies between language and motion, allowing early prediction errors to persist as fixed context and degrade both temporal coherence and cross-modal consistency. Masked discrete diffusion, which models sequences through iterative bidirectional prediction, offers a natural remedy. We therefore propose BiMoGen (Bidirectional Motion-text Generation), a unified masked discrete diffusion framework for bidirectional motion-text modeling. To stabilize training, we design Decoupled Uni- and Cross-Modal Training, in which masked pretraining first establishes cross-modal correspondence on paired motion-text sequences, after which supervised fine-tuning specializes the model for bidirectional generation. Masked diffusion nonetheless introduces its own source of error, as the model is trained on clean ground-truth context yet encounters self-generated and potentially erroneous context at inference, with errors committed under heavily masked states propagating through subsequent steps. We further introduce Generation-Aware Self-Correction that exposes the model to its own predictions during training and applies correction passes at early sampling steps to revise unreliably committed tokens. Extensive experiments on HumanML3D and KIT-ML demonstrate competitive performance on both tasks, validating the effectiveness of the proposed two-stage training and self-correction designs. The project page is available at https://wengwanjiang.github.io/BiMoGen-Page.

View source

Similar papers

Sep 2026

Timo: Taming Multimodal Diffusion Transformer for Human Motion Generation

Most existing human motion generation (HMG) methods use cross-attention modules to inject text semantics, but ignore the importance of bidirectional modeling between motion and text tokens, which limits text comprehension. A straightforward idea is introducing multimodal diffusion transformers (MMDiT), which have shown...

Zhao Wang, Jiang-Tao Hu, Jack Yu et al. · 0 citations
#diffusion models Book Open access Nov 2026

Dual-Granularity Text Guidance for Dyadic Motion Generation

Generating realistic dyadic motions from text is challenging due to complex inter-individual dynamics. Existing works lack in-depth exploration of input textual descriptions, leading to insufficient text-motion alignment. To address this problem, we propose DualGuidance, a diffusion model for dyadic motion generation t...

Xiang-Xing Liu, Lu Zhou, Hai-Xin Wang et al. · 0 citations
Book Open access Nov 2026

SegMoDiff: Semantic-Faithful Text-to-Motion Generation via Segmented Raw-Sequence Diffusion

Diffusion models have shown remarkable success in generating 3D human motion from text descriptions, demonstrating superior realism and diversity. However, directly modeling the full raw motion sequence is computationally intractable. Existing diffusion-based methods either compromise essential computations on the raw...

Jia-Xin Wang, Shuo-Yan Wei, Feng Li et al. · 0 citations
Book Open access Nov 2026

COME: Advancing Motion Representation and Generative Modeling for High-Quality Text-to-Motion

Text-to-Motion generation aims to synthesize 3D human motion from natural language. While diffusion models naturally capture motion continuity, they often underperform discrete token-based approaches. Emerging applications such as motion editing and multimodal control, however, require fine-grained control, composition...

Guangtao Lyu, Xin-Yi Cheng, Qi Liu et al. · 0 citations
Book Open access Nov 2026

TextZGCA: Text-Guided Zero-Gated Cross-Attention for 3D Human Motion Prediction

3D human motion prediction (HMP) forecasts future joint trajectories from a short observed pose sequence, but pose alone is often a narrow description of human behavior. A person who is turning, sitting down, or preparing to reach can have similar short-term pose dynamics while implying different futures. Modern vision...

Guan-Hui Qiao, Lu Zhou, Ding Jiang et al. · 0 citations
Preprint Mar 2025

Linear Multi-granular Body Modeling for Text-Driven Motion Generation

Text-to-motion generation is a rapidly growing field at the nexus of multimodal learning and computer graphics, enabling flexible and cost-effective applications in gaming, animation, robotics, and virtual reality. Existing part-aware pipelines typically split the body into segments and then recover whole-body coordina...

Xing-Zu Zhan, Gen-Pei Zhang, Chen-Xi Xie et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.