Creating diverse and realistic human motions is a fundamental cornerstone of computer animation, with numerous applications in games, movies, and AR/VR. While motion capture is a valuable tool for capturing motions across varied body sizes, obtaining unique motion data for a variety of characters is often prohibitively expensive. Motion retargeting addresses this limitation by adapting existing motions to different character morphologies, however, existing approaches often involve trade-offs between motion realism, user control, and adaptability to artistic needs. In this work, we propose Directable Motion Paraphrasing (DMP), a novel motion retargeting framework based on the concept of motion paraphrasing, analogous to text paraphrasing, where the core semantics of a motion are preserved while allowing expressive, user-directed variations. Our framework constructs a large-scale motion paraphrasing dataset, which captures the diversity of human motion across different body shapes, and trains a diffusion-based generative model that learns both invariances and variations in motion. To enable user control during inference, we introduce a flexible mechanism for specifying spatio-temporal constraints, such as joint positions, rotations, and object interactions, which can be incorporated into the generative process through masked inpainting and loss guidance. We demonstrate the effectiveness of our framework through various examples, showing its ability to produce realistic, diverse, and controllable retargeted motions that meet the artistic demands of animation pipelines. Extensive experiments demonstrate the system’s flexibility, motion plausibility, and directability, highlighting its potential as a tool for intuitive and high-quality motion retargeting.
Sunmin Lee, Davis Rempe, Yifeng Jiang et al.· ACM Transactions on Graphics· 0 citations
Generating realistic 3D human motions in real-time within interactive applications is key for animation, simulation, and humanoid robotics. While recent offline motion generation approaches offer precise control via text and kinematic constraints, they lack the inference speed required for interactive settings. Conversely, existing online methods enable real-time synthesis but often sacrifice controllability or struggle with complex text semantics and long-horizon goals due to limited context windows. In this work, we introduce ARDY, a streaming generation framework that bridges this gap by enabling high-fidelity motion generation controllable via online text prompts and flexible kinematic constraints. ARDY employs a hybrid representation that combines explicit root features with a latent body embedding, balancing precise trajectory control with efficient generative learning. We propose a two-stage autoregressive transformer denoiser that features variable history context and supports conditioning on flexible, long-horizon kinematic constraints. By training on a large-scale motion capture dataset and being directly conditioned on text labels and kinematic constraints sampled from ground truth poses, ARDY natively learns controllable generation that supports online prompting and flexible long-horizon goals. Extensive evaluations on the HumanML3D benchmark and the large-scale, high-fidelity Bones Rigplay dataset demonstrate ARDY's high motion quality and constraint adherence, validating the efficacy of our key architectural decisions. Finally, we demonstrate the method's practical versatility through an interactive demo featuring dynamic text control, diverse keyframe pose constraints, path following, and interactive locomotion control via mouse and keyboard. Supplementary video results, code, and model releases can be found at https://research.nvidia.com/labs/sil/projects/ardy/.
Kaifeng Zhao, Mathis Petrovich, Haotian Zhang et al.· ACM Transactions on Graphics· 1 citation