Venus++: Reliable Large-Scale mmWave Radar Data Generation via Few 2D Videos for Lying Pose Gesture Recognition

Abstract

Millimeter-wave (mmWave) radar enables privacy-preserving gesture recognition but is limited by a lack of training data, particularly for lying postures. Existing mmWave radar data generation methods are ineffective due to the insufficient availability of 2D video data. To this end, we propose a novel system, <monospace>Venus++</monospace>, that generates realistic radar data for lying postures using few 2D videos, but it faces three key challenges: i) simulating diverse reflected signals, ii) overcoming limited real-world data leading to low data fidelity, and iii) reducing low-quality or redundant generated samples. <monospace>Venus++</monospace> consists of three main components: (i) a <italic>gesture sequence generation and signal simulation network</italic>, which combines several key modules, <italic>movement information extractor</italic>, <italic>spatio-temporal latent diffusion model</italic>, and <italic>mmWave signal simulator</italic>, to generate various gesture vertex sequences under certain conditions and simulate signal propagation characteristics for obtaining coarse radar data; (ii) a <italic>meta-learning domain adaptation network</italic> generates realistic radar data with few real-world data via “meta-learning” strategy; and (iii) a newly introduced <italic>quality assessment network</italic> that evaluates both fidelity and diversity of the generated samples, retaining only high-quality and diverse data for reliable downstream recognition. Extensive experiments on both generated and self-collected datasets demonstrate that <monospace>Venus++</monospace> significantly outperforms state-of-the-art methods in recognizing gestures performed in lying postures.