Research on AI-Based Natural Interaction and Intelligent Control System for Robots

Abstract

Service robots require reliable intent recognition and physically safe control. This study proposes ANICS, a closed-loop system integrating multimodal perception, LLM-based planning, and adaptive robot control.Five sensory streams — speech, RGB-D vision, gesture, environment, and haptic feedback — were fused via four-head cross-attention with InfoNCE alignment. A retrieval-augmented LLaMA-2 converted intent representations into structured action plans. An uncertainty score selected among model predictive control, an attention policy, and soft actor-critic. Twenty-four participants completed 360 trials across pick-and-place, assembly, and handover tasks under fivefold cross-validation. ANICS achieved 94.2% intent accuracy (F1 = 0.93), 97.3% task success, and 95 ms end-to-end latency. The hybrid controller reduced position errors by 60.7–62.9% relative to standard MPC. Across 1,000 simulated executions, 98.7% collision avoidance and a 28 ms safety override response were recorded. These results confirm that cross-modal fusion, retrieval-grounded LLM planning, and uncertainty-driven controller selection effectively bridge the semantic-to-motor gap in human– robot interaction.