Intelligent Martial Arts Coaching Framework Using Artificial Intelligence for Real-Time Action Detection and Performance Feedback

Abstract

Karate training relies on subjective manual evaluation of intricate motions, hindering scalable, consistent coaching. This paper proposes MMAF-Net, a multimodal AI framework for real-time karate action recognition and automated performance feedback. Its three-branch deep learning architecture integrates visual, pose-estimation and inertial sensor streams to extract complementary motion features, fused via a temporal attention module to classify 23 karate action types accurately. Trained and validated on MS-KARD (2.8 million video frames and 5.6 million sensor readings from dual cameras and three IMUs), the model generates explainable coaching tips through rule-based modules referencing prediction confidence, posture bias and motion stability. Tests yield 96.3% accuracy and 95.1% F1-score, outperforming benchmarks like KarateNet. With only 24 ms inference latency, this real-time system suits interactive martial arts training scenarios.