Design and Implementation of Virtual Assembly System for Machine Tool Fixtures Based on Kinect and Unity3D

Abstract

Traditional machine-tool fixture assembly training relies on heavy physical equipment, entails substantial purchase and maintenance costs, and limits repeated practice because of safety and space constraints. This study develops a low-cost virtual assembly system by integrating Kinect V2 somatosensory interaction with Unity3D. Kinect captures 25 human skeletal joints in real time, while Unity3D provides model rendering, physical simulation, and interactive scene management. A constrained dynamic time warping method is introduced to reduce the gesture-matching search region and reject invalid actions through a distortion threshold. Ray casting, collision detection, and multi-coordinate transformation are combined to support part selection, movement, rotation, alignment, assembly, and viewpoint control. Fixture models are preprocessed in SolidWorks and 3ds Max, assigned rigid-body and collider properties, and rendered with differentiated metallic materials. Experimental evaluation shows that the improved recognition method achieves 92.5% accuracy with a response delay below 78.2 ms, while increasing computational efficiency by more than 40% relative to conventional DTW. Compared with traditional physical training, the system increases assembly efficiency by 32.5% and reduces the operation-error rate by 46.8%. The results demonstrate the feasibility of using natural gestures for repeatable fixture-assembly teaching and provide a practical digital platform for engineering education, design verification, and pre-job training.