LAKF-Based 3D Multi-Object Tracking: Hybrid KalmanNet for Nonlinear Motion Estimation

Abstract

Reliable 3D multi-object tracking (MOT) is essential for autonomous driving. The model-based Kalman filter (KF) is the standard motion estimator in 3D MOT, but its linear-dynamics assumption and constant noise covariances limit its adaptability to nonlinear motion and time-varying noise. The learning-aided Kalman filter (LAKF) addresses these issues by learning the Kalman gain (KG) through recurrent neural networks. We adopt CNPE-KalmanNet, a variant that integrates complex-number position encoding to capture semantic correlations among heterogeneous state components. However, the analytical KG is derived under the assumption that the process and measurement noise covariance matrices are positive definite, whereas the learned KG from CNPE-KalmanNet lacks structural constraints. The unconstrained learned KG tends to produce a dense matrix, leading to cross-dimensional coupling, particularly across different scales. This paper proposes a hybrid KG that fuses the model-based analytical KG with the data-driven learned KG. Compared to the AB3DMOT baseline on KITTI and the P3DMOT baseline on nuScenes, our approach improves average recall by 13% and 22%, and IoU by 6% and 7% in state estimation, and AMOTA by 0.7% and 0.5% for overall tracking accuracy. The hybrid design retains the numerical stability of analytical filtering while incorporating the strong nonlinear fitting capacity from deep learning.