Skip to content
Book Open access

DispTrack: Disparity-Aware RGB-IR Multi-Object Tracking via Association-Level Fusion

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 6 references

Abstract

RGB-IR multi-object tracking (MOT) typically assumes spatially aligned visible and thermal observations. In practice, distinct imaging geometries, imperfect calibration, and synchronization errors introduce cross-modal misalignment. Using M3OT as a testbed, we show that under large and temporally unstable disparity, conventional feature-level fusion, decision-level fusion, and tracker-side score modulation strategies fail to yield consistent tracking gains; detector-side interaction does not reliably improve temporal tracking and can degrade it instead. To address this, we propose DispTrack (Disparity-aware Tracking), which moves cross-modal interaction from spatial detection to temporal association. DispTrack employs a Dual-Decoder RGB-IR Detector, a Relative Position Matcher (RPMatcher) for cross-modal correspondence under spatial offset, and a two-stage TID-assisted association strategy for online association refinement and residual offline consolidation. On M3OT, DispTrack improves the standard IR-only references and remains beneficial under tracker-matched comparisons. These results support a narrower, empirical, and regime-specific conclusion: on M3OT-like large and temporally unstable disparity settings, association-stage integration is more reliable than the detector-side fusion interfaces studied here.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.