PEARL: A Lightweight Prompt-Based Feature Interpreter Framework for Real-Time, Anonymous, and Heterogeneous Collaborative Perception

Abstract

Heterogeneity across Collaborative Perception (CP) agents is a major challenge for emerging CP frameworks due to domain gaps from differing sensors, architectures, and training data. Prior works mitigate this challenge by aligning features in a unified space via model retraining or per-agent-type interpreters. These strategies (a) require access to neighbor configurations, (b) have not fully addressed realistic real-time CP cases, and (c) generalize poorly to unseen agents joining at run time. To overcome these challenges, we present PEARL, a Prompt-Embedding framework for Anonymous and Real-time Lightweight heterogeneous CP. PEARL supports multiple CP interpreters, one of which can be optimally selected for a new-joining agent in real-time by aligning intermediate BEV features using two lightweight, multi-scale interpreters trained in parallel, a sparse-detection (LWSD) interpreter that aligns salient regions for cooperative detection, and a dense, domain-invariant (LWDDI) interpreter that produces agent-invariant features for fast interpreter selection in real-time. To keep computation and storage low, both interpreters use low-rank visual prompts, significantly reducing model complexity. Experiments on OPV2V dataset show that PEARL's real-time model selection strategy yields 7% AP@0.5 and 13% AP@0.7 gain over a random-selection baseline, running in 1.67 ms on average, enabling viable real-time CP deployment. Although primarily designed for real-time CP, PEARL also outperforms state-of-theart heterogeneous CP under traditional offline training by 2% AP@0.5 and 3% AP@0.7 while reducing computation and preserving privacy. Code is available at https://github.com/arminmaleki007/PEARL.