Multi-modal Vehicle-Infrastructure Collaborative Perception via Deformable Attention Mechanism

Multimodal fusion perception techniques have advanced significantly in recent years. However, relying exclusively on ground-based vehicle sensors constrains the perception system to sparse data acquired within the limited field-of-view of terrestrial vehicles, consequently hindering the acquisition of information from alternative perspectives and beyond line-of-sight ranges. This paper presents a novel framework designed to address these issues. We first propose a deformable attention mechanism to adaptively refine feature representations at the local scale. Subsequently, we introduce a context-aware dynamic fusion (CADF) module that dynamically adjusts fusion weights according to environmental context, significantly enhancing both accuracy and robustness. Finally, we present a global-local weighted fusion (GLWF) approach that integrates global channel weighting with local spatial self-attention, effectively optimizing the fusion of perception data from infrastructure and ground vehicle. Experimental results demonstrate that the proposed framework substantially reduces latency, enhances fusion precision, and significantly improves overall perception performance under complex real-world scenarios.