KPV3D: Enhancing Multiview 3-D Vehicle Detection With 2-D Keypoint Priors

Three-dimensional detection from multiview images has advanced significantly with its cost-effective deployment in autonomous driving. The present approaches construct 3-D representations from multiview images or employ object queries distributed in 3-D space to achieve object localization, but relatively few methods incorporate more explicit image priors as guidance. Hence, there is still untapped potential for discovering significant information from more explicit and richer 2-D clues. Focusing on important perception categories in autonomous driving, we aim to improve vehicle detection performance and propose KPV3D, which leverages 2-D bounding box and vehicle keypoint information to construct spatial feature correlation. Based on high-quality 2-D object and vehicle keypoint priors, KPV3D generates directional and pivotal semantic information for adaptive 3-D queries and uses a point-global feature aggregation module for effective feature capture. In addition, a reprojection loss is proposed to further incorporate keypoint priors for refining 3-D prediction results. Experiments on public dataset nuScenes are conducted to verify the effectiveness of the proposed method.