Unifying Convolution and Transformer for Efficient Concealed Object Detection in Passive Millimeter-Wave Images

Passive millimeter-wave (PMMW) imaging is an ideal technique for concealed object detection in non-contact security inspection scenarios, and has been widely used in railway stations and airports. However, there are still several challenges that limit the accuracy of detection in PMMW images: low resolution, small objects and complex background interference. The existing deep learning-based methods mainly adopt two-stage architecture with convolutional neural networks (CNN) as the backbone for feature extraction, while the low speed of two-stage architecture and limited receptive field of CNN impede the further improvement of the intelligent inspection system. In this paper, we propose a one-stage anchor-free detector that combines the merits of CNN and transformer to solve these problems, and we avoid the tradeoff between accuracy and computational complexity that most hierarchical transformers make by constricting self-attention within pre-defined local windows. Specifically, we design a novel backbone to model low-level local and high-level global features at different scales via CNN and transformer. We also introduce object queries to guide the detector to perceive the concealed objects from the background noise, and these learnable queries are further utilized to form a full-size object-aware attention mechanism. Besides, to select the optimal positive samples for the anchor-free detector, we propose a novel label assignment strategy by employing Gaussian distribution to adaptively model the objects with various shapes. Experimental results on our self-developed PMMW imager demonstrate that the proposed method outperforms the state-of-the-art methods in terms of accuracy and speed.