MSTCNet: Toward Generalization Improving for Multiframe Infrared Small Target Detection

Multiframe infrared small target detection plays an important role in various fields, especially in remote sensing. In continuous-frame infrared small target videos, factors such as the background change with the movement of the target. These changes lead to differences between the data distribution in actual application scenarios and the training scenarios. Existing deep learning methods are mostly designed for fixed scenarios. When facing scenarios with complex backgrounds and diverse changes, the generalization performance of the model is insufficient, leading to a decrease in detection accuracy and an increase in false alarms rate. To solve the problems mentioned above, combining the concept of domain generalization (DG) in transfer learning, we propose a multiscale spatio-temporal feature combined network (MSTCNet). First, we utilize the advantages of convolutional neural networks and recurrent neural networks, integrating them to build a high-performance structure. In addition, to further enhance generalization performance, we designed a selective physical information fusion (SPIF) module based on domain-invariant representation learning. This module enhances domain-invariant infrared small target features and reduces the impact of other irrelevant interferences. By integrating wavelet transform within the neural network, along with spatial attention and contrastive learning, SPIF strengthens domain-invariant features crucial for the task. Finally, in the experimental part, we adopt the DG verification method, dividing the dataset into different source domains and target domains for experimental verification. We verified the generalization performance of the proposed MSTCNet on two different datasets (IDGA and DTBA), and the experimental results confirmed the practicality and effectiveness of our method.