MTFuse: Fast Multi-Task Network for Infrared and Visible Image Fusion in Nighttime Imaging

Electronic equipment used for videotaping is often not well imaged at night. Infrared-visible image fusion and image brightness enhancement are two important methods used in nighttime imaging. Existing methods do not unite these two approaches appropriately. In order to cross the gap between them, this paper proposes a multi-task fusion framework (MTFuse), which combines brightness enhancement and fusion tasks into an end-to-end network to solve the environment awareness problem in dark light. Firstly, a fast feature extraction module is constructed based on the reparameterisation theory. The module provides a rich stream of information to the network during training and degenerates losslessly into a convolutional layer during testing. Next, the brightness enhancement module is designed based on Retinex theory as the basic unit of the feature extraction part. Ultimately, the labels are generated using the brightness enhancement algorithm for training. A large number of experiments show that the proposed method has achieved the most advanced performance in vision and quantitative comparisons. In addition, the proposed method also shows great application prospects in the field of multispectral detection. Our code is available at https://github.com/UAVSwarm/MT-fuse-code.