Reference-based infrared image colorization via feature enhancement and context refinement

Reference-based infrared image colorization is a pioneering task that establishes a precise semantic relationship between infrared images and reference color images, facilitating the accurate transfer of color information from the reference image to the infrared image. However, existing methods often neglect the correlation between the input and reference images, which can result in the loss of essential details. To address these issues, we propose a reference-based infrared image colorization method that employs feature enhancement and context refinement (FCNet). Specifically, we design a dual-path feature mining module to extract features at different scales and fuse features from various levels. In the image reconstruction phase, we introduce a multi-information saliency transformer to capture global, frequency, and local features, allowing the network to better learn and model the target's features for precise colorization. Finally, by aggregating the correspondence between feature maps of varying depths and the target through a cross-scale information aggregation module and a context information refinement module, we enhance the feature information to aid the network in recovering edge and texture details. Extensive experiments demonstrate that the proposed FCNet achieves state-of-the-art performance, with PSNR and SSIM reaching 28.670 and 0.542 on the KAIST dataset, and 30.424 and 0.726 on the FLIR dataset, respectively. Our code and models are available on GitHub.