USTNet: Ultrafast style transfer between infrared and visible images

Visible light and infrared images belong to different, modalities, which is not conductive to the image registration. To achieve better image registration and fusion, it is necessary to convert the modality of visible light images into infrared modality before image registration. The paper proposes a lightweight real-time framework named USTNet for generating infrared-like visible-light images, which can bridge the modality gap in visible and infrared domains. To accelerate the feature representation at early stage, a Channel Acceleration block is adopted; Besides, the Faster Transfer Module (FTM) and the Detail Connection Module (DCM) ensure the speed of semantic embedding and the high-quality reconstructed image, respectively. Moreover, by introducing Matting Laplacian constraint and dWCT transform, the regionally semantically consistent features as well as coherent regional styles across source and target domain are enforced in this work. The proposed USTNet is evaluated on both near-infrared and far-infrared datasets and compared against six state-of-the-art style transfer methods. Extensive experiments show that the proposed USTNet achieves superior qualitative and quantitative results with a 2x speedup over the state-of-the-art methods, making it more compatible for real-time cross-modal vision applications.