PANSHARPENING BASED ON DIFFUSION MODEL VIA ADAPTIVE FEATURE SEPARATION REPRESENTATION

Generative diffusion models have shown great potential in pansharpening, but the generalization to unseen real images is still challenging. Current methods aim to improve this through unsupervised frameworks or sophisticated networks, yet most fail to distinguish common patterns from unique characteristics in remote sensing images, limiting their ability to learn general and flexible feature representations. The entanglement between features can result in single patterns generation and sub-optimal generalization. To address these issues, this paper proposes a separation representation diffusion model (SRDiff). Specifically, the encoded features are separated through representation separation module (RSM), obtaining unique features (UFs) and common features (CFs). With the CFs diffusing in the latent space, SRDiff tends to learn universal generative modes. For sufficient separation, we propose a randomized separation strategy (RandSep), which achieves better results by contrastively learning the CFs and the UFs of the reference. Moreover, a dense semantic module (DSM) is designed for the refinement of diverse generative modes and the interplay between the UFs and the CFs. So the dense information and the UFs would be injected in diffusion to obtain more spatial details. Finally, we design a comprehensive objective function for overall improvement of fusion products. Extensive experiments on Quickbird at all resolutions have demonstrated the effectiveness and great generalization of our model.