Computational spectral imaging reconstruction via a spatial–spectral cross-attention-driven network

Compared with traditional hyperspectral imaging, computational spectral imaging (CSI) has the advantage of snapshot imaging with high spatial and temporal resolution, which has attracted considerable attention. The core challenge of CSI is to achieve computational imaging reconstruction from a single 2D measurement image to the corresponding 3D spatial–hyperspectral image (HSI). Existing reconstruction methods still face problems in exploring spatial–spectral cross correlation, leading to significant spatial–spectral distortion. Furthermore, due to neglect of multi-scale feature reconstruction, their reconstruction quality still needs to be improved. In this paper, to solve the above problems, we propose a spatial–spectral cross-attention-driven network (SSCA-DN). In SSCA, a proposed multi-scale feature aggregation (MFA) module and a spectral-wise transformer (SpeT) are used for multi-scale spatial feature reconstruction and long-range spectral feature reconstruction, respectively. Using spatial attention and spectral attention to interactively guide the reconstruction of the target HSI in spectral and spatial dimensions, the proposed SSCA models spatial–spectral cross correlation with considering multi-scale features. Using the SSCA as a basic module, a novel SSCA-DN network is constructed, in which a proposed supervised preliminary reconstruction subnetwork (SPRNet) learns the generalized prior, and a proposed unsupervised multi-scale feature fusion and refinement subnetwork (UMFFRNet) learns the specific prior. The SSCA module ensures that the learned generalized and specific priors can capture the spatial–spectral cross correlation while considering multi-scale features. In addition, in UMFFRNet, driven by MFA and SSCA, a novel multi-scale fusion and refinement mechanism for multi-level adjacent features is proposed to effectively model the correlation between adjacent level features and the multi-scale spatial–spectral cross correlation, which further improves the reconstruction accuracy. Extensive experiments show that our method achieves state-of-the-art performance on both simulated and real datasets.