HSDG: A dual-prior semantic driven entropy grouping snapshot medical hyperspectral tongue image reconstruction method

Background and Objective: Tongue hyperspectral imaging (THSI) provides rich spectral information, which is crucial for various medical tasks. However, existing hyperspectral acquisition devices involve trade-offs between image quality, acquisition time, and cost. Additionally, the jitter characteristics during tongue image acquisition necessitate snapshot image reconstruction, and there is a lack of established tongue hyperspectral datasets. This research aims to develop a high-quality and application-potential reconstruction method for snapshot spectral imaging. The reconstruction method will be implemented in low-cost, self-developed spectral imaging devices to establish valuable tongue hyperspectral image datasets. Methodology: First, This study proposes a Hierarchical Semantic-Driven Grouping (HSDG) reconstruction method based on two types of semantic annotations. This method effectively addresses the phenomenon of metamerism and the disruption of local spectral correlations caused by considering only global associations. It achieves this by calculating the information entropy of two types of semantic features to perform a reasonable spectral grouping and reconstruction. We validated the effectiveness and advancement of our method through three evaluation metrics for segmentation experiments and three image quality assessment metrics, comparing it with ten state-of-the-art reconstruction algorithms. Results: According to the training results, compared to the best existing method, the proposed approach improved image quality by 1.36 dB and increased segmentation accuracy by 6.62 %. It can reconstruct clear detailed information and provide satisfactory spectral curves. The Peak Signal-to-Noise Ratio (PSNR) reached 34.15, the Structural Similarity (SSIM) index reached 0.8663, and the accuracy for image segmentation reached 0.9497. Conclusion: The proposed HSDG addresses the severe issue of information degradation caused by compressing 24 spectral bands into a single dimension under different position light source modulations. It achieves this through dual prior semantic information grouping. Additionally, through tongue feature segmentation experiments, the proposed reconstruction method has been validated as most closely resembling the actual images, capable of delineating clear distinctions in tongue cracks and coating areas.