Genomics-Aware Multimodal Self-Supervised Learning for Cancer Survival Prediction

Survival prediction in cancer diagnosis is a critical research task. Current methods often employ the multimodal feature fusion of pathological images and genomics data within a weakly-supervised learning paradigm. However, these approaches fail to efficiently learn the intrinsic features of large amount of unlabeled WSIs and neglect the strong associations between genomics data and pathological images, resulting in reduced prognostic accuracy. To address these challenges, we propose a novel Genomics-Aware Multimodal Self-Supervised Learning model that designs a multimodal pretext task, improving learning of intra-modal features and inter-modal correlations without additional annotations. Specifically, we randomly mask pathological patch features and fuse unmasked pathology representations with genomics representations via a cross-modal attention module. Then we add mask tokens to the genomicsguided pathology representation and reconstruct the missing parts via a reconstruction decoder. Experimental results on four TCGA datasets demonstrate the superior performance of our method compared to state-of-the-art methods, highlighting its potential for advancing survival prediction. Our code is available at https://github.com/sunkevin101/GMSL.