ProtoMM: Interpretable Prototype-Based Multimodal Model for Brain Cancer Survival Prediction

Recent advancements in deep learning have substantially improved medical imaging analysis. However, medical diagnosis often requires the integration of data from multiple modalities, and existing deep learning approaches frequently lack interpretability in multimodal contexts. These limitations lead to challenges in trust and reliability when making critical medical decisions. Current methods often rely on post-hoc explanations, which provide limited and sometimes unreliable insights. To address these challenges, we propose ProtoMM, a prototype-based multimodal model for medical data analysis that emphasizes interpretability. ProtoMM utilizes self-explanatory prototypes and transparent inference processes, providing reliable case explanations. By employing multimodal fusion, the model enhances inter-modal interactions, learning representative cases through prototype layers. We introduce two prototype layer levels: an aggregate layer that treats multimodal data as a unified prototype case and a singleton layer that distinguishes between individual modalities. We demonstrate the efficacy of ProtoMM in survival prediction tasks, where it achieves a Concordance Index (C-Index) of 0.793 ± 0.027. This result is comparable to state-of-the-art black-box models, yet our model provides fully interpretable insights into its decision-making process.