MCA: Multimodal Contrastive Augmentation for Medical Report Generation

Automated radiology report generation has significant clinical potential to alleviate the heavy workloads of radiologists and improve diagnostic interpretation efficiency. The challenge in this task lies in typical data biases, characterized by a predominance of normal regions with only a small portion of abnormalities. Recently, researchers have focused on mitigating this bias by comparing images with retrieved historical images and reports using contrastive attention mechanisms. However, there are two issues: 1) The retrieved images and the input images often come from different patients, resulting in variations in lung contours and sizes. 2) The retrieved reports tend to be verbose and may contain irrelevant or even contradictory information. To address the limitations, we propose a medical report generation framework that leverages normal medical image reconstruction and structured knowledge to enhance contrastive mechanisms. Extensive experiments on two widely-used benchmarks, IU X-Ray and MIMIC-ABN, demonstrate that the proposed model outperforms other state-of-art (SOTA) methods on almost all metrics.