Automatic Medical Report Generation Based on Detector Attention Module and GPT-Based Word LSTM

The writing of medical reports is extremely time-consuming for professional doctors. Despite the introduction of numerous deep learning-driven models for automated medical report generation, there remains considerable room for enhancing their performance. One obstacle is that general automatic medical report generation models have failed to consider the location information of the lesions. The other is that these models are limited by the small diagnostic report corpus that fails to improve the fluency of diagnostic reports. Therefore, we propose a novel automatic report generation model to address the aforementioned two challenges. We first propose Detector Attention Module to fuse the coarse-grained visual features and fine-grained location features, which improves the performance of automatic medical report generation. Meanwhile, we employ the Generative Pre-Trained Transformer (GPT) model, which can be pre-trained on unsupervised massive diagnostic reports, to extract contextual semantic information, aiming to enhance the fluency of diagnostic reports. On the open-source IU X-ray dataset, our model has shown an average improvement of 0.6% across six evaluation metrics compared to the current state-of-the-art (SOTA). This indicates that our model has the capacity to produce more precise and comprehensive diagnostic reports.