A Doctors Behavior Aware and Domain Knowledge Driven Model for Medical Report Generation

When doctors write a medical report, they first focus on key regions in the image, which usually contain abnormal information. Then, they write the report based on their experience and professional knowledge. Some existing medical report generation methods design corresponding schemes to model this process, but they still have some drawbacks. Some of them adopt the attention map to model the doctors' image-reading behavior, which requires additional cropping and feature extraction operations, bringing many extra computations. Other methods manually construct knowledge graphs to model the doctors' domain knowledge, which requires manual construction for different disease domains and has less generalization. Therefore, We propose DEKG, a doctors behavior aware and domain knowledge driven model, to improve the quality of generated reports by modeling the fixed pattern of doctors when writing reports. Specifically, feature clustering is utilized to model the doctors' image-reading behavior that only brings fewer computations, and then we capture key regions with possible abnormalities. We also design a method to automatically construct domain knowledge graphs, which can quickly accomplish the construction process for different disease domains without human intervention. Extensive experiments on datasets from different disease domains demonstrate that DEKG achieves competitive results with state-of-the-art methods and can generate high-quality reports.