TY - GEN
T1 - Recruiting the Best Teacher Modality
T2 - 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
AU - Dai, Ning
AU - Jiang, Lai
AU - Fu, Yibing
AU - Pan, Sai
AU - Xu, Mai
AU - Deng, Xin
AU - Chen, Pu
AU - Chen, Xiangmei
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
PY - 2023
Y1 - 2023
N2 - The joint use of multiple imaging modalities for medical image has been widely studied in recent years. The fusion of information from different modalities has demonstrated the performance improvement for some medical tasks. For nephropathy diagnosis, immunofluorescence (IF) is one of the most widely-used medical image due to its ease of acquisition with low cost, which is also an advanced multi-modality technique. However, the existing methods mainly integrate information from diverse sources by averaging or combining them, failing to exploit multi-modality knowledge in details. In this paper, we observe that the 7 modalities of IF images have different impact on different nephropathy categories. Accordingly, we propose a knowledge distillation framework to transfer knowledge from the trained single-modality teacher networks to a multi-modality student network. On top of this, given a input IF sequence, a recruitment module is developed to dynamically assign weights to teacher models and optimize the performance of student model. By applying on several different architectures, the extensive experimental results verify the effectiveness of our method for nephropathy diagnosis.
AB - The joint use of multiple imaging modalities for medical image has been widely studied in recent years. The fusion of information from different modalities has demonstrated the performance improvement for some medical tasks. For nephropathy diagnosis, immunofluorescence (IF) is one of the most widely-used medical image due to its ease of acquisition with low cost, which is also an advanced multi-modality technique. However, the existing methods mainly integrate information from diverse sources by averaging or combining them, failing to exploit multi-modality knowledge in details. In this paper, we observe that the 7 modalities of IF images have different impact on different nephropathy categories. Accordingly, we propose a knowledge distillation framework to transfer knowledge from the trained single-modality teacher networks to a multi-modality student network. On top of this, given a input IF sequence, a recruitment module is developed to dynamically assign weights to teacher models and optimize the performance of student model. By applying on several different architectures, the extensive experimental results verify the effectiveness of our method for nephropathy diagnosis.
KW - IF image
KW - Knowledge distillation
KW - Nephropathy diagnosis
UR - https://www.scopus.com/pages/publications/85174722878
U2 - 10.1007/978-3-031-43904-9_51
DO - 10.1007/978-3-031-43904-9_51
M3 - 会议稿件
AN - SCOPUS:85174722878
SN - 9783031439032
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 526
EP - 536
BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings
A2 - Greenspan, Hayit
A2 - Greenspan, Hayit
A2 - Madabhushi, Anant
A2 - Mousavi, Parvin
A2 - Salcudean, Septimiu
A2 - Duncan, James
A2 - Syeda-Mahmood, Tanveer
A2 - Taylor, Russell
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 8 October 2023 through 12 October 2023
ER -