Vision Transformer Based Multi-class Lesion Detection in IVOCT

Zixuan Wang; Yifan Shao; Jingyi Sun; Zhili Huang; Su Wang; Qiyong Li; Jinsong Li; Qian Yu

doi:10.1007/978-3-031-43987-2_32

Vision Transformer Based Multi-class Lesion Detection in IVOCT

Zixuan Wang
, Yifan Shao
, Jingyi Sun
, Zhili Huang
, Su Wang^*
, Qiyong Li
, Jinsong Li
, Qian Yu

^*Corresponding author for this work

School of Software

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

Cardiovascular disease is a high-fatality illness. Intravascular Optical Coherence Tomography (IVOCT) technology can significantly assist in diagnosing and treating cardiovascular diseases. However, locating and classifying lesions from hundreds of IVOCT images is time-consuming and challenging, especially for junior physicians. An automatic lesion detection and classification model is desirable. To achieve this goal, in this work, we first collect an IVOCT dataset, including 2,988 images from 69 IVOCT data and 4,734 annotations of lesions spanning over three categories. Based on the newly-collected dataset, we propose a multi-class detection model based on Vision Transformer, called G-Swin Transformer. The essential part of our model is grid attention which is used to model relations among consecutive IVOCT images. Through extensive experiments, we show that the proposed G-Swin Transformer can effectively localize different types of lesions in IVOCT images, significantly outperforming baseline methods in all evaluation metrics. Our code is available via this link. https://github.com/Shao1Fan/G-Swin-Transformer

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings
Editors	Hayit Greenspan, Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	327-336
Number of pages	10
ISBN (Print)	9783031439865
DOIs	https://doi.org/10.1007/978-3-031-43987-2_32
State	Published - 2023
Event	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 - Vancouver, Canada Duration: 8 Oct 2023 → 12 Oct 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14225 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023
Country/Territory	Canada
City	Vancouver
Period	8/10/23 → 12/10/23

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

IVOCT
Object Detection
Vision Transformer

Access to Document

10.1007/978-3-031-43987-2_32

Cite this

Wang, Z., Shao, Y., Sun, J., Huang, Z., Wang, S., Li, Q., Li, J., & Yu, Q. (2023). Vision Transformer Based Multi-class Lesion Detection in IVOCT. In H. Greenspan, H. Greenspan, A. Madabhushi, P. Mousavi, S. Salcudean, J. Duncan, T. Syeda-Mahmood, & R. Taylor (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings (pp. 327-336). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14225 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-43987-2_32

Wang, Zixuan ; Shao, Yifan ; Sun, Jingyi et al. / Vision Transformer Based Multi-class Lesion Detection in IVOCT. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. editor / Hayit Greenspan ; Hayit Greenspan ; Anant Madabhushi ; Parvin Mousavi ; Septimiu Salcudean ; James Duncan ; Tanveer Syeda-Mahmood ; Russell Taylor. Springer Science and Business Media Deutschland GmbH, 2023. pp. 327-336 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2afde68d9fb24641928d01e6ccf1dbe0,

title = "Vision Transformer Based Multi-class Lesion Detection in IVOCT",

abstract = "Cardiovascular disease is a high-fatality illness. Intravascular Optical Coherence Tomography (IVOCT) technology can significantly assist in diagnosing and treating cardiovascular diseases. However, locating and classifying lesions from hundreds of IVOCT images is time-consuming and challenging, especially for junior physicians. An automatic lesion detection and classification model is desirable. To achieve this goal, in this work, we first collect an IVOCT dataset, including 2,988 images from 69 IVOCT data and 4,734 annotations of lesions spanning over three categories. Based on the newly-collected dataset, we propose a multi-class detection model based on Vision Transformer, called G-Swin Transformer. The essential part of our model is grid attention which is used to model relations among consecutive IVOCT images. Through extensive experiments, we show that the proposed G-Swin Transformer can effectively localize different types of lesions in IVOCT images, significantly outperforming baseline methods in all evaluation metrics. Our code is available via this link. https://github.com/Shao1Fan/G-Swin-Transformer",

keywords = "IVOCT, Object Detection, Vision Transformer",

author = "Zixuan Wang and Yifan Shao and Jingyi Sun and Zhili Huang and Su Wang and Qiyong Li and Jinsong Li and Qian Yu",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.; 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023 ; Conference date: 08-10-2023 Through 12-10-2023",

year = "2023",

doi = "10.1007/978-3-031-43987-2\_32",

language = "英语",

isbn = "9783031439865",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "327--336",

editor = "Hayit Greenspan and Hayit Greenspan and Anant Madabhushi and Parvin Mousavi and Septimiu Salcudean and James Duncan and Tanveer Syeda-Mahmood and Russell Taylor",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings",

address = "德国",

}

Wang, Z, Shao, Y, Sun, J, Huang, Z, Wang, S, Li, Q, Li, J & Yu, Q 2023, Vision Transformer Based Multi-class Lesion Detection in IVOCT. in H Greenspan, H Greenspan, A Madabhushi, P Mousavi, S Salcudean, J Duncan, T Syeda-Mahmood & R Taylor (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14225 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 327-336, 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023, Vancouver, Canada, 8/10/23. https://doi.org/10.1007/978-3-031-43987-2_32

Vision Transformer Based Multi-class Lesion Detection in IVOCT. / Wang, Zixuan; Shao, Yifan; Sun, Jingyi et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. ed. / Hayit Greenspan; Hayit Greenspan; Anant Madabhushi; Parvin Mousavi; Septimiu Salcudean; James Duncan; Tanveer Syeda-Mahmood; Russell Taylor. Springer Science and Business Media Deutschland GmbH, 2023. p. 327-336 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14225 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Vision Transformer Based Multi-class Lesion Detection in IVOCT

AU - Wang, Zixuan

AU - Shao, Yifan

AU - Sun, Jingyi

AU - Huang, Zhili

AU - Wang, Su

AU - Li, Qiyong

AU - Li, Jinsong

AU - Yu, Qian

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.

PY - 2023

Y1 - 2023

N2 - Cardiovascular disease is a high-fatality illness. Intravascular Optical Coherence Tomography (IVOCT) technology can significantly assist in diagnosing and treating cardiovascular diseases. However, locating and classifying lesions from hundreds of IVOCT images is time-consuming and challenging, especially for junior physicians. An automatic lesion detection and classification model is desirable. To achieve this goal, in this work, we first collect an IVOCT dataset, including 2,988 images from 69 IVOCT data and 4,734 annotations of lesions spanning over three categories. Based on the newly-collected dataset, we propose a multi-class detection model based on Vision Transformer, called G-Swin Transformer. The essential part of our model is grid attention which is used to model relations among consecutive IVOCT images. Through extensive experiments, we show that the proposed G-Swin Transformer can effectively localize different types of lesions in IVOCT images, significantly outperforming baseline methods in all evaluation metrics. Our code is available via this link. https://github.com/Shao1Fan/G-Swin-Transformer

AB - Cardiovascular disease is a high-fatality illness. Intravascular Optical Coherence Tomography (IVOCT) technology can significantly assist in diagnosing and treating cardiovascular diseases. However, locating and classifying lesions from hundreds of IVOCT images is time-consuming and challenging, especially for junior physicians. An automatic lesion detection and classification model is desirable. To achieve this goal, in this work, we first collect an IVOCT dataset, including 2,988 images from 69 IVOCT data and 4,734 annotations of lesions spanning over three categories. Based on the newly-collected dataset, we propose a multi-class detection model based on Vision Transformer, called G-Swin Transformer. The essential part of our model is grid attention which is used to model relations among consecutive IVOCT images. Through extensive experiments, we show that the proposed G-Swin Transformer can effectively localize different types of lesions in IVOCT images, significantly outperforming baseline methods in all evaluation metrics. Our code is available via this link. https://github.com/Shao1Fan/G-Swin-Transformer

KW - IVOCT

KW - Object Detection

KW - Vision Transformer

UR - https://www.scopus.com/pages/publications/85174727669

U2 - 10.1007/978-3-031-43987-2_32

DO - 10.1007/978-3-031-43987-2_32

M3 - 会议稿件

AN - SCOPUS:85174727669

SN - 9783031439865

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 327

EP - 336

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings

A2 - Greenspan, Hayit

A2 - Madabhushi, Anant

A2 - Mousavi, Parvin

A2 - Salcudean, Septimiu

A2 - Duncan, James

A2 - Syeda-Mahmood, Tanveer

A2 - Taylor, Russell

PB - Springer Science and Business Media Deutschland GmbH

T2 - 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023

Y2 - 8 October 2023 through 12 October 2023

ER -

Wang Z, Shao Y, Sun J, Huang Z, Wang S, Li Q et al. Vision Transformer Based Multi-class Lesion Detection in IVOCT. In Greenspan H, Greenspan H, Madabhushi A, Mousavi P, Salcudean S, Duncan J, Syeda-Mahmood T, Taylor R, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 327-336. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-43987-2_32