Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning

Gökay Sezen; İlkay Öksüz

doi:10.1007/978-3-031-16919-9_7

Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning

Gökay Sezen^*, İlkay Öksüz

^*Corresponding author for this work

Department of Computer Engineering

Istanbul Technical University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Citations (Scopus)

Abstract

Magnetic Resonance Images (MRI) examinations are widely used for diagnosing injuries in the knee. Automatic interpretable detection of meniscus, Anterior Cruciate Ligament (ACL) tears, and general abnormalities from knee MRI is an essential task for automating the clinical diagnosis of knee MRI. This paper proposes a combination of convolution neural network and sequential network deep learning models for detecting general anomalies, ACL tears, and meniscal tears on knee MRI. We combine information from multiple MRI views with transformer blocks for final diagnosis. Also, we did an ablation study which is training with only CNN, and saw the impact of the transformer blocks on the learning. On average, we achieve a performance of 0.905 AUC for three injury cases on MRNet data.

Original language	English
Title of host publication	Predictive Intelligence in Medicine - 5th International Workshop, PRIME 2022, Held in Conjunction with MICCAI 2022, Proceedings
Editors	Islem Rekik, Ehsan Adeli, Sang Hyun Park, Celia Cintas
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	71-78
Number of pages	8
ISBN (Print)	9783031169182
DOIs	https://doi.org/10.1007/978-3-031-16919-9_7
Publication status	Published - 2022
Event	5th International Workshop on Predictive Intelligence in Medicine, PRIME 2022, held in conjunction with 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 - Virtual, Online Duration: 22 Sept 2022 → 22 Sept 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13564 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	5th International Workshop on Predictive Intelligence in Medicine, PRIME 2022, held in conjunction with 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022
City	Virtual, Online
Period	22/09/22 → 22/09/22

Bibliographical note

Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Funding

Acknowledgments. This paper has been produced benefiting from the 2232 International Fellowship for Outstanding Researchers Program of TUBITAK (Project No: 118C353). However, the entire responsibility of the publication/paper belongs to the owner of the paper. The financial support received from TUBITAK does not mean that the content of the publication is approved in a scientific sense by TUBITAK.

Funders	Funder number
Türkiye Bilimsel ve Teknolojik Araştırma Kurumu	118C353

Keywords

Abnormal
ACL
Knee MRI
Meniscus
MRNet

Access to Document

10.1007/978-3-031-16919-9_7

Cite this

Sezen, G., & Öksüz, İ. (2022). Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning. In I. Rekik, E. Adeli, S. H. Park, & C. Cintas (Eds.), Predictive Intelligence in Medicine - 5th International Workshop, PRIME 2022, Held in Conjunction with MICCAI 2022, Proceedings (pp. 71-78). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13564 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-16919-9_7

Sezen, Gökay ; Öksüz, İlkay. / Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning. Predictive Intelligence in Medicine - 5th International Workshop, PRIME 2022, Held in Conjunction with MICCAI 2022, Proceedings. editor / Islem Rekik ; Ehsan Adeli ; Sang Hyun Park ; Celia Cintas. Springer Science and Business Media Deutschland GmbH, 2022. pp. 71-78 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3706ac67d856400bacdf0ef55259bc6e,

title = "Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning",

abstract = "Magnetic Resonance Images (MRI) examinations are widely used for diagnosing injuries in the knee. Automatic interpretable detection of meniscus, Anterior Cruciate Ligament (ACL) tears, and general abnormalities from knee MRI is an essential task for automating the clinical diagnosis of knee MRI. This paper proposes a combination of convolution neural network and sequential network deep learning models for detecting general anomalies, ACL tears, and meniscal tears on knee MRI. We combine information from multiple MRI views with transformer blocks for final diagnosis. Also, we did an ablation study which is training with only CNN, and saw the impact of the transformer blocks on the learning. On average, we achieve a performance of 0.905 AUC for three injury cases on MRNet data.",

keywords = "Abnormal, ACL, Knee MRI, Meniscus, MRNet",

author = "G{\"o}kay Sezen and İlkay {\"O}ks{\"u}z",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 5th International Workshop on Predictive Intelligence in Medicine, PRIME 2022, held in conjunction with 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 ; Conference date: 22-09-2022 Through 22-09-2022",

year = "2022",

doi = "10.1007/978-3-031-16919-9\_7",

language = "English",

isbn = "9783031169182",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "71--78",

editor = "Islem Rekik and Ehsan Adeli and Park, \{Sang Hyun\} and Celia Cintas",

booktitle = "Predictive Intelligence in Medicine - 5th International Workshop, PRIME 2022, Held in Conjunction with MICCAI 2022, Proceedings",

address = "Germany",

}

Sezen, G & Öksüz, İ 2022, Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning. in I Rekik, E Adeli, SH Park & C Cintas (eds), Predictive Intelligence in Medicine - 5th International Workshop, PRIME 2022, Held in Conjunction with MICCAI 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13564 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 71-78, 5th International Workshop on Predictive Intelligence in Medicine, PRIME 2022, held in conjunction with 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022, Virtual, Online, 22/09/22. https://doi.org/10.1007/978-3-031-16919-9_7

Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning. / Sezen, Gökay; Öksüz, İlkay.
Predictive Intelligence in Medicine - 5th International Workshop, PRIME 2022, Held in Conjunction with MICCAI 2022, Proceedings. ed. / Islem Rekik; Ehsan Adeli; Sang Hyun Park; Celia Cintas. Springer Science and Business Media Deutschland GmbH, 2022. p. 71-78 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13564 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning

AU - Sezen, Gökay

AU - Öksüz, İlkay

PY - 2022

Y1 - 2022

N2 - Magnetic Resonance Images (MRI) examinations are widely used for diagnosing injuries in the knee. Automatic interpretable detection of meniscus, Anterior Cruciate Ligament (ACL) tears, and general abnormalities from knee MRI is an essential task for automating the clinical diagnosis of knee MRI. This paper proposes a combination of convolution neural network and sequential network deep learning models for detecting general anomalies, ACL tears, and meniscal tears on knee MRI. We combine information from multiple MRI views with transformer blocks for final diagnosis. Also, we did an ablation study which is training with only CNN, and saw the impact of the transformer blocks on the learning. On average, we achieve a performance of 0.905 AUC for three injury cases on MRNet data.

AB - Magnetic Resonance Images (MRI) examinations are widely used for diagnosing injuries in the knee. Automatic interpretable detection of meniscus, Anterior Cruciate Ligament (ACL) tears, and general abnormalities from knee MRI is an essential task for automating the clinical diagnosis of knee MRI. This paper proposes a combination of convolution neural network and sequential network deep learning models for detecting general anomalies, ACL tears, and meniscal tears on knee MRI. We combine information from multiple MRI views with transformer blocks for final diagnosis. Also, we did an ablation study which is training with only CNN, and saw the impact of the transformer blocks on the learning. On average, we achieve a performance of 0.905 AUC for three injury cases on MRNet data.

KW - Abnormal

KW - ACL

KW - Knee MRI

KW - Meniscus

KW - MRNet

UR - https://www.scopus.com/pages/publications/85140435009

U2 - 10.1007/978-3-031-16919-9_7

DO - 10.1007/978-3-031-16919-9_7

M3 - Conference contribution

AN - SCOPUS:85140435009

SN - 9783031169182

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 71

EP - 78

BT - Predictive Intelligence in Medicine - 5th International Workshop, PRIME 2022, Held in Conjunction with MICCAI 2022, Proceedings

A2 - Rekik, Islem

A2 - Adeli, Ehsan

A2 - Park, Sang Hyun

A2 - Cintas, Celia

PB - Springer Science and Business Media Deutschland GmbH

T2 - 5th International Workshop on Predictive Intelligence in Medicine, PRIME 2022, held in conjunction with 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022

Y2 - 22 September 2022 through 22 September 2022

ER -

Sezen G, Öksüz İ. Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning. In Rekik I, Adeli E, Park SH, Cintas C, editors, Predictive Intelligence in Medicine - 5th International Workshop, PRIME 2022, Held in Conjunction with MICCAI 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2022. p. 71-78. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-16919-9_7

Diagnosing Knee Injuries from MRI with Transformer Based Deep Learning

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this