Standardized evaluation methodology and reference database for evaluating IVUS image segmentation

Simone Balocco; Carlo Gatta; Francesco Ciompi; Andreas Wahle; Petia Radeva; Stephane Carlier; Gozde Unal; Elias Sanidas; Josepa Mauri; Xavier Carillo; Tomas Kovarnik; Ching Wei Wang; Hsiang Chou Chen; Themis P. Exarchos; Dimitrios I. Fotiadis; François Destrempes; Guy Cloutier; Oriol Pujol; Marina Alberti; E. Gerardo Mendizabal-Ruiz; Mariano Rivera; Timur Aksoy; Richard W. Downe; Ioannis A. Kakadiaris

doi:10.1016/j.compmedimag.2013.07.001

Standardized evaluation methodology and reference database for evaluating IVUS image segmentation

Simone Balocco^*, Carlo Gatta, Francesco Ciompi, Andreas Wahle, Petia Radeva, Stephane Carlier, Gozde Unal, Elias Sanidas, Josepa Mauri, Xavier Carillo, Tomas Kovarnik, Ching Wei Wang, Hsiang Chou Chen, Themis P. Exarchos, Dimitrios I. Fotiadis, François Destrempes, Guy Cloutier, Oriol Pujol, Marina Alberti, E. Gerardo Mendizabal-RuizMariano Rivera, Timur Aksoy, Richard W. Downe, Ioannis A. Kakadiaris

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

124 Citations (Scopus)

Abstract

This paper describes an evaluation framework that allows a standardized and quantitative comparison of IVUS lumen and media segmentation algorithms. This framework has been introduced at the MICCAI 2011 Computing and Visualization for (Intra)Vascular Imaging (CVII) workshop, comparing the results of eight teams that participated.We describe the available data-base comprising of multi-center, multi-vendor and multi-frequency IVUS datasets, their acquisition, the creation of the reference standard and the evaluation measures. The approaches address segmentation of the lumen, the media, or both borders; semi- or fully-automatic operation; and 2-D vs. 3-D methodology. Three performance measures for quantitative analysis have been proposed. The results of the evaluation indicate that segmentation of the vessel lumen and media is possible with an accuracy that is comparable to manual annotation when semi-automatic methods are used, as well as encouraging results can be obtained also in case of fully-automatic segmentation. The analysis performed in this paper also highlights the challenges in IVUS segmentation that remains to be solved.

Original language	English
Pages (from-to)	70-90
Number of pages	21
Journal	Computerized Medical Imaging and Graphics
Volume	38
Issue number	2
DOIs	https://doi.org/10.1016/j.compmedimag.2013.07.001
Publication status	Published - Mar 2014
Externally published	Yes

Funding

Ioannis A. Kakadiaris is Hugh Roy and Lillie Cranz Cullen University Professor of Computer Science, Electrical & Computer Engineering, and Biomedical Engineering at UH. His research interests include computer vision, pattern recognition, biomedical image analysis, and biometrics. Dr. Kakadiaris is the recipient of a number of awards, including the NSF Early Career Development Award, Schlumberger Technical Foundation Award, UH Computer Science Research Excellence Award, UH Enron Teaching Excellence Award, and the James Muller Vulnerable Plaque Young Investigator Prize. His research has been featured on The Discovery Channel, National Public Radio, KPRC NBC News, KTRH ABC News, and KHOU CBS News. T. P. E. and D. I. F. are partially funded by ARTREAT , FP7-224297. G.C. and F.D. are partially funded by MDEIE, Canada ; Boston Scientific, Fremont, CA, USA ; NSERC (grant # 138570 ). B.S., F.C. and M.A. are partially funded by TIN2009-14404-C02; TIN2012-38187-C03-01; Boston Scientific, USA and SGR00696. C. G. is supported by MICINN (Ramon y Cajal Grant) . The work of C. W. W. and H. C. C. is partially funded by NSC , 101-2628-E-011-006-MY3 . A.W. and R.W.D.: National Institutes of Health, U.S.A. (R01EB004640, R01HL063373). T.K.: Czech Ministry of Health, Czech Republic (IGA NR9214-3). E. G. M. was supported by CONACYT. I. A. K. was partially supported by NSF Grant DMS-0915242 and the UH Hugh Roy and Lillie Cranz Cullen Endowment Fund.

Funders	Funder number
ARTREAT	FP7-224297
James Muller Vulnerable Plaque
Schlumberger Technical Foundation
UH Hugh Roy and Lillie Cranz Cullen Endowment Fund
National Science Foundation	DMS-0915242
National Heart, Lung, and Blood Institute	R01HL063373
Ministère du Développement Économique, de l’Innovation et de l’Exportation
Natural Sciences and Engineering Research Council of Canada	SGR00696, TIN2012-38187-C03-01, TIN2009-14404-C02, 138570
National Science Council	101-2628-E-011-006-MY3
Consejo Nacional de Ciencia y Tecnología
Ministerio de Ciencia e Innovación

Keywords

Algorithm comparison
Evaluation framework
IVUS (intravascular ultrasound)
Image segmentation

Access to Document

10.1016/j.compmedimag.2013.07.001

Cite this

Balocco, S., Gatta, C., Ciompi, F., Wahle, A., Radeva, P., Carlier, S., Unal, G., Sanidas, E., Mauri, J., Carillo, X., Kovarnik, T., Wang, C. W., Chen, H. C., Exarchos, T. P., Fotiadis, D. I., Destrempes, F., Cloutier, G., Pujol, O., Alberti, M., ... Kakadiaris, I. A. (2014). Standardized evaluation methodology and reference database for evaluating IVUS image segmentation. Computerized Medical Imaging and Graphics, 38(2), 70-90. https://doi.org/10.1016/j.compmedimag.2013.07.001

@article{f3ee0aabfe7c4655ac2dd9c02a577b24,

title = "Standardized evaluation methodology and reference database for evaluating IVUS image segmentation",

abstract = "This paper describes an evaluation framework that allows a standardized and quantitative comparison of IVUS lumen and media segmentation algorithms. This framework has been introduced at the MICCAI 2011 Computing and Visualization for (Intra)Vascular Imaging (CVII) workshop, comparing the results of eight teams that participated.We describe the available data-base comprising of multi-center, multi-vendor and multi-frequency IVUS datasets, their acquisition, the creation of the reference standard and the evaluation measures. The approaches address segmentation of the lumen, the media, or both borders; semi- or fully-automatic operation; and 2-D vs. 3-D methodology. Three performance measures for quantitative analysis have been proposed. The results of the evaluation indicate that segmentation of the vessel lumen and media is possible with an accuracy that is comparable to manual annotation when semi-automatic methods are used, as well as encouraging results can be obtained also in case of fully-automatic segmentation. The analysis performed in this paper also highlights the challenges in IVUS segmentation that remains to be solved.",

keywords = "Algorithm comparison, Evaluation framework, IVUS (intravascular ultrasound), Image segmentation",

author = "Simone Balocco and Carlo Gatta and Francesco Ciompi and Andreas Wahle and Petia Radeva and Stephane Carlier and Gozde Unal and Elias Sanidas and Josepa Mauri and Xavier Carillo and Tomas Kovarnik and Wang, {Ching Wei} and Chen, {Hsiang Chou} and Exarchos, {Themis P.} and Fotiadis, {Dimitrios I.} and Fran{\c c}ois Destrempes and Guy Cloutier and Oriol Pujol and Marina Alberti and Mendizabal-Ruiz, {E. Gerardo} and Mariano Rivera and Timur Aksoy and Downe, {Richard W.} and Kakadiaris, {Ioannis A.}",

year = "2014",

month = mar,

doi = "10.1016/j.compmedimag.2013.07.001",

language = "English",

volume = "38",

pages = "70--90",

journal = "Computerized Medical Imaging and Graphics",

issn = "0895-6111",

publisher = "Elsevier Ltd",

number = "2",

}

Balocco, S, Gatta, C, Ciompi, F, Wahle, A, Radeva, P, Carlier, S, Unal, G, Sanidas, E, Mauri, J, Carillo, X, Kovarnik, T, Wang, CW, Chen, HC, Exarchos, TP, Fotiadis, DI, Destrempes, F, Cloutier, G, Pujol, O, Alberti, M, Mendizabal-Ruiz, EG, Rivera, M, Aksoy, T, Downe, RW & Kakadiaris, IA 2014, 'Standardized evaluation methodology and reference database for evaluating IVUS image segmentation', Computerized Medical Imaging and Graphics, vol. 38, no. 2, pp. 70-90. https://doi.org/10.1016/j.compmedimag.2013.07.001

TY - JOUR

T1 - Standardized evaluation methodology and reference database for evaluating IVUS image segmentation

AU - Balocco, Simone

AU - Gatta, Carlo

AU - Ciompi, Francesco

AU - Wahle, Andreas

AU - Radeva, Petia

AU - Carlier, Stephane

AU - Unal, Gozde

AU - Sanidas, Elias

AU - Mauri, Josepa

AU - Carillo, Xavier

AU - Kovarnik, Tomas

AU - Wang, Ching Wei

AU - Chen, Hsiang Chou

AU - Exarchos, Themis P.

AU - Fotiadis, Dimitrios I.

AU - Destrempes, François

AU - Cloutier, Guy

AU - Pujol, Oriol

AU - Alberti, Marina

AU - Mendizabal-Ruiz, E. Gerardo

AU - Rivera, Mariano

AU - Aksoy, Timur

AU - Downe, Richard W.

AU - Kakadiaris, Ioannis A.

PY - 2014/3

Y1 - 2014/3

N2 - This paper describes an evaluation framework that allows a standardized and quantitative comparison of IVUS lumen and media segmentation algorithms. This framework has been introduced at the MICCAI 2011 Computing and Visualization for (Intra)Vascular Imaging (CVII) workshop, comparing the results of eight teams that participated.We describe the available data-base comprising of multi-center, multi-vendor and multi-frequency IVUS datasets, their acquisition, the creation of the reference standard and the evaluation measures. The approaches address segmentation of the lumen, the media, or both borders; semi- or fully-automatic operation; and 2-D vs. 3-D methodology. Three performance measures for quantitative analysis have been proposed. The results of the evaluation indicate that segmentation of the vessel lumen and media is possible with an accuracy that is comparable to manual annotation when semi-automatic methods are used, as well as encouraging results can be obtained also in case of fully-automatic segmentation. The analysis performed in this paper also highlights the challenges in IVUS segmentation that remains to be solved.

AB - This paper describes an evaluation framework that allows a standardized and quantitative comparison of IVUS lumen and media segmentation algorithms. This framework has been introduced at the MICCAI 2011 Computing and Visualization for (Intra)Vascular Imaging (CVII) workshop, comparing the results of eight teams that participated.We describe the available data-base comprising of multi-center, multi-vendor and multi-frequency IVUS datasets, their acquisition, the creation of the reference standard and the evaluation measures. The approaches address segmentation of the lumen, the media, or both borders; semi- or fully-automatic operation; and 2-D vs. 3-D methodology. Three performance measures for quantitative analysis have been proposed. The results of the evaluation indicate that segmentation of the vessel lumen and media is possible with an accuracy that is comparable to manual annotation when semi-automatic methods are used, as well as encouraging results can be obtained also in case of fully-automatic segmentation. The analysis performed in this paper also highlights the challenges in IVUS segmentation that remains to be solved.

KW - Algorithm comparison

KW - Evaluation framework

KW - IVUS (intravascular ultrasound)

KW - Image segmentation

UR - http://www.scopus.com/inward/record.url?scp=84895070065&partnerID=8YFLogxK

U2 - 10.1016/j.compmedimag.2013.07.001

DO - 10.1016/j.compmedimag.2013.07.001

M3 - Article

C2 - 24012215

AN - SCOPUS:84895070065

SN - 0895-6111

VL - 38

SP - 70

EP - 90

JO - Computerized Medical Imaging and Graphics

JF - Computerized Medical Imaging and Graphics

IS - 2

ER -

Standardized evaluation methodology and reference database for evaluating IVUS image segmentation

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this