Evaluation of Explainability Methods in Medical Image Analysis

In the field of medical image analysis, artificial intelligence (AI) has made significant research-level advancements and is gradually being integrated into real-world applications. Explaining the decisions made by AI in the context of medical image analysis is crucial. However, it is equally important that these explanations are evaluated quantitatively with appropriate metrics to be able to quantify the success of various explainability methods. Considering that many medical decisions are subjective and vary with experience, we can deduce how challenging yet necessary this task is. In this chapter, we present the various challenges and the multitude of algorithms designed to make this quantitative evaluation meaningful. We explain the current state of the literature for quantitative evaluation metrics in medical image analysis and highlight the limitations with possible future directions.