Assessing the Generalization Ability of a Global Model for Rapid Building Damage Assessment in Real-world Disaster Scenarios

After a natural disaster, it is very important to be able to make a building damage assessment quickly and accurately. Considering real-world scenarios, one of the most important issues related to the deep learning based computer vision models used in this field is their generalization ability. Ideally, we would like to have a global model which is capable of making damage assessment successfully in all disasters in the future. However, this is a difficult task as images of one disaster varies greatly from images of another due to reasons like different geographical region structure, structure of houses, position of clouds, etc. On the other hand, many damage assessment models are evaluated on an unrealistic in-domain (ID) test set, which contains data from the same disasters in the training and test sets. In this study, we focus on the generalization problem and compare the model performance in an ID test setting with that in an out-of-domain (OOD) test setting, which is more realistic since the test data is formed from disasters not in the training set. We show that the performance of the model is comparatively lower for the OOD test setting and there is a large generalization gap in this case. We conclude that the models should be evaluated by using the OOD test setting. Then, we propose to generate post-disaster images and masks by using generative adversarial network (GAN) models and use the generated data to fine-tune the damage assessment model. It is seen that the generalization ability of the damage assessment model is increased by using this approach.