A Hybrid Face Recognition Approach Using Local Appearance and Deep Models

Visible and thermal face recognition are highly important topics in computer vision. Existing face recognition models generally focus on facial images in the visible domain. However, they fail in the lack of the light and become non-functional at night. In addition, the performance of the models decreases in the case of occlusion. This work aims to build a hybrid two-branch pipeline that detects, aligns, represents, and recognizes a face from either thermal or visible domains using both a local appearance-based and a deep learning-based method. In addition, we present a fusion scheme to combine the outputs of these methods in the final stage. The recent state-of-the-art deep learning-based face recognition approaches mainly focus on eye region for identification. This leads to a performance drop when these models are confronted with occluded faces. On the other hand, local appearance-based approaches have been shown to be robust to occlusion as they extract features from all parts of the face. Therefore, in order to enable a high-accuracy face recognition pipeline, we combine deep learning and local appearance based models. We have conducted extensive experiments on the EURECOM and ROF datasets to assess the performance of the proposed approach. Experimental results show that in both domains there are significant improvements in classification accuracies under various facial appearances variations due to the factors, such as facial expressions, illumination conditions, and occlusion.