Why is facial occlusion a challenging problem?

This paper investigates the main reason for the obtained low performance when the face recognition algorithms are tested on partially occluded face images. It has been observed that in the case of upper face occlusion, missing discriminative information due to occlusion only accounts for a very small part of the performance drop. The main factor is found to be the registration errors due to erroneous facial feature localization. It has been shown that by solving the misalignment problem, very high correct recognition rates can be achieved with a generic local appearance-based face recognition algorithm. In the case of a lower face occlusion, only a slight decrease in the performance is observed, when a local appearance-based face representation approach is used. This indicates the importance of local processing when dealing with partial face occlusion. Moreover, improved alignment increases the correct recognition rate also in the experiments against the lower face occlusion, which shows that face registration plays a key role on face recognition performance.