On Recognizing Occluded Faces in the Wild

Mustafa Ekrem Erakin; Ugur Demir; Hazim Kemal Ekenel

doi:10.1109/BIOSIG52210.2021.9548293

On Recognizing Occluded Faces in the Wild

Mustafa Ekrem Erakin, Ugur Demir, Hazim Kemal Ekenel

Department of Computer Engineering

Istanbul Technical University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

12 Citations (Scopus)

Abstract

Facial appearance variations due to occlusion has been one of the main challenges for face recognition systems. To facilitate further research in this area, it is necessary and important to have occluded face datasets collected from real-world, as synthetically generated occluded faces cannot represent the nature of the problem. In this paper, we present the Real World Occluded Faces (ROF) dataset, that contains faces with both upper face occlusion, due to sunglasses, and lower face occlusion, due to masks. We propose two evaluation protocols for this dataset. Benchmark experiments on the dataset have shown that no matter how powerful the deep face representation models are, their performance degrades significantly when they are tested on real-world occluded faces. It is observed that the performance drop is far less when the models are tested on synthetically generated occluded faces. The ROF dataset and the associated evaluation protocols are publicly available at the following link https://github.com/ekremerakin/RealWorldOccludedFaces.

Original language	English
Title of host publication	BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group
Editors	Arslan Bromme, Christoph Busch, Naser Damer, Antitza Dantcheva, Marta Gomez-Barrero, Kiran Raja, Christian Rathgeb, Ana F. Sequeira, Andreas Uhl
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9783885797098
DOIs	https://doi.org/10.1109/BIOSIG52210.2021.9548293
Publication status	Published - Sept 2021
Event	20th International Conference of the Biometrics Special Interest Group, BIOSIG 2021 - Darmstadt, Germany Duration: 15 Sept 2021 → 17 Sept 2021

Publication series

Name	BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group

Conference

Conference	20th International Conference of the Biometrics Special Interest Group, BIOSIG 2021
Country/Territory	Germany
City	Darmstadt
Period	15/09/21 → 17/09/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Funding

This study is supported by the Istanbul Technical University Research Fund, ITU BAP, project no. 42547 and by the Scientific and Technological Research Council of Turkey (TUBITAK) project no. 120N011.

Funders	Funder number
ITU BAP	42547
Istanbul Technical University Research Fund
TUBITAK	120N011
Türkiye Bilimsel ve Teknolojik Araştirma Kurumu

Keywords

deep learning
face occlusion
Face recognition
real-world occluded faces

Access to Document

10.1109/BIOSIG52210.2021.9548293

Cite this

Erakin, M. E., Demir, U., & Ekenel, H. K. (2021). On Recognizing Occluded Faces in the Wild. In A. Bromme, C. Busch, N. Damer, A. Dantcheva, M. Gomez-Barrero, K. Raja, C. Rathgeb, A. F. Sequeira, & A. Uhl (Eds.), BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group (BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BIOSIG52210.2021.9548293

Erakin, Mustafa Ekrem ; Demir, Ugur ; Ekenel, Hazim Kemal. / On Recognizing Occluded Faces in the Wild. BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group. editor / Arslan Bromme ; Christoph Busch ; Naser Damer ; Antitza Dantcheva ; Marta Gomez-Barrero ; Kiran Raja ; Christian Rathgeb ; Ana F. Sequeira ; Andreas Uhl. Institute of Electrical and Electronics Engineers Inc., 2021. (BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group).

@inproceedings{b6db89576293469fb1b4abbc1ad4e036,

title = "On Recognizing Occluded Faces in the Wild",

abstract = "Facial appearance variations due to occlusion has been one of the main challenges for face recognition systems. To facilitate further research in this area, it is necessary and important to have occluded face datasets collected from real-world, as synthetically generated occluded faces cannot represent the nature of the problem. In this paper, we present the Real World Occluded Faces (ROF) dataset, that contains faces with both upper face occlusion, due to sunglasses, and lower face occlusion, due to masks. We propose two evaluation protocols for this dataset. Benchmark experiments on the dataset have shown that no matter how powerful the deep face representation models are, their performance degrades significantly when they are tested on real-world occluded faces. It is observed that the performance drop is far less when the models are tested on synthetically generated occluded faces. The ROF dataset and the associated evaluation protocols are publicly available at the following link https://github.com/ekremerakin/RealWorldOccludedFaces.",

keywords = "deep learning, face occlusion, Face recognition, real-world occluded faces",

author = "Erakin, {Mustafa Ekrem} and Ugur Demir and Ekenel, {Hazim Kemal}",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 20th International Conference of the Biometrics Special Interest Group, BIOSIG 2021 ; Conference date: 15-09-2021 Through 17-09-2021",

year = "2021",

month = sep,

doi = "10.1109/BIOSIG52210.2021.9548293",

language = "English",

series = "BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

editor = "Arslan Bromme and Christoph Busch and Naser Damer and Antitza Dantcheva and Marta Gomez-Barrero and Kiran Raja and Christian Rathgeb and Sequeira, {Ana F.} and Andreas Uhl",

booktitle = "BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group",

address = "United States",

}

Erakin, ME, Demir, U & Ekenel, HK 2021, On Recognizing Occluded Faces in the Wild. in A Bromme, C Busch, N Damer, A Dantcheva, M Gomez-Barrero, K Raja, C Rathgeb, AF Sequeira & A Uhl (eds), BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group. BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group, Institute of Electrical and Electronics Engineers Inc., 20th International Conference of the Biometrics Special Interest Group, BIOSIG 2021, Darmstadt, Germany, 15/09/21. https://doi.org/10.1109/BIOSIG52210.2021.9548293

On Recognizing Occluded Faces in the Wild. / Erakin, Mustafa Ekrem; Demir, Ugur; Ekenel, Hazim Kemal.
BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group. ed. / Arslan Bromme; Christoph Busch; Naser Damer; Antitza Dantcheva; Marta Gomez-Barrero; Kiran Raja; Christian Rathgeb; Ana F. Sequeira; Andreas Uhl. Institute of Electrical and Electronics Engineers Inc., 2021. (BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - On Recognizing Occluded Faces in the Wild

AU - Erakin, Mustafa Ekrem

AU - Demir, Ugur

AU - Ekenel, Hazim Kemal

PY - 2021/9

Y1 - 2021/9

N2 - Facial appearance variations due to occlusion has been one of the main challenges for face recognition systems. To facilitate further research in this area, it is necessary and important to have occluded face datasets collected from real-world, as synthetically generated occluded faces cannot represent the nature of the problem. In this paper, we present the Real World Occluded Faces (ROF) dataset, that contains faces with both upper face occlusion, due to sunglasses, and lower face occlusion, due to masks. We propose two evaluation protocols for this dataset. Benchmark experiments on the dataset have shown that no matter how powerful the deep face representation models are, their performance degrades significantly when they are tested on real-world occluded faces. It is observed that the performance drop is far less when the models are tested on synthetically generated occluded faces. The ROF dataset and the associated evaluation protocols are publicly available at the following link https://github.com/ekremerakin/RealWorldOccludedFaces.

AB - Facial appearance variations due to occlusion has been one of the main challenges for face recognition systems. To facilitate further research in this area, it is necessary and important to have occluded face datasets collected from real-world, as synthetically generated occluded faces cannot represent the nature of the problem. In this paper, we present the Real World Occluded Faces (ROF) dataset, that contains faces with both upper face occlusion, due to sunglasses, and lower face occlusion, due to masks. We propose two evaluation protocols for this dataset. Benchmark experiments on the dataset have shown that no matter how powerful the deep face representation models are, their performance degrades significantly when they are tested on real-world occluded faces. It is observed that the performance drop is far less when the models are tested on synthetically generated occluded faces. The ROF dataset and the associated evaluation protocols are publicly available at the following link https://github.com/ekremerakin/RealWorldOccludedFaces.

KW - deep learning

KW - face occlusion

KW - Face recognition

KW - real-world occluded faces

UR - http://www.scopus.com/inward/record.url?scp=85116686968&partnerID=8YFLogxK

U2 - 10.1109/BIOSIG52210.2021.9548293

DO - 10.1109/BIOSIG52210.2021.9548293

M3 - Conference contribution

AN - SCOPUS:85116686968

T3 - BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group

BT - BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group

A2 - Bromme, Arslan

A2 - Busch, Christoph

A2 - Damer, Naser

A2 - Dantcheva, Antitza

A2 - Gomez-Barrero, Marta

A2 - Raja, Kiran

A2 - Rathgeb, Christian

A2 - Sequeira, Ana F.

A2 - Uhl, Andreas

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 20th International Conference of the Biometrics Special Interest Group, BIOSIG 2021

Y2 - 15 September 2021 through 17 September 2021

ER -

Erakin ME, Demir U, Ekenel HK. On Recognizing Occluded Faces in the Wild. In Bromme A, Busch C, Damer N, Dantcheva A, Gomez-Barrero M, Raja K, Rathgeb C, Sequeira AF, Uhl A, editors, BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group. Institute of Electrical and Electronics Engineers Inc. 2021. (BIOSIG 2021 - Proceedings of the 20th International Conference of the Biometrics Special Interest Group). doi: 10.1109/BIOSIG52210.2021.9548293

On Recognizing Occluded Faces in the Wild

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this