Abstract
In this paper, we present a multimodal, multitask deep convolutional neural network framework for age and gender classification. In the developed framework, we have employed two different biometric modalities: ear and profile face. We have explored three different fusion methods, namely, data, feature, and score fusion, to combine the information extracted from ear and profile face images. In the framework, we have utilized VGG-16 and ResNet-50 models with center loss to obtain more discriminative features. Moreover, we have performed two-stage fine-tuning to increase the representation capacity of the models. To assess the performance of the proposed approach, we have conducted extensive experiments on the FERET, UND-F, and UND-J2 datasets. Experimental results indicate that ear and profile face images contain useful features to extract soft biometric traits. We have shown that when frontal face view of the subject is not available, use of ear and profile face images can be a good alternative for the soft biometric recognition systems. The presented multimodal system achieves very high age and gender classification accuracies, matching the ones obtained by using frontal face images. The multimodal approach has outperformed both the unimodal approaches and the previous state-of-the-art profile face image or ear image-based age and gender classification methods, significantly in both tasks.
Original language | English |
---|---|
Pages (from-to) | 22695-22713 |
Number of pages | 19 |
Journal | Multimedia Tools and Applications |
Volume | 81 |
Issue number | 16 |
DOIs | |
Publication status | Published - Jul 2022 |
Bibliographical note
Publisher Copyright:© 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature.
Funding
This study is supported by the Istanbul Technical University Research Fund, ITU BAP, project no.42547 and Cost Action CA16101 - MULTI-modal Imaging of FOREnsic SciEnce Evidence - tools for Forensic Science (MULTI-FORESEE). This study is supported by the Istanbul Technical University Research Fund, ITU BAP, project no.42547 and Cost Action CA16101 - MULTI-modal Imaging of FOREnsic SciEnce Evidence - tools for Forensic Science (MULTI-FORESEE).
Funders | Funder number |
---|---|
ITU BAP | |
Istanbul Technical University Research Fund | |
Bilimsel Araştırma Projeleri Birimi, İstanbul Teknik Üniversitesi | CA16101, 42547 |
Keywords
- Age estimation
- Convolutional neural networks
- Gender classification
- Multimodal learning
- Multitask learning
- Soft biometrics