Multimodal soft biometrics: combining ear and face biometrics for age and gender classification

Dogucan Yaman*, Fevziye Irem Eyiokur, Hazım Kemal Ekenel

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

15 Citations (Scopus)

Abstract

In this paper, we present a multimodal, multitask deep convolutional neural network framework for age and gender classification. In the developed framework, we have employed two different biometric modalities: ear and profile face. We have explored three different fusion methods, namely, data, feature, and score fusion, to combine the information extracted from ear and profile face images. In the framework, we have utilized VGG-16 and ResNet-50 models with center loss to obtain more discriminative features. Moreover, we have performed two-stage fine-tuning to increase the representation capacity of the models. To assess the performance of the proposed approach, we have conducted extensive experiments on the FERET, UND-F, and UND-J2 datasets. Experimental results indicate that ear and profile face images contain useful features to extract soft biometric traits. We have shown that when frontal face view of the subject is not available, use of ear and profile face images can be a good alternative for the soft biometric recognition systems. The presented multimodal system achieves very high age and gender classification accuracies, matching the ones obtained by using frontal face images. The multimodal approach has outperformed both the unimodal approaches and the previous state-of-the-art profile face image or ear image-based age and gender classification methods, significantly in both tasks.

Original languageEnglish
Pages (from-to)22695-22713
Number of pages19
JournalMultimedia Tools and Applications
Volume81
Issue number16
DOIs
Publication statusPublished - Jul 2022

Bibliographical note

Publisher Copyright:
© 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature.

Funding

This study is supported by the Istanbul Technical University Research Fund, ITU BAP, project no.42547 and Cost Action CA16101 - MULTI-modal Imaging of FOREnsic SciEnce Evidence - tools for Forensic Science (MULTI-FORESEE). This study is supported by the Istanbul Technical University Research Fund, ITU BAP, project no.42547 and Cost Action CA16101 - MULTI-modal Imaging of FOREnsic SciEnce Evidence - tools for Forensic Science (MULTI-FORESEE).

FundersFunder number
ITU BAP
Istanbul Technical University Research Fund
Bilimsel Araştırma Projeleri Birimi, İstanbul Teknik ÜniversitesiCA16101, 42547

    Keywords

    • Age estimation
    • Convolutional neural networks
    • Gender classification
    • Multimodal learning
    • Multitask learning
    • Soft biometrics

    Fingerprint

    Dive into the research topics of 'Multimodal soft biometrics: combining ear and face biometrics for age and gender classification'. Together they form a unique fingerprint.

    Cite this