Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function

Muhammad Ammar Ali; Yusuf Sahin; Sureyya Özögür-Akyüz; Gozde Ünal; Buse Cisil Otar

doi:10.23919/ELECO54474.2021.9677845

Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function

Muhammad Ammar Ali, Yusuf Sahin, Sureyya Özögür-Akyüz, Gozde Ünal, Buse Cisil Otar

Bahcesehir University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Ensemble methods are used in machine learning by combining several models which produce an optimal predictive model. Neural network ensemble learning is a technique, which uses multiple individual deep neural networks (DNNs). Ensemble pruning methods are used to reduce the computational complexity of ensemble models. In this study, a novel optimization model is proposed to increase error independence in classifiers via entropy measurement and thus better prune the ensemble. An ensemble of 300 DNNs is trained and tested on the CIFAR-10 dataset and results show an increase in accuracy while main-taining a level of relative entropy measured by Kullback-Leibler divergence (KL-divergence).

Original language	English
Title of host publication	2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	365-368
Number of pages	4
ISBN (Electronic)	9786050114379
DOIs	https://doi.org/10.23919/ELECO54474.2021.9677845
Publication status	Published - 2021
Event	13th International Conference on Electrical and Electronics Engineering, ELECO 2021 - Virtual, Bursa, Turkey Duration: 25 Nov 2021 → 27 Nov 2021

Publication series

Name	2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021

Conference

Conference	13th International Conference on Electrical and Electronics Engineering, ELECO 2021
Country/Territory	Turkey
City	Virtual, Bursa
Period	25/11/21 → 27/11/21

Bibliographical note

Publisher Copyright:
© 2021 Chamber of Turkish Electrical Engineers.

Funding

ACKNOWLEDGMENT This study is supported by The Scientific and Technological Research Council Of Turkey (TUBITAK) Project No: 119E100.

Funders	Funder number
TUBITAK	119E100
Türkiye Bilimsel ve Teknolojik Araştirma Kurumu

Access to Document

10.23919/ELECO54474.2021.9677845

Cite this

Ali, M. A., Sahin, Y., Özögür-Akyüz, S., Ünal, G., & Otar, B. C. (2021). Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function. In 2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021 (pp. 365-368). (2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/ELECO54474.2021.9677845

Ali, Muhammad Ammar ; Sahin, Yusuf ; Özögür-Akyüz, Sureyya et al. / Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function. 2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021. Institute of Electrical and Electronics Engineers Inc., 2021. pp. 365-368 (2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021).

@inproceedings{b4eb7d91d22448b1bbfaa3f2599e2bbf,

title = "Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function",

abstract = "Ensemble methods are used in machine learning by combining several models which produce an optimal predictive model. Neural network ensemble learning is a technique, which uses multiple individual deep neural networks (DNNs). Ensemble pruning methods are used to reduce the computational complexity of ensemble models. In this study, a novel optimization model is proposed to increase error independence in classifiers via entropy measurement and thus better prune the ensemble. An ensemble of 300 DNNs is trained and tested on the CIFAR-10 dataset and results show an increase in accuracy while main-taining a level of relative entropy measured by Kullback-Leibler divergence (KL-divergence).",

author = "Ali, {Muhammad Ammar} and Yusuf Sahin and Sureyya {\"O}z{\"o}g{\"u}r-Aky{\"u}z and Gozde {\"U}nal and Otar, {Buse Cisil}",

note = "Publisher Copyright: {\textcopyright} 2021 Chamber of Turkish Electrical Engineers.; 13th International Conference on Electrical and Electronics Engineering, ELECO 2021 ; Conference date: 25-11-2021 Through 27-11-2021",

year = "2021",

doi = "10.23919/ELECO54474.2021.9677845",

language = "English",

series = "2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "365--368",

booktitle = "2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021",

address = "United States",

}

Ali, MA, Sahin, Y, Özögür-Akyüz, S, Ünal, G & Otar, BC 2021, Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function. in 2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021. 2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021, Institute of Electrical and Electronics Engineers Inc., pp. 365-368, 13th International Conference on Electrical and Electronics Engineering, ELECO 2021, Virtual, Bursa, Turkey, 25/11/21. https://doi.org/10.23919/ELECO54474.2021.9677845

Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function. / Ali, Muhammad Ammar; Sahin, Yusuf; Özögür-Akyüz, Sureyya et al.
2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 365-368 (2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function

AU - Ali, Muhammad Ammar

AU - Sahin, Yusuf

AU - Özögür-Akyüz, Sureyya

AU - Ünal, Gozde

AU - Otar, Buse Cisil

PY - 2021

Y1 - 2021

N2 - Ensemble methods are used in machine learning by combining several models which produce an optimal predictive model. Neural network ensemble learning is a technique, which uses multiple individual deep neural networks (DNNs). Ensemble pruning methods are used to reduce the computational complexity of ensemble models. In this study, a novel optimization model is proposed to increase error independence in classifiers via entropy measurement and thus better prune the ensemble. An ensemble of 300 DNNs is trained and tested on the CIFAR-10 dataset and results show an increase in accuracy while main-taining a level of relative entropy measured by Kullback-Leibler divergence (KL-divergence).

AB - Ensemble methods are used in machine learning by combining several models which produce an optimal predictive model. Neural network ensemble learning is a technique, which uses multiple individual deep neural networks (DNNs). Ensemble pruning methods are used to reduce the computational complexity of ensemble models. In this study, a novel optimization model is proposed to increase error independence in classifiers via entropy measurement and thus better prune the ensemble. An ensemble of 300 DNNs is trained and tested on the CIFAR-10 dataset and results show an increase in accuracy while main-taining a level of relative entropy measured by Kullback-Leibler divergence (KL-divergence).

UR - http://www.scopus.com/inward/record.url?scp=85125232658&partnerID=8YFLogxK

U2 - 10.23919/ELECO54474.2021.9677845

DO - 10.23919/ELECO54474.2021.9677845

M3 - Conference contribution

AN - SCOPUS:85125232658

T3 - 2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021

SP - 365

EP - 368

BT - 2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 13th International Conference on Electrical and Electronics Engineering, ELECO 2021

Y2 - 25 November 2021 through 27 November 2021

ER -

Ali MA, Sahin Y, Özögür-Akyüz S, Ünal G, Otar BC. Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function. In 2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021. Institute of Electrical and Electronics Engineers Inc. 2021. p. 365-368. (2021 13th International Conference on Electrical and Electronics Engineering, ELECO 2021). doi: 10.23919/ELECO54474.2021.9677845

Tuning Accuracy-Diversity Trade-off in Neural Network Ensemble via Novel Entropy Loss Function

Abstract

Publication series

Conference

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this