Interval type-2 fuzzy systems as deep neural network activation functions

Aykut Beke; Tufan Kumbasar

Interval type-2 fuzzy systems as deep neural network activation functions

Aykut Beke
, Tufan Kumbasar

Kontrol ve Otomasyon Mühendisliği Bölümü

Istanbul Technical University

Araştırma çıktısı: Kitap/Rapor/Konferans Bildirisinde Bölüm › Konferans katkısı › Hakem

3 Atıf (Scopus)

Özet

In this paper, we propose a novel activation function, namely, Interval Type-2 (IT2) Fuzzy Rectifying Unit (FRU), to improve the performance of the Deep Neural Networks (DNNs). The IT2-FRU can generate linear or sophisticated activation functions by simply tuning the size of the footprint of uncertainty of the IT2 Fuzzy Sets. The novel IT2-FRU also alleviates vanishing gradient problem and has a fast convergence rate since it pushes the mean activation to zero by allowing the negative outputs. In order to test the performance of the IT2-FRU, comparative experimental studies are performed on the CIFAR-10 dataset. IT2-FRU is compared with widely used conventional activation functions. Experimental results show that IT2-FRU significantly speeds up the learning and has a superior performance compared to other handled activation functions.

Orijinal dil	İngilizce
Ana bilgisayar yayını başlığı	Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019
Editörler	Vilem Novak, Vladimir Marik, Martin Stepnicka, Mirko Navara, Petr Hurtik
Yayınlayan	Atlantis Press
Sayfalar	267-273
Sayfa sayısı	7
ISBN (Elektronik)	9789462527706
Yayın durumu	Yayınlandı - 2020
Etkinlik	11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019 - Prague, Czech Republic Süre: 9 Eyl 2019 → 13 Eyl 2019

Yayın serisi

Adı	Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019

???event.eventtypes.event.conference???

???event.eventtypes.event.conference???	11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019
Ülke/Bölge	Czech Republic
Şehir	Prague
Periyot	9/09/19 → 13/09/19

Bibliyografik not

Publisher Copyright:
Copyright © 2019, the Authors. Published by Atlantis Press. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Finansman

This research is supported by the project (118E807) of Scientific and Technological Research Council of Turkey (TUBITAK). All of these supports are appreciated.

Finansörler
TUBITAK
Türkiye Bilimsel ve Teknolojik Araştirma Kurumu

Diğer Dosyalar ve Linkler

Link to publication in Scopus

Alıntı Yap

Beke, A., & Kumbasar, T. (2020). Interval type-2 fuzzy systems as deep neural network activation functions. In V. Novak, V. Marik, M. Stepnicka, M. Navara, & P. Hurtik (Eds.), Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019 (pp. 267-273). (Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019). Atlantis Press.

Beke, Aykut ; Kumbasar, Tufan. / Interval type-2 fuzzy systems as deep neural network activation functions. Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019. editör / Vilem Novak ; Vladimir Marik ; Martin Stepnicka ; Mirko Navara ; Petr Hurtik. Atlantis Press, 2020. pp. 267-273 (Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019).

@inproceedings{c2f5a938611545c1b718311a4fa823d9,

title = "Interval type-2 fuzzy systems as deep neural network activation functions",

abstract = "In this paper, we propose a novel activation function, namely, Interval Type-2 (IT2) Fuzzy Rectifying Unit (FRU), to improve the performance of the Deep Neural Networks (DNNs). The IT2-FRU can generate linear or sophisticated activation functions by simply tuning the size of the footprint of uncertainty of the IT2 Fuzzy Sets. The novel IT2-FRU also alleviates vanishing gradient problem and has a fast convergence rate since it pushes the mean activation to zero by allowing the negative outputs. In order to test the performance of the IT2-FRU, comparative experimental studies are performed on the CIFAR-10 dataset. IT2-FRU is compared with widely used conventional activation functions. Experimental results show that IT2-FRU significantly speeds up the learning and has a superior performance compared to other handled activation functions.",

keywords = "Activation unit, Deep learning, Footprint of Uncertainty, Interval type-2 fuzzy system",

author = "Aykut Beke and Tufan Kumbasar",

note = "Publisher Copyright: Copyright {\textcopyright} 2019, the Authors. Published by Atlantis Press. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).; 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019 ; Conference date: 09-09-2019 Through 13-09-2019",

year = "2020",

language = "English",

series = "Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019",

publisher = "Atlantis Press",

pages = "267--273",

editor = "Vilem Novak and Vladimir Marik and Martin Stepnicka and Mirko Navara and Petr Hurtik",

booktitle = "Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019",

}

Beke, A & Kumbasar, T 2020, Interval type-2 fuzzy systems as deep neural network activation functions. in V Novak, V Marik, M Stepnicka, M Navara & P Hurtik (eds), Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019. Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019, Atlantis Press, pp. 267-273, 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019, Prague, Czech Republic, 9/09/19.

Interval type-2 fuzzy systems as deep neural network activation functions. / Beke, Aykut; Kumbasar, Tufan.
Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019. ed. / Vilem Novak; Vladimir Marik; Martin Stepnicka; Mirko Navara; Petr Hurtik. Atlantis Press, 2020. p. 267-273 (Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019).

Araştırma çıktısı: Kitap/Rapor/Konferans Bildirisinde Bölüm › Konferans katkısı › Hakem

TY - GEN

T1 - Interval type-2 fuzzy systems as deep neural network activation functions

AU - Beke, Aykut

AU - Kumbasar, Tufan

PY - 2020

Y1 - 2020

N2 - In this paper, we propose a novel activation function, namely, Interval Type-2 (IT2) Fuzzy Rectifying Unit (FRU), to improve the performance of the Deep Neural Networks (DNNs). The IT2-FRU can generate linear or sophisticated activation functions by simply tuning the size of the footprint of uncertainty of the IT2 Fuzzy Sets. The novel IT2-FRU also alleviates vanishing gradient problem and has a fast convergence rate since it pushes the mean activation to zero by allowing the negative outputs. In order to test the performance of the IT2-FRU, comparative experimental studies are performed on the CIFAR-10 dataset. IT2-FRU is compared with widely used conventional activation functions. Experimental results show that IT2-FRU significantly speeds up the learning and has a superior performance compared to other handled activation functions.

AB - In this paper, we propose a novel activation function, namely, Interval Type-2 (IT2) Fuzzy Rectifying Unit (FRU), to improve the performance of the Deep Neural Networks (DNNs). The IT2-FRU can generate linear or sophisticated activation functions by simply tuning the size of the footprint of uncertainty of the IT2 Fuzzy Sets. The novel IT2-FRU also alleviates vanishing gradient problem and has a fast convergence rate since it pushes the mean activation to zero by allowing the negative outputs. In order to test the performance of the IT2-FRU, comparative experimental studies are performed on the CIFAR-10 dataset. IT2-FRU is compared with widely used conventional activation functions. Experimental results show that IT2-FRU significantly speeds up the learning and has a superior performance compared to other handled activation functions.

KW - Activation unit

KW - Deep learning

KW - Footprint of Uncertainty

KW - Interval type-2 fuzzy system

UR - https://www.scopus.com/pages/publications/85090901716

M3 - Conference contribution

AN - SCOPUS:85090901716

T3 - Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019

SP - 267

EP - 273

BT - Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019

A2 - Novak, Vilem

A2 - Marik, Vladimir

A2 - Stepnicka, Martin

A2 - Navara, Mirko

A2 - Hurtik, Petr

PB - Atlantis Press

T2 - 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019

Y2 - 9 September 2019 through 13 September 2019

ER -

Beke A, Kumbasar T. Interval type-2 fuzzy systems as deep neural network activation functions. In Novak V, Marik V, Stepnicka M, Navara M, Hurtik P, editörler, Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019. Atlantis Press. 2020. p. 267-273. (Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology, EUSFLAT 2019).

Interval type-2 fuzzy systems as deep neural network activation functions

Özet

Yayın serisi

???event.eventtypes.event.conference???

Bibliyografik not

Finansman

Diğer Dosyalar ve Linkler

Parmak izi

Alıntı Yap