Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme

Esra Ergun; Behcet Ugur Toreyin

doi:10.1109/SIU49456.2020.9302115

Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme

Translated title of the contribution: Continual Learning with Sparse Progressive Neural Networks

Esra Ergun, Behcet Ugur Toreyin

Informatics Institute

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Human brain effectively integrates prior knowledge to new skills by transferring experience across tasks without suffering from catastrophic forgetting. In this study, to continuously learn a visual classication task sequence (PermutedMNIST), we employed a neural network model with lateral connections, sparse group Least Absolute Shrinkage And Selection Operator (LASSO) regularization and projection regularization to decrease feature redundancy. We show that encouraging feature novelty on progressive neural networks (PNN) prevents major performance decrease on sparsication, sparsication of a progressive neural network produces fair results and decreases the number of learned task-specic parameters on novel tasks.

Translated title of the contribution	Continual Learning with Sparse Progressive Neural Networks
Original language	Turkish
Title of host publication	2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728172064
DOIs	https://doi.org/10.1109/SIU49456.2020.9302115
Publication status	Published - 5 Oct 2020
Event	28th Signal Processing and Communications Applications Conference, SIU 2020 - Gaziantep, Turkey Duration: 5 Oct 2020 → 7 Oct 2020

Publication series

Name	2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings

Conference

Conference	28th Signal Processing and Communications Applications Conference, SIU 2020
Country/Territory	Turkey
City	Gaziantep
Period	5/10/20 → 7/10/20

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Access to Document

10.1109/SIU49456.2020.9302115

Cite this

Ergun, E., & Toreyin, B. U. (2020). Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme. In 2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings Article 9302115 (2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SIU49456.2020.9302115

@inproceedings{c66474185bb842689da8f132bad8a54d,

title = "Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme",

abstract = "Human brain effectively integrates prior knowledge to new skills by transferring experience across tasks without suffering from catastrophic forgetting. In this study, to continuously learn a visual classication task sequence (PermutedMNIST), we employed a neural network model with lateral connections, sparse group Least Absolute Shrinkage And Selection Operator (LASSO) regularization and projection regularization to decrease feature redundancy. We show that encouraging feature novelty on progressive neural networks (PNN) prevents major performance decrease on sparsication, sparsication of a progressive neural network produces fair results and decreases the number of learned task-specic parameters on novel tasks.",

keywords = "Catastrophic Forgetting, Continual Learning, Neural Networks, Sparse Neural Networks",

author = "Esra Ergun and Toreyin, {Behcet Ugur}",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 28th Signal Processing and Communications Applications Conference, SIU 2020 ; Conference date: 05-10-2020 Through 07-10-2020",

year = "2020",

month = oct,

day = "5",

doi = "10.1109/SIU49456.2020.9302115",

language = "T{\"u}rk{\c c}e",

series = "2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings",

address = "United States",

}

Ergun, E & Toreyin, BU 2020, Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme. in 2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings., 9302115, 2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 28th Signal Processing and Communications Applications Conference, SIU 2020, Gaziantep, Turkey, 5/10/20. https://doi.org/10.1109/SIU49456.2020.9302115

Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme. / Ergun, Esra ; Toreyin, Behcet Ugur.
2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2020. 9302115 (2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme

AU - Ergun, Esra

AU - Toreyin, Behcet Ugur

PY - 2020/10/5

Y1 - 2020/10/5

N2 - Human brain effectively integrates prior knowledge to new skills by transferring experience across tasks without suffering from catastrophic forgetting. In this study, to continuously learn a visual classication task sequence (PermutedMNIST), we employed a neural network model with lateral connections, sparse group Least Absolute Shrinkage And Selection Operator (LASSO) regularization and projection regularization to decrease feature redundancy. We show that encouraging feature novelty on progressive neural networks (PNN) prevents major performance decrease on sparsication, sparsication of a progressive neural network produces fair results and decreases the number of learned task-specic parameters on novel tasks.

AB - Human brain effectively integrates prior knowledge to new skills by transferring experience across tasks without suffering from catastrophic forgetting. In this study, to continuously learn a visual classication task sequence (PermutedMNIST), we employed a neural network model with lateral connections, sparse group Least Absolute Shrinkage And Selection Operator (LASSO) regularization and projection regularization to decrease feature redundancy. We show that encouraging feature novelty on progressive neural networks (PNN) prevents major performance decrease on sparsication, sparsication of a progressive neural network produces fair results and decreases the number of learned task-specic parameters on novel tasks.

KW - Catastrophic Forgetting

KW - Continual Learning

KW - Neural Networks

KW - Sparse Neural Networks

UR - http://www.scopus.com/inward/record.url?scp=85100317048&partnerID=8YFLogxK

U2 - 10.1109/SIU49456.2020.9302115

DO - 10.1109/SIU49456.2020.9302115

M3 - Konferans katkısı

AN - SCOPUS:85100317048

T3 - 2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings

BT - 2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 28th Signal Processing and Communications Applications Conference, SIU 2020

Y2 - 5 October 2020 through 7 October 2020

ER -

Ergun E , Toreyin BU. Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme. In 2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2020. 9302115. (2020 28th Signal Processing and Communications Applications Conference, SIU 2020 - Proceedings). doi: 10.1109/SIU49456.2020.9302115

Seyrek Ilerlemeli Sinir Aglari ile Surekli Ogrenme

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this