Bayesian learning for policy search in trajectory control of a planar manipulator

Vahid Tavakol Aghaei; Arda Agababaoglu; Ahmet Onat; Sinan Yildirim

doi:10.1109/CCWC.2019.8666449

Bayesian learning for policy search in trajectory control of a planar manipulator

Vahid Tavakol Aghaei
, Arda Agababaoglu
, Ahmet Onat
, Sinan Yildirim

Sabanci University

Araştırma sonucu: Kitap/Rapor/Konferans Bildirisinde Bölüm › Konferans katkısı › bilirkişi

1 Atıf (Scopus)

Özet

Application of learning algorithms to robotics and control problems with highly nonlinear dynamics to obtain a plausible control policy in a continuous state space is expected to greatly facilitate the design process. Recently, policy search methods such as policy gradient in Reinforcement Learning (RL) have succeeded in coping with such complex systems. Nevertheless, they are slow in convergence speed and are prone to get stuck in local optima. To alleviate this, a Bayesian inference method based on Markov Chain Monte Carlo (MCMC), utilizing a multiplicative reward function, is proposed. This study aims to compare eNAC, a popular gradient based RL method, with the proposed Bayesian learning method, where the objective is trajectory control of a complex model of a 2-DOF planar manipulator. The results obtained for the convergence speed of the proposed algorithm and time response performance, illustrate that the proposed MCMC algorithm is qualified for complex problems in robotics.

Orijinal dil	İngilizce
Ana bilgisayar yayını başlığı	2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019
Editörler	Satyajit Chakrabarti, Himadri Nath Saha
Yayınlayan	Institute of Electrical and Electronics Engineers Inc.
Sayfalar	240-246
Sayfa sayısı	7
ISBN (Elektronik)	9781728105543
DOI'lar	https://doi.org/10.1109/CCWC.2019.8666449
Yayın durumu	Yayınlandı - 12 Mar 2019
Harici olarak yayınlandı	Evet
Etkinlik	9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019 - Las Vegas, United States Süre: 7 Oca 2019 → 9 Oca 2019

Yayın serisi

Adı	2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019

???event.eventtypes.event.conference???

???event.eventtypes.event.conference???	9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019
Ülke/Bölge	United States
Şehir	Las Vegas
Periyot	7/01/19 → 9/01/19

Bibliyografik not

Publisher Copyright:
© 2019 IEEE.

Belgeye Erişim

10.1109/CCWC.2019.8666449

Diğer Dosyalar ve Linkler

Link to publication in Scopus

Alıntı Yap

Aghaei, V. T., Agababaoglu, A., Onat, A., & Yildirim, S. (2019). Bayesian learning for policy search in trajectory control of a planar manipulator. In S. Chakrabarti, & H. N. Saha (Eds.), 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019 (pp. 240-246). Makale 8666449 (2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CCWC.2019.8666449

Aghaei, Vahid Tavakol ; Agababaoglu, Arda ; Onat, Ahmet et al. / Bayesian learning for policy search in trajectory control of a planar manipulator. 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019. editör / Satyajit Chakrabarti ; Himadri Nath Saha. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 240-246 (2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019).

@inproceedings{dc0dd087958545779a7c02776bd736af,

title = "Bayesian learning for policy search in trajectory control of a planar manipulator",

abstract = "Application of learning algorithms to robotics and control problems with highly nonlinear dynamics to obtain a plausible control policy in a continuous state space is expected to greatly facilitate the design process. Recently, policy search methods such as policy gradient in Reinforcement Learning (RL) have succeeded in coping with such complex systems. Nevertheless, they are slow in convergence speed and are prone to get stuck in local optima. To alleviate this, a Bayesian inference method based on Markov Chain Monte Carlo (MCMC), utilizing a multiplicative reward function, is proposed. This study aims to compare eNAC, a popular gradient based RL method, with the proposed Bayesian learning method, where the objective is trajectory control of a complex model of a 2-DOF planar manipulator. The results obtained for the convergence speed of the proposed algorithm and time response performance, illustrate that the proposed MCMC algorithm is qualified for complex problems in robotics.",

keywords = "Bayesian Learning, Control Systems, Markov Chain Monte Carlo, Reinforcement Learning, Robotics",

author = "Aghaei, \{Vahid Tavakol\} and Arda Agababaoglu and Ahmet Onat and Sinan Yildirim",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019 ; Conference date: 07-01-2019 Through 09-01-2019",

year = "2019",

month = mar,

day = "12",

doi = "10.1109/CCWC.2019.8666449",

language = "English",

series = "2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "240--246",

editor = "Satyajit Chakrabarti and Saha, \{Himadri Nath\}",

booktitle = "2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019",

address = "United States",

}

Aghaei, VT, Agababaoglu, A, Onat, A & Yildirim, S 2019, Bayesian learning for policy search in trajectory control of a planar manipulator. in S Chakrabarti & HN Saha (eds), 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019., 8666449, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019, Institute of Electrical and Electronics Engineers Inc., pp. 240-246, 9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019, Las Vegas, United States, 7/01/19. https://doi.org/10.1109/CCWC.2019.8666449

Bayesian learning for policy search in trajectory control of a planar manipulator. / Aghaei, Vahid Tavakol; Agababaoglu, Arda; Onat, Ahmet et al.
2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019. ed. / Satyajit Chakrabarti; Himadri Nath Saha. Institute of Electrical and Electronics Engineers Inc., 2019. p. 240-246 8666449 (2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019).

Araştırma sonucu: Kitap/Rapor/Konferans Bildirisinde Bölüm › Konferans katkısı › bilirkişi

TY - GEN

T1 - Bayesian learning for policy search in trajectory control of a planar manipulator

AU - Aghaei, Vahid Tavakol

AU - Agababaoglu, Arda

AU - Onat, Ahmet

AU - Yildirim, Sinan

PY - 2019/3/12

Y1 - 2019/3/12

N2 - Application of learning algorithms to robotics and control problems with highly nonlinear dynamics to obtain a plausible control policy in a continuous state space is expected to greatly facilitate the design process. Recently, policy search methods such as policy gradient in Reinforcement Learning (RL) have succeeded in coping with such complex systems. Nevertheless, they are slow in convergence speed and are prone to get stuck in local optima. To alleviate this, a Bayesian inference method based on Markov Chain Monte Carlo (MCMC), utilizing a multiplicative reward function, is proposed. This study aims to compare eNAC, a popular gradient based RL method, with the proposed Bayesian learning method, where the objective is trajectory control of a complex model of a 2-DOF planar manipulator. The results obtained for the convergence speed of the proposed algorithm and time response performance, illustrate that the proposed MCMC algorithm is qualified for complex problems in robotics.

AB - Application of learning algorithms to robotics and control problems with highly nonlinear dynamics to obtain a plausible control policy in a continuous state space is expected to greatly facilitate the design process. Recently, policy search methods such as policy gradient in Reinforcement Learning (RL) have succeeded in coping with such complex systems. Nevertheless, they are slow in convergence speed and are prone to get stuck in local optima. To alleviate this, a Bayesian inference method based on Markov Chain Monte Carlo (MCMC), utilizing a multiplicative reward function, is proposed. This study aims to compare eNAC, a popular gradient based RL method, with the proposed Bayesian learning method, where the objective is trajectory control of a complex model of a 2-DOF planar manipulator. The results obtained for the convergence speed of the proposed algorithm and time response performance, illustrate that the proposed MCMC algorithm is qualified for complex problems in robotics.

KW - Bayesian Learning

KW - Control Systems

KW - Markov Chain Monte Carlo

KW - Reinforcement Learning

KW - Robotics

UR - https://www.scopus.com/pages/publications/85063880200

U2 - 10.1109/CCWC.2019.8666449

DO - 10.1109/CCWC.2019.8666449

M3 - Conference contribution

AN - SCOPUS:85063880200

T3 - 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019

SP - 240

EP - 246

BT - 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019

A2 - Chakrabarti, Satyajit

A2 - Saha, Himadri Nath

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019

Y2 - 7 January 2019 through 9 January 2019

ER -

Aghaei VT, Agababaoglu A, Onat A, Yildirim S. Bayesian learning for policy search in trajectory control of a planar manipulator. In Chakrabarti S, Saha HN, editörler, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 240-246. 8666449. (2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019). doi: 10.1109/CCWC.2019.8666449

Bayesian learning for policy search in trajectory control of a planar manipulator

Özet

Yayın serisi

???event.eventtypes.event.conference???

Bibliyografik not

Belgeye Erişim

Diğer Dosyalar ve Linkler

Parmak izi

Alıntı Yap