Bayesian learning for policy search in trajectory control of a planar manipulator

Vahid Tavakol Aghaei; Arda Agababaoglu; Ahmet Onat; Sinan Yildirim

doi:10.1109/CCWC.2019.8666449

Bayesian learning for policy search in trajectory control of a planar manipulator

Vahid Tavakol Aghaei, Arda Agababaoglu, Ahmet Onat, Sinan Yildirim

Sabanci University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Application of learning algorithms to robotics and control problems with highly nonlinear dynamics to obtain a plausible control policy in a continuous state space is expected to greatly facilitate the design process. Recently, policy search methods such as policy gradient in Reinforcement Learning (RL) have succeeded in coping with such complex systems. Nevertheless, they are slow in convergence speed and are prone to get stuck in local optima. To alleviate this, a Bayesian inference method based on Markov Chain Monte Carlo (MCMC), utilizing a multiplicative reward function, is proposed. This study aims to compare eNAC, a popular gradient based RL method, with the proposed Bayesian learning method, where the objective is trajectory control of a complex model of a 2-DOF planar manipulator. The results obtained for the convergence speed of the proposed algorithm and time response performance, illustrate that the proposed MCMC algorithm is qualified for complex problems in robotics.

Original language	English
Title of host publication	2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019
Editors	Satyajit Chakrabarti, Himadri Nath Saha
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	240-246
Number of pages	7
ISBN (Electronic)	9781728105543
DOIs	https://doi.org/10.1109/CCWC.2019.8666449
Publication status	Published - 12 Mar 2019
Externally published	Yes
Event	9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019 - Las Vegas, United States Duration: 7 Jan 2019 → 9 Jan 2019

Publication series

Name	2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019

Conference

Conference	9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019
Country/Territory	United States
City	Las Vegas
Period	7/01/19 → 9/01/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

Bayesian Learning
Control Systems
Markov Chain Monte Carlo
Reinforcement Learning
Robotics

Access to Document

10.1109/CCWC.2019.8666449

Cite this

Aghaei, V. T., Agababaoglu, A., Onat, A., & Yildirim, S. (2019). Bayesian learning for policy search in trajectory control of a planar manipulator. In S. Chakrabarti, & H. N. Saha (Eds.), 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019 (pp. 240-246). Article 8666449 (2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CCWC.2019.8666449

Aghaei, Vahid Tavakol ; Agababaoglu, Arda ; Onat, Ahmet et al. / Bayesian learning for policy search in trajectory control of a planar manipulator. 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019. editor / Satyajit Chakrabarti ; Himadri Nath Saha. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 240-246 (2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019).

@inproceedings{dc0dd087958545779a7c02776bd736af,

title = "Bayesian learning for policy search in trajectory control of a planar manipulator",

abstract = "Application of learning algorithms to robotics and control problems with highly nonlinear dynamics to obtain a plausible control policy in a continuous state space is expected to greatly facilitate the design process. Recently, policy search methods such as policy gradient in Reinforcement Learning (RL) have succeeded in coping with such complex systems. Nevertheless, they are slow in convergence speed and are prone to get stuck in local optima. To alleviate this, a Bayesian inference method based on Markov Chain Monte Carlo (MCMC), utilizing a multiplicative reward function, is proposed. This study aims to compare eNAC, a popular gradient based RL method, with the proposed Bayesian learning method, where the objective is trajectory control of a complex model of a 2-DOF planar manipulator. The results obtained for the convergence speed of the proposed algorithm and time response performance, illustrate that the proposed MCMC algorithm is qualified for complex problems in robotics.",

keywords = "Bayesian Learning, Control Systems, Markov Chain Monte Carlo, Reinforcement Learning, Robotics",

author = "Aghaei, {Vahid Tavakol} and Arda Agababaoglu and Ahmet Onat and Sinan Yildirim",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019 ; Conference date: 07-01-2019 Through 09-01-2019",

year = "2019",

month = mar,

day = "12",

doi = "10.1109/CCWC.2019.8666449",

language = "English",

series = "2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "240--246",

editor = "Satyajit Chakrabarti and Saha, {Himadri Nath}",

booktitle = "2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019",

address = "United States",

}

Aghaei, VT, Agababaoglu, A, Onat, A & Yildirim, S 2019, Bayesian learning for policy search in trajectory control of a planar manipulator. in S Chakrabarti & HN Saha (eds), 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019., 8666449, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019, Institute of Electrical and Electronics Engineers Inc., pp. 240-246, 9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019, Las Vegas, United States, 7/01/19. https://doi.org/10.1109/CCWC.2019.8666449

Bayesian learning for policy search in trajectory control of a planar manipulator. / Aghaei, Vahid Tavakol; Agababaoglu, Arda; Onat, Ahmet et al.
2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019. ed. / Satyajit Chakrabarti; Himadri Nath Saha. Institute of Electrical and Electronics Engineers Inc., 2019. p. 240-246 8666449 (2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Bayesian learning for policy search in trajectory control of a planar manipulator

AU - Aghaei, Vahid Tavakol

AU - Agababaoglu, Arda

AU - Onat, Ahmet

AU - Yildirim, Sinan

PY - 2019/3/12

Y1 - 2019/3/12

N2 - Application of learning algorithms to robotics and control problems with highly nonlinear dynamics to obtain a plausible control policy in a continuous state space is expected to greatly facilitate the design process. Recently, policy search methods such as policy gradient in Reinforcement Learning (RL) have succeeded in coping with such complex systems. Nevertheless, they are slow in convergence speed and are prone to get stuck in local optima. To alleviate this, a Bayesian inference method based on Markov Chain Monte Carlo (MCMC), utilizing a multiplicative reward function, is proposed. This study aims to compare eNAC, a popular gradient based RL method, with the proposed Bayesian learning method, where the objective is trajectory control of a complex model of a 2-DOF planar manipulator. The results obtained for the convergence speed of the proposed algorithm and time response performance, illustrate that the proposed MCMC algorithm is qualified for complex problems in robotics.

AB - Application of learning algorithms to robotics and control problems with highly nonlinear dynamics to obtain a plausible control policy in a continuous state space is expected to greatly facilitate the design process. Recently, policy search methods such as policy gradient in Reinforcement Learning (RL) have succeeded in coping with such complex systems. Nevertheless, they are slow in convergence speed and are prone to get stuck in local optima. To alleviate this, a Bayesian inference method based on Markov Chain Monte Carlo (MCMC), utilizing a multiplicative reward function, is proposed. This study aims to compare eNAC, a popular gradient based RL method, with the proposed Bayesian learning method, where the objective is trajectory control of a complex model of a 2-DOF planar manipulator. The results obtained for the convergence speed of the proposed algorithm and time response performance, illustrate that the proposed MCMC algorithm is qualified for complex problems in robotics.

KW - Bayesian Learning

KW - Control Systems

KW - Markov Chain Monte Carlo

KW - Reinforcement Learning

KW - Robotics

UR - http://www.scopus.com/inward/record.url?scp=85063880200&partnerID=8YFLogxK

U2 - 10.1109/CCWC.2019.8666449

DO - 10.1109/CCWC.2019.8666449

M3 - Conference contribution

AN - SCOPUS:85063880200

T3 - 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019

SP - 240

EP - 246

BT - 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019

A2 - Chakrabarti, Satyajit

A2 - Saha, Himadri Nath

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 9th IEEE Annual Computing and Communication Workshop and Conference, CCWC 2019

Y2 - 7 January 2019 through 9 January 2019

ER -

Aghaei VT, Agababaoglu A, Onat A, Yildirim S. Bayesian learning for policy search in trajectory control of a planar manipulator. In Chakrabarti S, Saha HN, editors, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 240-246. 8666449. (2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019). doi: 10.1109/CCWC.2019.8666449

Bayesian learning for policy search in trajectory control of a planar manipulator

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this