A markov chain monte carlo algorithm for bayesian policy search

Vahid Tavakol Aghaei*, Ahmet Onat, Sinan Yıldırım

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)

Abstract

Policy search algorithms have facilitated application of Reinforcement Learning (RL) to dynamic systems, such as control of robots. Many policy search algorithms are based on the policy gradient, and thus may suffer from slow convergence or local optima complications. In this paper, we take a Bayesian approach to policy search under RL paradigm, for the problem of controlling a discrete time Markov decision process with continuous state and action spaces and with a multiplicative reward structure. For this purpose, we assume a prior over policy parameters and aim for the ‘posterior’ distribution where the ‘likelihood’ is the expected reward. We propound a Markov chain Monte Carlo algorithm as a method of generating samples for policy parameters from this posterior. The proposed algorithm is compared with certain well-known policy gradient-based RL methods and exhibits more appropriate performance in terms of time response and convergence rate, when applied to a nonlinear model of a Cart-Pole benchmark.

Original languageEnglish
Pages (from-to)438-455
Number of pages18
JournalSystems Science and Control Engineering
Volume6
Issue number1
DOIs
Publication statusPublished - 1 Jan 2018
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2018 The Author(s).

Keywords

  • Control
  • Markov chain Monte Carlo
  • Particle filtering
  • Policy search
  • Reinforcement learning
  • Risk sensitive reward

Fingerprint

Dive into the research topics of 'A markov chain monte carlo algorithm for bayesian policy search'. Together they form a unique fingerprint.

Cite this