Partially Observable MDPS (POMDPS): Introduction and Examples

A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process where the states of the model are not completely observable by the decision maker. Noisy observations provide a belief regarding the underlying state, while the decision maker has some control over the progression of the model through the selection of actions. In this article, we introduce POMDPs and discuss the relationship between Markov models and POMDPs. A general POMDP formulation and a wide range of POMDP applications from the literature are also presented.