This paper considers the discounted continuous-time Markov decision processes (MDPs) in Borel spaces and with unbounded transition rates. The discount factors are allowed to depend on states and actions. Main attention is concentrated on the set Fg of stationary policies attaining a given mean performance g up to the first passage of the continuous-time MDP to an arbitrarily fixed target set. Under suitable conditions, we prove the existence of a g-mean-variance optimal policy that minimizes the first passage variance over the set Fg using a transformation technique, and also give the value iteration and policy iteration algorithms for computing the g-variance value function and a g-mean-variance optimal policy, respectively. Two examples are analytically solved to demonstrate the application of our results.
Bibliographical notePublisher Copyright:
© 2015 Society for Industrial and Applied Mathematics.
ASJC Scopus subject areas
- Control and Optimization
- Applied Mathematics