On the first passage g-mean-variance optimality for discounted continuous-time Markov decision processes

Xianping Guo, Xiangxiang Huang, Yi Zhang

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)

Abstract

This paper considers the discounted continuous-time Markov decision processes (MDPs) in Borel spaces and with unbounded transition rates. The discount factors are allowed to depend on states and actions. Main attention is concentrated on the set Fg of stationary policies attaining a given mean performance g up to the first passage of the continuous-time MDP to an arbitrarily fixed target set. Under suitable conditions, we prove the existence of a g-mean-variance optimal policy that minimizes the first passage variance over the set Fg using a transformation technique, and also give the value iteration and policy iteration algorithms for computing the g-variance value function and a g-mean-variance optimal policy, respectively. Two examples are analytically solved to demonstrate the application of our results.

Original languageEnglish
Pages (from-to)1406-1424
Number of pages19
JournalSIAM Journal on Control and Optimization
Volume53
Issue number3
DOIs
Publication statusPublished - 2015

ASJC Scopus subject areas

  • Control and Optimization
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'On the first passage g-mean-variance optimality for discounted continuous-time Markov decision processes'. Together they form a unique fingerprint.

Cite this