Note on discounted continuous-time Markov decision processes with a lower bounding function

Xin Guo*, Alexey Piunovskiy, Yi Zhang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

We consider the discounted continuous-time Markov decision process (CTMDP), where the negative part of each cost rate is bounded by a drift function, say w, whereas the positive part is allowed to be arbitrarily unbounded. Our focus is on the existence of a stationary optimal policy for the discounted CTMDP problems out of the more general class. Both constrained and unconstrained problems are considered. Our investigations are based on the continuous-time version of the Veinott transformation. This technique has not been widely employed in the previous literature on CTMDPs, but it clarifies the roles of the imposed conditions in a rather transparent way.

Original languageEnglish
Pages (from-to)1071-1088
Number of pages18
JournalJournal of Applied Probability
Volume54
Issue number4
DOIs
Publication statusPublished - 1 Dec 2017

Bibliographical note

Publisher Copyright:
Copyright © Applied Probability Trust 2017.

Keywords

  • Continuous-time Markov decision process
  • discounted criterion

ASJC Scopus subject areas

  • Statistics and Probability
  • Mathematics(all)
  • Statistics, Probability and Uncertainty

Fingerprint

Dive into the research topics of 'Note on discounted continuous-time Markov decision processes with a lower bounding function'. Together they form a unique fingerprint.

Cite this