Discounted continuous-time Markov decision processes with unbounded rates: The convex analytic approach

Alexey Piunovskiy, Yi Zhang

Research output: Contribution to journalArticlepeer-review

38 Citations (Scopus)

Abstract

This paper deals with constrained discounted continuous-time Markov decision processes, also known as controlled jump Markov processes, with Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above and below) cost rates, first, we study the space of occupation measures. Then we reformulate the original problem as a linear program over the space of those measures and undertake the duality analysis. Finally, under some compactness-continuity conditions, we show the existence of a stationary optimal policy out of the class of randomized history-dependent policies.

Original languageEnglish
Pages (from-to)2032-2061
Number of pages30
JournalSIAM Journal on Control and Optimization
Volume49
Issue number5
DOIs
Publication statusPublished - 2011

Keywords

  • Borel space
  • Constrained continuous-time Markov decision process
  • Convex analytic approach
  • Duality
  • History-dependent policies
  • Unbounded rates

ASJC Scopus subject areas

  • Control and Optimization
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Discounted continuous-time Markov decision processes with unbounded rates: The convex analytic approach'. Together they form a unique fingerprint.

Cite this