Abstract
This paper deals with constrained discounted continuous-time Markov decision processes, also known as controlled jump Markov processes, with Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above and below) cost rates, first, we study the space of occupation measures. Then we reformulate the original problem as a linear program over the space of those measures and undertake the duality analysis. Finally, under some compactness-continuity conditions, we show the existence of a stationary optimal policy out of the class of randomized history-dependent policies.
Original language | English |
---|---|
Pages (from-to) | 2032-2061 |
Number of pages | 30 |
Journal | SIAM Journal on Control and Optimization |
Volume | 49 |
Issue number | 5 |
DOIs | |
Publication status | Published - 2011 |
Keywords
- Borel space
- Constrained continuous-time Markov decision process
- Convex analytic approach
- Duality
- History-dependent policies
- Unbounded rates
ASJC Scopus subject areas
- Control and Optimization
- Applied Mathematics