Continuous-Time Markov Decision Processes

Onésimo Hernández-Lerma received the Science and Arts National Award from the Government of MEXICO in 2001, an honorary doctorate from the University of Sonora in 2003, and the Scopus Prize from Elsevier in 2008. Xianping Guo received the He-Pan-Qing-Yi Best Paper Award from the 7th Word Congress on Intelligent Control and Automation in 2008.

• and Summary.- Continuous-Time Markov Decision Processes.- Average Optimality for Finite Models.- Discount Optimality for Nonnegative Costs.- Average Optimality for Nonnegative Costs.- Discount Optimality for Unbounded Rewards.- Average Optimality for Unbounded Rewards.- Average Optimality for Pathwise Rewards.- Advanced Optimality Criteria.- Variance Minimization.- Constrained Optimality for Discount Criteria.- Constrained Optimality for Average Criteria.

From the reviews: "The book consists of 12 chapters. ... this is the first monograph on continuous-time Markov decision process. ... This is an important book written by leading experts on a mathematically rich topic which has many applications to engineering, business, and biological problems. ... scholars and students interested in developing the theory of continuous-time Markov decision Processes or working on their applications should have this book." (E. A. Feinberg, Mathematical Reviews, Issue 2011 b)