This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro- grams; sometimes, particularly when the state space is a countable set, they are also called Markov...
This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discr...
Onesimo Hernandez-Lerma Cnesimo Hernandez-Lerma O. Hernandez-Lerma
This book presents the second part of a two-volume series devoted to a sys- tematic exposition of some recent developments in the theory of discrete- time Markov control processes (MCPs). As in the first part, hereafter re- ferred to as "Volume I" (see Hernandez-Lerma and Lasserre 1]), interest is mainly confined to MCPs with Borel state and control spaces, and possibly unbounded costs. However, an important feature of the present volume is that it is essentially self-contained and can be read independently of Volume I. The reason for this independence is that even though both volumes deal...
This book presents the second part of a two-volume series devoted to a sys- tematic exposition of some recent developments in the theory of discrete- ...
Siegfried Wichmann Onesimo Hernandez-Lerma Jean Bernard Lasserre
This book concerns discrete-time homogeneous Markov chains that admit an invariant probability measure. The main objective is to give a systematic, self-contained presentation on some key issues about the ergodic behavior of that class of Markov chains. These issues include, in particular, the various types of convergence of expected and pathwise occupation measures, and ergodic decompositions of the state space.
This book concerns discrete-time homogeneous Markov chains that admit an invariant probability measure. The main objective is to give a systematic,...
This volume provides the first book entirely devoted to recent developments on the theory and applications of continuous-time Markov decision processes (MDPs). The MDPs presented here include most of the cases that arise in applications.
This volume provides the first book entirely devoted to recent developments on the theory and applications of continuous-time Markov decision processe...
This book is concerned with a class of discrete-time stochastic control processes known as controlled Markov processes (CMP's), also known as Markov decision processes or Markov dynamic programs. Starting in the mid-1950swith Richard Bellman, many contributions to CMP's have been made, and applications to engineering, statistics and operations research, among other areas, have also been developed. The purpose of this book is to present some recent developments on the theory of adaptive CMP's, i. e., CMP's that depend on unknown parameters. Thus at each decision time, the controller or...
This book is concerned with a class of discrete-time stochastic control processes known as controlled Markov processes (CMP's), also known as Markov d...
Devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes, the text is mainly confined to MCPs with Borel state and control spaces. Although the book follows on from the author's earlier work, an important feature of this volume is that it is self-contained and can thus be read independently of the first. The control model studied is sufficiently general to include virtually all the usual discrete-time stochastic control models that appear in applications to engineering, economics, mathematical population processes, operations...
Devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes, the text is mainly confined to...
This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro- grams; sometimes, particularly when the state space is a countable set, they are also called Markov...
This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discr...
This book is about discrete-time, time-homogeneous, Markov chains (Mes) and their ergodic behavior. To this end, most of the material is in fact about stable Mes, by which we mean Mes that admit an invariant probability measure. To state this more precisely and give an overview of the questions we shall be dealing with, we will first introduce some notation and terminology. Let (X, B) be a measurable space, and consider a X-valued Markov chain . = { k' k = 0, 1, ... } with transition probability function (t.pJ.) P(x, B), i.e., P(x, B): = Prob ( k+1 E B I k = x) for each x E X, B E B, and k =...
This book is about discrete-time, time-homogeneous, Markov chains (Mes) and their ergodic behavior. To this end, most of the material is in fact about...