This book serves as a standard reference, making this area accessible not only to researchers in probability and statistics, but also to graduate students and practitioners. The book assumes only a first-year graduate course in probability. Each chapter begins with a brief overview and concludes with a wide range of exercises at varying levels of difficulty. The authors supply detailed hints for the more challenging problems, and cover many advances made in recent years.

This book provides a unified approach for the study of constrained Markov decision processes with a finite state space and unbounded costs. Unlike the single controller case considered in many other books, the author considers a single controller with several objectives, such as minimizing delays and loss, probabilities, and maximization of throughputs. It is desirable to design a controller that minimizes one cost objective, subject to inequality constraints on other cost objectives. This framework describes dynamic decision problems arising frequently in many engineering fields. A thorough...