This book concerns two broad classes of Markov decision problems that fall within the family of "restless bandits", an intractable class of decision processes introduced by P. Whittle. For the first class of problems, we employ Gittins index theory to establish the indexability of (inter- alia) general families of restless bandits that arise in problems of machine maintenance and stochastic scheduling with switching penalties. We also give formulae for the resulting indices. The second class of problems involves two families of Markov decision problems that can be identified as...
This book concerns two broad classes of Markov decision problems that fall within the family of "restless bandits", an intractable class of decision p...
