This book presents an algebraic development of the theory of countable state space Markov chains with discrete- and continuous-time parameters. A Markov chain is a stochastic process characterized by the Markov prop- erty that the distribution of future depends only on the current state, not on the whole history. Despite its simple form of dependency, the Markov property has enabled us to develop a rich system of concepts and theorems and to derive many results that are useful in applications. In fact, the areas that can be modeled, with varying degrees of success, by Markov chains are vast...