Adaptive controllers and optimal controllers are two distinct methods for the design of automatic control systems. Adaptive controllers learn online in real time how to control systems but do not yield optimal performance, whereas optimal controllers must be designed offline using full knowledge of the systems dynamics. This book shows how approximate dynamic programming - a reinforcement machine learning technique that is motivated by learning mechanisms in biological and animal systems can be used to design a family of adaptive optimal control algorithms that converge in realtime to optimal...