This book provides tools and algorithms for solving a wide class of optimization tasks by learning from their repetitions. A unified framework is provided for learning algorithms that are based on the stochastic gradient (a golden standard in learning), including random simultaneous perturbations and the response surface the methodology. Original algorithms include model-free learning of short decision sequences as well as long sequences—relying on model-supported gradient estimation. Learning is based on whole sequences of a process observation that are either vectors or images. This...