This book presents algorithms for constrained and unconstrained optimization and for reinforcement learning. These are demonstrated in a wide range of applications including service systems, vehicular traffic control, communications networks and more. The breadth of applications makes the book appropriate for reader from similarly diverse backgrounds: workers in relevant areas of computer science, control engineering, management science, applied mathematics, industrial engineering and operations research will find the content of value.