Deep Reinforcement Learning: Frontiers of Artificial Intelligence

ISBN-13: 9789811382840 / Angielski / Twarda / 2019 / 203 str.

Mohit Sewak

Spis treści

​Chapter 1: Introduction to Reinforcement Learning

Chapter Goal: In this chapter the reader will learn about the basic building blocks of Reinforcement Learning like state, actor, action and policy, and some traditional approaches of RL including Dynamic Programming and Monte Carlo. The readers will also learn the difference between On-Policy and Off-Policy methods and when to use which one. The chapter also covers some advanced non-Deep RL techniques. The chapter ends with some hands-on experience of coding some of the stated techniques. 

No of pages : 30 

Chapter 2: Introduction to Deep Learning

Chapter Goal: In this chapter the readers will understand the building blocks of Deep Learning, and some popular types like DNN, RNN and CNN. The chapter also covers some example application of DL in areas like language modeling and image classification, and the challenges with using these techniques, where RL could help. 

No of pages: 25 

Chapter 3:  Introduction to Open AI Gym

Chapter Goal: In this chapter, the readers will learn about the Open AI Gym. We would explore different games and sections within Open AI gym, and then would also explore the Open AI Gym Python API for the important games and section in Open AI Gym. The chapter will be concluded by coding an actual RL application in Open AI Gym using Python. 

No of pages: 10 

Chapter 3: Temporal Difference Learning and TD-Lambda

Chapter Goal: This chapter begins with introducing the readers to the very effective, and highly in demand algorithm called Temporal Difference Learning, in its simplest form. To draw some intuition, we would cover RL algorithms like SARSA in brief and how does TD Learning compare with it. The chapter then builds upon these concepts, to introduce the advance TD-Lambda formulation of TD Learning which aims to combine insights across difference steps to choose the next action. The chapter concludes with giving the reader some hands-on experience of coding some of the discussed TD algorithm in Python. 

No of pages: 30 

Chapter 4: Attention Mechanism and Reinforcement Learning

Chapter Goal: This chapter steps into the realm of combining both Deep Learning and Reinforcement Learning, to power many of the most powerful applications and real-life business use-cases. Much of discussions for this chapter will have its roots into cutting-edge research and the innovative applications. The chapter will start with introducing the problems with DL and how Attention Mechanism can solve it. We would next discuss the types of Attention, and then the role of RL in Attention, especially in Hard Attention. The chapter also covers use cases in Advanced NLP and Image Captioning with the role of Attention and RL in them. 

No of pages: 25 

Chapter 5: Deep Q Learning

Chapter Goal: In the previous chapter we discovered how RL helps Deep Learning networks, especially the one that could benefit using Attention Mechanism, in this chapter we will reverse the role and will discuss how Deep Learning could take RL to the next step and answer many challenges that we highlighted earlier. It starts with the introduction to Policy Optimization and the Q Learning method. From there the chapter logically leads into Deep Q Learning while combining insights from the earlier covered chapter on Deep Learning, and earlier section on Q-Learning. The chapter next covers some of the active research in this area and advancements happening in this very powerful RL technique. 

No of pages: 25 

Chapter 6: Applying Deep Q Learning in Code

Chapter Goal: This chapter covers step by step the coding of Deep Q Learning, in a modular and Object-Oriented manner, covering class by class, explaining the complicated parts of the code in detail. 

No of pages: 15 

Chapter 7: Asynchronous Advantage Actor Critic Model

Chapter Goal: This chapter introduces the readers to the future of Artificial Intelligence and Reinforcement Learning with the Asynchronous Advantage Actor-Critic Model. This is a fairly new model with much of the recent and advanced researches being done in recent times, and a very powerful one as well, as this is the model behind the most successful applications from Google Deep Mind. But at the same time, this model is also very complex to understand and more complicated to implement. This chapter tries to take a lot of example and intuitions to explain the concepts of Actor-Critic models and then slowly moves to explaining and then mastering the A3C model. 

No of pages: 25 

Chapter 8: Applying A3C Model in Code

Chapter Goal: This chapter covers step by step the coding of Deep Q Learning, in a modular and Object-Oriented manner, covering class by class, explaining the complicated parts of the code in detail. 

No of pages: 15

O autorze

Mr. Sewak has been the Lead Data Scientist/Analytics Architect for a number of important international AI/DL/ML software and industry solutions and has also been involved in providing solutions and research for a series of cognitive features for IBM Watson Commerce. He has 14 years of experience working as a solutions architect using technologies like TensorFlow, Torch, Caffe, Theano, Keras, Open AI, SpaCy, Gensim, NLTK, Watson, SPSS, Spark, H2O, Kafka, ES, and others.

Blurb

This book starts by presenting the basics of reinforcement learning using highly intuitive and easy-to-understand examples and applications, and then introduces the cutting-edge research advances that make reinforcement learning capable of out-performing most state-of-art systems, and even humans in a number of applications. The book not only equips readers with an understanding of multiple advanced and innovative algorithms, but also prepares them to implement systems such as those created by Google Deep Mind in actual code.

 This book is intended for readers who want to both understand and apply advanced concepts in a field that combines the best of two worlds – deep learning and reinforcement learning – to tap the potential of ‘advanced artificial intelligence’ for creating real-world applications and game-winning algorithms.