Design of Experiments for Reinforcement Learning

ISBN-13: 9783319121963 / Angielski / Twarda / 2014 / 191 str.

Christopher Gatti

Opis

This thesis takes an empirical approach to understanding of the behavior and interactions between the two main components of reinforcement learning: the learning algorithm and the functional representation of learned knowledge. The author approaches these entities using design of experiments not commonly employed to study machine learning methods. The results outlined in this work provide insight as to what enables and what has an effect on successful reinforcement learning implementations so that this learning method can be applied to more challenging problems.

Spis treści

GLOSSARY ACKNOWLEDGMENT FOREWARD 1. INTRODUCTION 2. REINFORCEMENT LEARNING 2.1 Applications of reinforcement learning 2.1.1 Benchmark problems 2.1.2 Games 2.1.3 Real-world applications 2.1.4 Generalized domains 2.2 Components of reinforcement learning 2.2.1 Domains 2.2.2 Representations 2.2.3 Learning algorithms 2.3 Heuristics and performance effectors 3. DESIGN OF EXPERIMENTS 3.1 Classical design of experiments 3.2 Contemporary design of experiments 3.3 Design of experiments for empirical algorithm analysis 4. METHODOLOGY 4.1 Sequential CART 4.1.1 CART modeling 4.1.2 Sequential CART modeling 4.1.3 Analysis of sequential CART 4.1.4 Empirical convergence criteria 4.1.5 Example: 2-D 6-hump camelback function 4.2 Kriging metamodeling 4.2.1 Kriging 4.2.2 Deterministic kriging 4.2.3 Stochastic kriging 4.2.4 Covariance function 4.2.5 Implementation 4.2.6 Analysis of kriging metamodels 5. THE MOUNTAIN CAR PROBLEM 5.1 Reinforcement learning implementation 5.2 Sequential CART 5.3 Response surface metamodeling 5.4 Discussion 6. THE TRUCK BACKER-UPPER PROBLEM 6.1 Reinforcement learning implementation 6.2 Sequential CART 6.3 Response surface metamodeling 6.4 Discussion 7. THE TANDEM TRUCK BACKER-UPPER PROBLEM 7.1 Reinforcement learning implementation 7.2 Sequential CART 7.3 Discussion 8. DISCUSSION 8.1 Reinforcement learning 8.2 Experimentation 8.3 Innovations 8.4 Future work APPENDICES A. Parameter effects in the game of Chung Toi B. Design of experiments for the mountain car problem C. Supporting tables

O autorze

Christopher Gatti received his PhD in Decision Sciences and Engineering Systems from Rensselaer Polytechnic Institute (RPI). During his time at RPI, his work focused on machine learning and statistics, with applications in reinforcement learning, graph search, stem cell RNA analysis, and neuro-electrophysiological signal analysis. Prior to beginning his graduate work at RPI, he received a BSE in mechanical engineering and an MSE in biomedical engineering, both from the University of Michigan. He then continued to work at the University of Michigan for three years doing computational biomechanics focusing on the shoulder and knee. He has been a gymnast since he was a child and is currently an acrobat for Cirque du Soleil.