Automated Deep Learning Using Neural Network Intelligence: Develop and Design Pytorch and Tensorflow Models Using Python

ISBN-13: 9781484281482 / Angielski / Miękka / 2022

Ivan Gridin

Opis

Optimize, develop, and design PyTorch and TensorFlow models for a specific problem using the Microsoft Neural Network Intelligence (NNI) toolkit. This book includes practical examples illustrating automated deep learning approaches and provides techniques to facilitate your deep learning model development.

The first chapters of this book cover the basics of NNI toolkit usage and methods for solving hyper-parameter optimization tasks. You will understand the black-box function maximization problem using NNI, and know how to prepare a TensorFlow or PyTorch model for hyper-parameter tuning, launch an experiment, and interpret the results. The book dives into optimization tuners and the search algorithms they are based on: Evolution search, Annealing search, and the Bayesian Optimization approach. The Neural Architecture Search is covered and you will learn how to develop deep learning models from scratch. Multi-trial and one-shot searching approaches of automatic neural network design are presented. The book teaches you how to construct a search space and launch an architecture search using the latest state-of-the-art exploration strategies: Efficient Neural Architecture Search (ENAS) and Differential Architectural Search (DARTS). You will learn how to automate the construction of a neural network architecture for a particular problem and dataset. The book focuses on model compression and feature engineering methods that are essential in automated deep learning. It also includes performance techniques that allow the creation of large-scale distributive training platforms using NNI.

After reading this book, you will know how to use the full toolkit of automated deep learning methods. The techniques and practical examples presented in this book will allow you to bring your neural network routines to a higher level.

What You Will Learn

• Know the basic concepts of optimization tuners, search space, and trials
• Apply different hyper-parameter optimization algorithms to develop effective neural networks
• Construct new deep learning models from scratch
• Execute the automated Neural Architecture Search to create state-of-the-art deep learning models
• Compress the model to eliminate unnecessary deep learning layers

Who This Book Is For 

Intermediate to advanced data scientists and machine learning engineers involved in deep learning and practical neural network development

Intermediate-Advanced user level

Spis treści

Chapter 1: Introduction to Neural Network Intelligence

1.1 Installation

1.2 Trial, search space, experiment

1.3 Finding maxima of multivariate function

1.4 Interacting with NNI

Chapter 2:Hyper-Parameter Tuning

2.1 Preparing a model for hyper-parameter tuning

2.2 Running experiment

2.3 Interpreting results

2.4 Debugging

Chapter 3:  Hyper-Parameter Tuners

Chapter 4:  Neural Architecture Search: Multi-trial

4.1 Constructing a search space

4.2 Running architecture search

4.3 Exploration strategies

4.4 Comparing exploration strategies

Chapter 5: Neural Architecture Search: One-shot

5.1 What is one-shot NAS?

5.2 ENAS

5.3 DARTS

Chapter 6: Model Compression

6.1 What is model compression?

6.2 Compressing your model

6.3 Pruning

6.4 Quantization

Chapter 7: Advanced NNI

O autorze

Ivan Gridin is a machine learning expert from Moscow who has worked on distributive high-load systems and implemented different machine learning approaches in practice. One of the primary areas of his research is the design and analysis of predictive time series models. Ivan has fundamental math skills in probability theory, random process theory, time series analysis, machine learning, deep learning, and optimization. He has published books on genetic algorithms and time series analysis.

Blurb

Optimize, develop, and design PyTorch and TensorFlow models for a specific problem using the Microsoft Neural Network Intelligence (NNI) toolkit. This book includes practical examples illustrating automated deep learning approaches and provides techniques to facilitate your deep learning model development.

The first chapters of this book cover the basics of NNI toolkit usage and methods for solving hyper-parameter optimization tasks. You will understand the black-box function maximization problem using NNI, and know how to prepare a TensorFlow or PyTorch model for hyper-parameter tuning, launch an experiment, and interpret the results. The book dives into optimization tuners and the search algorithms they are based on: Evolution search, Annealing search, and the Bayesian Optimization approach. The Neural Architecture Search is covered and you will learn how to develop deep learning models from scratch. Multi-trial and one-shot searching approaches of automatic neural network design are presented. The book teaches you how to construct a search space and launch an architecture search using the latest state-of-the-art exploration strategies: Efficient Neural Architecture Search (ENAS) and Differential Architectural Search (DARTS). You will learn how to automate the construction of a neural network architecture for a particular problem and dataset. The book focuses on model compression and feature engineering methods that are essential in automated deep learning. It also includes performance techniques that allow the creation of large-scale distributive training platforms using NNI.

After reading this book, you will know how to use the full toolkit of automated deep learning methods. The techniques and practical examples presented in this book will allow you to bring your neural network routines to a higher level.

What You Will Learn

• Know the basic concepts of optimization tuners, search space, and trials
• Apply different hyper-parameter optimization algorithms to develop effective neural networks
• Construct new deep learning models from scratch
• Execute the automated Neural Architecture Search to create state-of-the-art deep learning models
• Compress the model to eliminate unnecessary deep learning layers