Advanced Reinforcement Learning in Python: cutting-edge DQNs

Course Curriculum

Chapter 1: Introduction

Lecture 1: Introduction

Lecture 2: Reinforcement Learning series

Lecture 3: Google Colab

Lecture 4: Where to begin

Lecture 5: Complete code

Lecture 6: Connect with me on social media

Chapter 2: Refresher: The Markov Decision Process (MDP)

Lecture 1: Module overview

Lecture 2: Elements common to all control tasks

Lecture 3: The Markov decision process (MDP)

Lecture 4: Types of Markov decision process

Lecture 5: Trajectory vs episode

Lecture 6: Reward vs Return

Lecture 7: Discount factor

Lecture 8: Policy

Lecture 9: State values v(s) and action values q(s,a)

Lecture 10: Bellman equations

Lecture 11: Solving a Markov decision process

Chapter 3: Refresher: Q-Learning

Lecture 1: Module overview

Lecture 2: Temporal difference methods

Lecture 3: Solving control tasks with temporal difference method

Lecture 4: Q-Learning

Lecture 5: Advantages of temporal difference methods

Chapter 4: Refresher: Brief introduction to Neural Networks

Lecture 1: Module overview

Lecture 2: Function approximators

Lecture 3: Artificial Neural Networks

Lecture 4: Artificial Neurons

Lecture 5: How to represent a Neural Network

Lecture 6: Stochastic Gradient Descent

Lecture 7: Neural Network optimization

Chapter 5: Refresher: Deep Q-Learning

Lecture 1: Module overview

Lecture 2: Deep Q-Learning

Lecture 3: Experience replay

Lecture 4: Target Network

Chapter 6: PyTorch Lightning

Lecture 1: PyTorch Lightning

Lecture 2: Link to the code notebook

Lecture 3: Introduction to PyTorch Lightning

Lecture 4: Create the Deep Q-Network

Lecture 5: Create the policy

Lecture 6: Create the replay buffer

Lecture 7: Create the environment

Lecture 8: Define the class for the Deep Q-Learning algorithm

Lecture 9: Define the play_episode() function

Lecture 10: Prepare the data loader and the optimizer

Lecture 11: Define the train_step() method

Lecture 12: Define the train_epoch_end() method

Lecture 13: [Important] Lecture correction

Lecture 14: Train the Deep Q-Learning algorithm

Lecture 15: Explore the resulting agent

Chapter 7: Hyperparameter tuning with Optuna

Lecture 1: Hyperparameter tuning with Optuna

Lecture 2: Link to the code notebook

Lecture 3: Log average return

Lecture 4: Define the objective function

Lecture 5: Create and launch the hyperparameter tuning job

Lecture 6: Explore the best trial

Chapter 8: Double Deep Q-Learning

Lecture 1: Maximization bias and Double Deep Q-Learning

Lecture 2: Link to the code notebook

Lecture 3: Create the Double Deep Q-Learning algorithm

Lecture 4: Check the resulting agent

Chapter 9: Dueling Deep Q-Networks

Lecture 1: Dueling Deep Q-Networks

Lecture 2: Link to the code notebook

Lecture 3: Create the dueling DQN

Lecture 4: Observation and reward normalization

Lecture 5: Create the environment – Part 1

Lecture 6: Create the environment – Part 2

Lecture 7: Implement Deep Q-Learning

Lecture 8: Check the resulting agent

Chapter 10: Prioritized Experience Replay

Lecture 1: Prioritized Experience Replay

Lecture 2: Link to the code notebook

Lecture 3: DQN for visual inputs

Lecture 4: Prioritized Experience Repay Buffer

Lecture 5: Create the environment

Lecture 6: Implement the Deep Q-Learning algorithm with Prioritized Experience Replay

Lecture 7: Errata Lecture

Lecture 8: Launch the training process

Lecture 9: Check the resulting agent

Chapter 11: Noisy Deep Q-Networks

Lecture 1: Noisy Deep Q-Networks

Lecture 2: Link to the code notebook

Lecture 3: Create the noisy linear layer class

Lecture 4: Create the Deep Q-Network

Lecture 5: Create the policy

Lecture 6: Create the environment

Lecture 7: Train the algorithm

Lecture 8: Check the results

Chapter 12: N-step Deep Q-Learning

Lecture 1: N-step Deep Q-Learning

Lecture 2: Link to the code notebook

Lecture 3: N-step Deep Q-Learning – Part 1

Lecture 4: N-step Deep Q-Learning – Part 2

Lecture 5: N-step Deep Q-Learning – Part 3