Advanced Reinforcement Learning in Python: from DQN to SAC

Course Curriculum

Chapter 1: Introduction

Lecture 1: Introduction

Lecture 2: Reinforcement Learning series

Lecture 3: Google Colab

Lecture 4: Where to begin

Lecture 5: Complete code

Lecture 6: Connect with me on social media

Chapter 2: Refresher: The Markov Decision Process (MDP)

Lecture 1: Module Overview

Lecture 2: Elements common to all control tasks

Lecture 3: The Markov decision process (MDP)

Lecture 4: Types of Markov decision process

Lecture 5: Trajectory vs episode

Lecture 6: Reward vs Return

Lecture 7: Discount factor

Lecture 8: Policy

Lecture 9: State values v(s) and action values q(s,a)

Lecture 10: Bellman equations

Lecture 11: Solving a Markov decision process

Chapter 3: Refresher: Q-Learning

Lecture 1: Module overview

Lecture 2: Temporal difference methods

Lecture 3: Solving control tasks with temporal difference methods

Lecture 4: Q-Learning

Lecture 5: Advantages of temporal difference methods

Chapter 4: Refresher: Brief introduction to Neural Networks

Lecture 1: Module overview

Lecture 2: Function approximators

Lecture 3: Artificial Neural Networks

Lecture 4: Artificial Neurons

Lecture 5: How to represent a Neural Network

Lecture 6: Stochastic Gradient Descent

Lecture 7: Neural Network optimization

Chapter 5: Refresher: Deep Q-Learning

Lecture 1: Module overview

Lecture 2: Deep Q-Learning

Lecture 3: Experience Replay

Lecture 4: Target Network

Chapter 6: PyTorch Lightning

Lecture 1: PyTorch Lightning

Lecture 2: Link to the code notebook

Lecture 3: Introduction to PyTorch Lightning

Lecture 4: Create the Deep Q-Network

Lecture 5: Create the policy

Lecture 6: Create the replay buffer

Lecture 7: Create the environment

Lecture 8: Define the class for the Deep Q-Learning algorithm

Lecture 9: Define the play_episode() function

Lecture 10: Prepare the data loader and the optimizer

Lecture 11: Define the train_step() method

Lecture 12: Define the train_epoch_end() method

Lecture 13: [Important] Lecture correction.

Lecture 14: Train the Deep Q-Learning algorithm

Lecture 15: Explore the resulting agent

Chapter 7: Hyperparameter tuning with Optuna

Lecture 1: Hyperparameter tuning with Optuna

Lecture 2: Link to the code notebook

Lecture 3: Log average return

Lecture 4: Define the objective function

Lecture 5: Create and launch the hyperparameter tuning job

Lecture 6: Explore the best trial

Chapter 8: Deep Q-Learning for continuous action spaces (Normalized Advantage Function)

Lecture 1: Continuous action spaces

Lecture 2: The advantage function

Lecture 3: Normalized Advantage Function (NAF)

Lecture 4: Normalized Advantage Function pseudocode

Lecture 5: Link to the code notebook

Lecture 6: Hyperbolic tangent

Lecture 7: Creating the (NAF) Deep Q-Network 1

Lecture 8: Creating the (NAF) Deep Q-Network 2

Lecture 9: Creating the (NAF) Deep Q-Network 3

Lecture 10: Creating the (NAF) Deep Q-Network 4

Lecture 11: Creating the policy

Lecture 12: Create the environment

Lecture 13: Polyak averaging

Lecture 14: Implementing Polyak averaging

Lecture 15: Create the (NAF) Deep Q-Learning algorithm

Lecture 16: Implement the training step

Lecture 17: Implement the end-of-epoch logic

Lecture 18: Debugging and launching the algorithm

Lecture 19: Checking the resulting agent

Chapter 9: Refresher: Policy gradient methods

Lecture 1: Policy gradient methods

Lecture 2: Policy performance

Lecture 3: Representing policies using neural networks

Lecture 4: The policy gradient theorem

Lecture 5: Entropy Regularization

Chapter 10: Deep Deterministic Policy Gradient (DDPG)

Lecture 1: The Brax Physics engine

Lecture 2: Deep Deterministic Policy Gradient (DDPG)

Lecture 3: DDPG pseudocode

Lecture 4: Link to the code notebook

Lecture 5: Important – updated code

Lecture 6: Deep Deterministic Policy Gradient (DDPG)

Lecture 7: Create the gradient policy

Lecture 8: Create the gradient policy – Correction

Lecture 9: Create the Deep Q-Network

Lecture 10: Create the DDPG class

Lecture 11: Define the play method

Lecture 12: Define the play method – Correction