Lecture notes: 14
Schedule, assessment and deadlines live in the course overview.
Lecture notes
- L01 Intro, MDPs & Bandits
Reinforcement Learning, Multi-Armed Bandit, Markov Decision Process, Value Function, … - L02 Dynamic Programming
Dynamic Programming, Policy Evaluation, Policy Improvement, Policy Iteration, … - L03 Monte Carlo Methods
Monte Carlo Methods, First-Visit MC, Every-Visit MC, Monte Carlo Control, … - L04 Temporal Difference Learning
Temporal Difference Learning, TD(0), TD Error, SARSA, … - L05 Tabular to Approximation
Function Approximation, Value Function Approximation, Stochastic Gradient Descent, Semi-Gradient Methods, … - L06 On-Policy TD with Approximation
Linear Function Approximation, Feature Construction, Tile Coding, Semi-Gradient Methods, … - L07 Off-Policy RL with Approximation
Episodic Semi-Gradient Control, Semi-Gradient Methods, On-Policy vs Off-Policy, Deadly Triad, … - L08 Deep RL Value-Based
Deep Q-Network (DQN), Experience Replay, Target Network, Conservative Q-Learning (CQL), … - L09 Policy Gradient Methods
Policy Gradient Methods, REINFORCE, Policy Gradient Theorem, Baseline, … - L10 Advanced Policy Search
Policy Gradient Methods, Policy Gradient Theorem, Actor-Critic, Deterministic Policy Gradient, … - L11 SAC, Decision Transformer & Diffuser
Soft Actor-Critic (SAC), Maximum Entropy RL, Decision Transformer, Decision Diffuser, … - L12 Model-Based RL
Model-Based Reinforcement Learning, Dyna, Monte Carlo Tree Search (MCTS), AlphaGo Zero, … - L13 Partial Observability
POMDP, Belief State, Predictive State Representation, Partial Observability, … - L14 Recap
Reinforcement Learning
Book Notes
- Book Ch1: Introduction
- Book Ch2: Multi-Armed Bandits
- Book Ch3: Finite MDPs
- Book Ch4: Dynamic Programming
- Book Ch5: Monte Carlo Methods
- Book Ch6: Temporal-Difference Learning
- Book Ch8: Planning and Learning
- Book Ch9: On-Policy Prediction with Approximation
- Book Ch10: On-Policy Control with Approximation
- Book Ch11: Off-Policy Methods with Approximation
- Book Ch13: Policy Gradient Methods
- Book Ch16: Applications and Case Studies
- Book Ch17: Frontiers
Exercises
- ES01: Exercise Set Week 1
- ES02: Exercise Set Week 2
- ES03: Exercise Set Week 3
- HW01: Homework 1
- HW02: Homework 2
- HW03: Homework 3