Decisions and learning from reward
Planning with Markov decision processes and value iteration, and learning without a model through returns, temporal-difference learning and Q-learning.
About 4 hours of study
Learning objectives
- Write the Bellman equation and run value iteration
- Extract an optimal policy
- Learn values and action values from experience
Lessons
- 1
Markov decision processes Advanced
Plan when actions have uncertain results. Describe a task as a Markov decision process, write the Bellman equation, compute values with value iteration, and read the best policy from them.
- 2
Reinforcement learning Advanced
Learn to act well from rewards alone, without knowing the transition model. Compute returns, estimate values with temporal-difference learning, and learn action values with Q-learning and ε-greedy exploration.
Module practice
- AI: One Bellman backupEasy
- AI: Value iteration in a gridMedium
- AI: The optimal policy in a slippery worldHard
- AI: Discounted returnsEasy
- AI: Temporal-difference learningMedium
- AI: Q-learning from experienceHard