Decisions and learning from reward

Planning with Markov decision processes and value iteration, and learning without a model through returns, temporal-difference learning and Q-learning.

About 4 hours of study

Learning objectives

Lessons

  1. 1

    Markov decision processes Advanced

    Plan when actions have uncertain results. Describe a task as a Markov decision process, write the Bellman equation, compute values with value iteration, and read the best policy from them.

  2. 2

    Reinforcement learning Advanced

    Learn to act well from rewards alone, without knowing the transition model. Compute returns, estimate values with temporal-difference learning, and learn action values with Q-learning and ε-greedy exploration.

Module practice

Module quiz

1. What does a policy give?
2. What is the return for rewards 2, 2 with γ = 0.5?
3. What does value iteration repeat?
4. What is the target of a Q-learning update?
5. Why does an agent use ε-greedy action selection?
6. What does reinforcement learning need that value iteration does not?

‹ Reasoning under uncertaintyFinal assessment ›