Policy Evaluation

Policy Evaluation (Prediction)

Computing the state-value function $v_{\pi }(s)$ for a given policy $\pi$. Also called the prediction problem.

Iterative update: $V_{k+1}(s)={\sum }_a\pi (a|s){\sum }_{s^{\prime },r}p(s^{\prime },r|s,a)[r+\gamma V_k(s^{\prime })]$

Converges to $v_{\pi }$ as $k\to \infty$. Used as a subroutine in Policy Iteration.

Appears In

• RL-L02 - Dynamic Programming, RL-Book Ch4 - Dynamic Programming