Every-Visit MC

Every-Visit MC

A Monte Carlo prediction method that averages returns from every visit to a state within each episode. Returns within an episode are correlated, but the estimator is still consistent (converges asymptotically).

Compared to First-Visit MC: slightly biased for finite samples but also converges. In practice, often comparable performance.