Epsilon-Greedy Policy

ε-Greedy Policy

An ε-greedy policy selects the greedy action (highest estimated value) with probability $1-\epsilon$, and a uniformly random action with probability $\epsilon$.

ε-Greedy Action Probabilities

$\pi (a|s)=\{\begin{array}{ll}1-\epsilon +\frac{\epsilon }{|\mathcal{A}(s)|}& \text{if }a=\arg {\max }_{a^{\prime }}Q(s,a^{\prime })\\ \frac{\epsilon }{|\mathcal{A}(s)|}& \text{otherwise}\end{array}$

• Simplest method for Exploration vs Exploitation balance
• ε-greedy is a special case of ε-soft policies (where $\pi (a|s)\ge \epsilon /|\mathcal{A}|$ for all $a$)
• Common to decay $\epsilon$ over time: high early (explore) → low later (exploit)

Connections

• Used by: SARSA, Q-Learning, Monte Carlo Control, Deep Q-Network (DQN)
• Alternative: Upper Confidence Bound, Boltzmann/softmax

Appears In

• RL-L01 - Intro, MDPs & Bandits, RL-L03 - Monte Carlo Methods, RL-L04 - Temporal Difference Learning