Follow the greedy strategy (the one which is projected to be the best) with probability and explore (a random action) with probability .
Constant rates on a k-armed bandit:
- No exploration rises most quickly in the very beginning, but then performs the worst.
- Low exploration (0.01) has a slower learning trajectory, but in the end performs the best.
- High exploration (0.1) learns considerably faster, but the total reward will not go as high as lower exploration when run indefinitely.