SARSA

SARSA is on-policy TD learning with the Q-function.
Here we are always doing what we think is the best thing, so no epsilon greedy exploration. It’s more conservative “save” due to that (no random actions), and converges slower than Q-Learning, but we gain more cumulative reward.

In contrast to Q-learning, we update our Q-values based on the actual actions we take, not the maximum Q-value of the next state:

$$\begin{array}{rl}\text{SARSA: }& Q^{\text{new}}(s_t,a_t)=Q^{\text{old}}(s_t,a_t)+\alpha (r_t+\gamma Q^{\text{old}}(s_{t+1},a_{t+1})-Q^{\text{old}}(s_t,a_t))\\ \text{Q-Learning: }& Q^{\text{new}}(s_t,a_t)=Q^{\text{old}}(s_t,a_t)+\alpha (r_t+\gamma \underset{a}{\max }Q(s_{t+1},a)-Q^{\text{old}}(s_t,a_t))\end{array}$$

reinforcement learning
on-policy, temporal difference learning
https://lilianweng.github.io/posts/2018-02-19-rl-overview/#sarsa-on-policy-td-control