negative log-likelihood loss

Categorical Cross-Entropy Loss

For one-hot encodings, the cross-entropy loss simplifies to Categorical Cross-Entropy, aka negative log-likelihood loss:

$${\ell }_{\text{CCE}}^{(i)}=-\sum _k{y}_{i,k}\log (p_{\theta }(k|x_i))=-\log (p_{\theta }(y_{i,k}|x_i))$$

$i$ … $i^{th}$ sample
$k$ … index of the correct class
$y$ … true label (one-hot, all probability mass is on the true class)
$p$ … predicted probability distribution (from softmax, …)
$\theta$ … model parameters
$x$ … input features
→ ${\hat{p}}_i=\text{softmax}(f_{\theta }(x_i)),{\ell }_{\text{CCE}}^{(i)}=-\log {\hat{p}}_{i,y_i}$

$\log \frac{1}{p}=-\log p$, see surprise:

References

Understanding softmax and the negative log-likelihood

cross-entropy
classification