classification

Training a classifier

We train a classifier $f_{\theta }$ with parameters $\theta$ on a dataset $\mathcal{D}=\{({\mathbf{x}}_i,y_i){\}}_{i=1}^n$ to output the probability of a datapoint belonging to the class $y$ given the feature vector $x$, $P_{\theta }(y|\mathbf{x})$.
The optimal parameters maximize the probability of the true labels:

$$\begin{array}{r}{\theta }^{\ast }=\arg \underset{\theta }{\max }{\mathbb{E}}_{(\mathbf{x},y)\in \mathcal{D}}[P_{\theta }(y|\mathbf{x})])]\end{array}$$

When training with mini-batches $B\subset \mathcal{D}$:

$$\begin{array}{r}{\theta }^{\ast }=\arg \underset{\theta }{\max }{\mathbb{E}}_{B\subset \mathcal{D}}\left[\sum _{(\mathbf{x},y)\in B}{P}_{\theta }(y|\mathbf{x})\right]\end{array}$$

---

softmax | cross-entropy loss

References

alfcnz NYU