notation

Linalg-y stuff:

A note on notation

$a,\alpha$ … scalars (single numbers); normal/italic font
$\mathbf{x}\in {\mathbb{R}}^{d\times 1}$ … vector, by default a column vector; bold font
$\mathbf{A}\in {\mathbb{R}}^{m\times n}$ … matrix, by default capital bold font
$\hat{\mathbf{u}}$ … arbitrary unit vector
${\mathbf{e}}_i$ … basis vector (1 at position $i$, 0 elsewhere)
TODO: move the below to where they actually fit better
$X,Y$ … random variables
$\mathbf{X},\mathbf{Y}$ … random vectors
$x\in X$ … realization of random variable $X$
$\mathbf{x}\in \mathbf{X}$ … realization of random vector $\mathbf{X}$
$\hat{\theta },\hat{y},\hat{f}$ … estimates or predictions
$\mathcal{X},\Omega ,\mathcal{D}$ … set / space

$f:A\to B$ … function $f$ mapping from set $A$ to set $B$
$f(a;b,c)$ … function $f$ with argument $a$ and parameters $b,c$

Link to original

Probability stuff:

A note on notation

$\Omega$ … the sample space, the set of all possible outcomes of a random experiment.
$A,B,C\subseteq \Omega$ … events.
$\mathbb{P}:\mathcal{F}\to [0,1]$ … probability measure $\mathbb{P}(A)$ aka $\mathrm{Pr}(A)$, aka $P(A)$.

$X,Y,X_i$ … random variables; $\mathbf{X}=(X_1,X_2,\dots ,X_d)$ … a vector of random variables.
$x,y,z$ … realized values; $\mathbf{x}=(x_1,x_2,\dots ,x_d)$

$P_X$ … the probability distribution “law” of a random variable $X$.
$P(y|x)=P(Y=y|X=x)P(x_i|y)=P(X=x_i,Y=y)$ … shorthand notation for probabilities of random variables taking on specific values.

$p(x)=p_X(x)$ … the probability of a random variable $X$ taking on value $x$, general notation; specific:
… $=P(X=x)$ PMF (discrete; is a probability)
… $=f_X(x)$ PDF (continuous; not a probability, probabilities come from integrals $P(a\le X\le b)={\int }_a^b{f}_X(x)dx$)
$P(A|B),p(x|\theta )$ … conditional probability, discrete and continuous, respectively.

$\{p_{\theta }:\theta \in \Theta \}$ … a family of distributions indexed by $\theta$.
Its use can depend on context:
$\text{PMF/PDF}:=p(x;\theta )=p_{\theta }(x)=p(x|\theta )$ … the probability of data $x$ given fixed parameters $\theta$. (PMF/PDF of $x$)
$L(\theta ;x):=p(x;\theta )=p_{\theta }(x)=p(x|\theta )$ … the likelihood of fixed data $x$ given parameters $\theta$. (not a density in $\theta$!)

$p(x)$ … a distribution without explicit parameters, e.g. the true data distribution, or a marginal distribution.
$p(\theta )$ … a distribution over parameters, e.g. a prior.
$p(\theta |x)$ or $p(\theta ;x)$ … the posterior distribution over parameters given data $x$.

Link to original

¹

Sets:

A note on terminology

https://mathworld.wolfram.com/Collection.html says that a collection refers to a multiset.
But gpt5 says that’s almost certainly not the case in most contexts, a collection just a set.
But collection is so generic and context dependent, I only use it as “a bunch of things” and then define if necessary, else use a more specific term.

Link to original

Algebra, functions

Category theory:

Transclude of category-theorey#^04cef3

Footnotes

1. I haven’t been consistent with $x,\mathbf{x},\vec{x},\mathbf{x}$ notation for vectors, nor consistent with whether I treat them as column or row vectors so far. Trying to more consistently use $\mathbf{x}$ them as described in notation. Standard notation treats vectors as column vectors, which makes sense for having the same order as function composition in matmuls $B(A\mathbf{x})=(BA)\mathbf{x}$, which would otherwise be $(xA)B=x(AB)$², but in ML practice at least, data is usually represented as row vectors, soooo I vary the notation case by case but make it explicit ¯\(ツ)/¯ ↩

2. Which seems more intuitive, but function composition is defined to mirror substitution / nesting $f(g(x))=(f\circ g)(x)$, so matmuls should follow that order too etc. pp whatever ↩