Definition ReLU=max(0,x) ReLU is a fancy way of saying “if, then”. Turns out a simple threshold is good enough as a nonlinearity.
