Hypothesis testing is a framework for making decisions about populations based on sample data.
The basic setup: You have a null hypothesis (default assumption of no effect) and an alternative hypothesis. You collect data and compute a test statistic that measures how extreme your data is under the null hypothesis. This gives you a p-value - the probability of seeing data this extreme or more if the null were true.
Two main philosophical approaches:
- Frequentist inference - treats parameters as fixed but unknown
- Baysian inference - treats parameters as random variables with distributions
The p-value measures how extreme your data is under the null hypothesis - but it’s widely misunderstood and misused. See the ASA’s 2016 statement warning about common pitfalls like confusing P(data|null) with P(null|data).