year: 2024/05
paper: https://arxiv.org/pdf/2405.07987
website: https://phillipi.github.io/prh/
code: https://github.com/minyoungg/platonic-rep/, YT-Vid
connections: representation learning, platonic space
The Platonic Representation Hypothesis
Neural networks, trained with different objectives on different data and modalities, are converging to a shared statistical model of reality in their representation spaces.
The Multitask Scaling Hypothesis
There are fewer representations that are competent for N tasks than there are for M < N tasks. As we train more general models that solve more tasks at once, we should expect fewer possible solutions.
The Capacity Hypothesis
Bigger models are more likely to converge to a shared representation than smaller models.