Why do we sleep / dream? (from a cognitive perspective)
During sleep, you form connections between different events - events that are not directly connected in time and space, connections that were not made when actively experiencing them.
Unlearning & Simplifying Models
Dreaming could have the function of unlearning (like in ^75c0ea):
During the day you store lots of things and learn spurious minima, which come from an overflow of information, i.e. some minima/memories overlap, creating fake/bad minima.
At night (during REM sleep), you let the network settle to a minimum from a random state and then unlearn what you settled to.
You dream about something for hours and when you wake up you can’t remember anything at all (unless you wake up during the dream).
Looks like you are not storing what you are dreaming about.
Why bother dreaming at all?
Dreaming is contradictory, as the state of the brain looks very much the same as being awake, but you are not thinking mainly on the basis of real input, but input produced by the relay station just after the real input, the Thalamus!
So it seems the purpose of dreaming is to “clean” the “loss landscape” of your brain of spurious minima (suboptimal representations / models).
Another made up analogy (by me) would be, that dreaming is where you refine your world model based on real-life interactions you had stored over the course of the day, where you mainly used your trained world model and RL to adapt to live events.
Dreaming is like resolving dependencies of new ideas that you can no longer hold in working memory. - Joscha Bach
Transclude of free-energy-principle#^f80b15
Transclude of Fully-autonomous-robots-are-much-closer-than-you-think-–-Sergey-Levine#^a66212
References
Machine Consciousness - From Large Language Models to General Artificial Intelligence - Joscha Bach