See also: short-term memory
Memory is an association between two stimuli.
There is no such thing as static or read-only memory in the brain. Whenever you try to read a memory, you modify it.
Memory has to be not just stored, but it has to be generalized and imprinted onto a new, onto a potentially greatly changing substrate. A dynamic, living agent cannot just keep things the same. 1
Transclude of michael-levin#^bef2d1
Context is critical for biological memories because evolution knows that memories are going to be reinterpreted by a future you that is not the same as past you.
Your future brain might have undergone puberty and remodeling with the hormones. It might have aged. It might have learned all kinds of stuff that makes the past knowledge seen in a new way. Memories are living and they are constantly adapted. I wonder how much of that capacity is in the cleverness of the host’s mechanisms, or is it in some sort of basal competency of memories to adapt and survive and maintain themselves in whatever medium they can?
As William James said, “thoughts are the thinkers”. Maybe it’s a collaboration of both – the drive of memories to persist and the agency of the plastic cognitive apparatus that helps them adjust to a new environment.
We as beings at any point in time, don’t have access to the past. What we have access to is the engrams, the memory traces that the past has left in our brain or body.
We don’t have direct access to what actually happened. So what that means is that at any given moment, you and I and all cognitive beings are a collection of temporal slices, with a little bit of thickness, maybe a couple of hundred milliseconds or similar. We have to reconstruct in real time, a story of who we are, what we are, what our past history is. It is a real-time process.
Transclude of michael-levin#^bef2d1
Memories are of the past but for the future.
Biological memories or life in general are so robust to changes (in environment, noise, corruption, …), because it is forced to “play the hand you’re dealt”
If you label something as memory: Who is the recipient? What utility does it serve? How does it get reinterpreted?
Memory in classical ML systems:
Presumably, a main limitation of LLMs is that they currently uses a lot of its weights more as a knowledge database than for reasoning capabilities.
Writing to “memory cells” like the Neural Turing Machine e.g.
Learning what to store and what to access e.g. Recurrent Memory Transformer does this.