memorization is an active, interpretative process
thoughts are thinkers
Memories are of the past but for the future.
Biological memories or life in general are so robust to changes (in environment, noise, corruption, …), because it is forced to “play the hand you’re dealt”
If you label something as memory: Who is the recipient? What utility does it serve? How does it get reinterpreted?
Presumably, a main limitation of LLMs is that they currently uses a lot of its weights more as a knowledge database than for reasoning capabilities.
Writing to “memory cells” like the Neural Turing Machine e.g.
Learning what to store and what to access e.g. Recurrent Memory Transformer does this.