year: 2024/12
paper: https://arxiv.org/abs/2412.17799
website: https://pub.sakana.ai/asal/ | Yt: “Automating the search for artificial life with foundation models” by Akarsh Kumar
code: https://github.com/SakanaAI/asal
connections: sakana AI, ALIFE, open-ended, The Platonic Representation Hypothesis, CA, computational irreducibility, cma-es
Many naive things become possible in/with CLIP’s/LLM representation space, such as
- specifying goal states in natural language
- stopping simulation/training once change in latent space plateaus (magnitude of clip vector change)
- (heuristic metrics that fall short in other cases)
Methodology
CLIP encodes image(s) & text goal-prompt, cosine similarity is the fitness/objective.
They evolve interaction rules, e.g. parametrized through neural nets.
The boids use weight-sharing: simple rules + local interaction → complex behavior
The main? optimization algorithm they use is cma-es.
What’s the current limiting factor for this kind of evolution reaching the complexity we find in nature?
Bottlenecks:
- Expressivity of the substrates (the simulations being searched over by current research…)
- Search algorithms (need better search algorithms than cma-es or gradient descent)
- Scale … we’ve invested billions into training LLMs, but many orders of magnitude less in ALIFE simulations.
Open-ended curriculum learning for agentic AI:
Compare LLM summary with current knowledge/interaction-history/capabilities & goal-prompt.
Goal prompt can (needs to) also be meta, with an LLM automatically increasing difficulty. Memorization → Generalization, as per GPICL