Organize and link
Traditional algorithm you always only have one stepping stone.
In a QD algorithm, you want to collect a variety of stepping stones to leap off of.
You grow an archive of stepping stones, leap off of any of them, combine, …
Constantly trying new, very different things (like evolution does, massively parallel).
Search algo, but ure not optimizing to one specific objective and optimize one specific reward signal.
You shouldn’t be competing a cheetah and an ant. You can simultaneously work on both getting faster without one precluding another.
“Population” where you replace the current one with the next. So this is different from traditional population based methods. So you don’t loose those stepping stones.
But you can’t save everything (practically). You want some filtering.
Map elites discretizes along axes of interest. You can only keep one per bin.
If you hand code what’s interesting, you’ll always get pathologies.
Pre-training is incredibly important, because otherwise you have no idea what to explore / what’s important.
His lab is currently working on population-based approach or the judge of interestingness…
The vast majority of science is filling up the archive little by little. And once you’ve got the right shoulders to stand on, you get these massive leaps.
Archive also works with self-play where agents take turn playing against each other and each can see the strategies of the other (Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Models).
Eventually your system needs some real-world grounding for the interestingness again eventually, else you’ll eventually run into analogs of the “memorizing infinite digit numbers”.
Good thing to study: How long until the MoI breaks down? How to improve it when it breaks down?
The question on what’s intersting is not that hard because you can condition on the past.
open-ended
open-ended exploration
quality diversity
stepping stone
Why Greatness Cannot Be Planned
MAP-Elites
AI-GAs - AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
Slides
(sadly unsorted, ugh)
![[Jeff Clune-1758309436034.webp]]
![[Jeff Clune-1758309436028.webp]]
![[Jeff Clune-1758309436027.webp]]
![[Jeff Clune-1758309436022.webp]]
![[Jeff Clune-1758309436033.webp]]
![[Jeff Clune-1758309436023.webp]]
![[Jeff Clune-1758309436020.webp]]
![[Jeff Clune-1758309436029.webp]]
![[Jeff Clune-1758309436018.webp]]
![[Jeff Clune-1758309436015.webp]]
![[Jeff Clune-1758309436012.webp]]
![[Jeff Clune-1758309436009.webp]]
![[Jeff Clune-1758309436008.webp]]
![[Jeff Clune-1758309436004.webp]]
![[Jeff Clune-1758309436002.webp]]
![[Jeff Clune-1758309435999.webp]]
![[Jeff Clune-1758309435997.webp]]
![[Jeff Clune-1758309435994.webp]]
![[Jeff Clune-1758309435991.webp]]
![[Jeff Clune-1758309435988.webp]]
![[Jeff Clune-1758309435986.webp]]
![[Jeff Clune-1758309435985.webp]]
![[Jeff Clune-1758309435984.webp]]
![[Jeff Clune-1758309435984.webp]]
![[Jeff Clune-1758309435981.webp]]
![[Jeff Clune-1758309435981.webp]]
![[Jeff Clune-1758309435980.webp]]
![[Jeff Clune-1758309435979.webp]]
![[Jeff Clune-1758309435978.webp]]
![[Jeff Clune-1758309435978.webp]]
![[Jeff Clune-1758309435975.webp]]
![[Jeff Clune-1758309435976.webp]]
![[Jeff Clune-1758309435974.webp]]
![[Jeff Clune-1758309435972.webp]]
![[Jeff Clune-1758309435971.webp]]
![[Jeff Clune-1758309435970.webp]]
![[Jeff Clune-1758309435967.webp]]
![[Jeff Clune-1758309435969.webp]]