Emergent Models: Intelligence from Tiny Substrates

Giacomo Bocchese${}^{1,2}$, Nicola Giacobbo${}^2$, Etienne Guichard${}^{3,2}$, James Wiles${}^1$, Akshaj Devireddy${}^{1,2}$

${}^1$Wolfram Institute, ${}^2$Emergent Computing, ${}^3$Østfold University College

Abstract

Emergent Models (EMs) are a machine learning paradigm based on simple yet open-ended substrates, such as cellular automata, in which modeling is treated not as the learning of a closed-form input-output map but as the emergence, within simple dynamical systems, of computational behaviors that solve external tasks. Such substrates typically iterate a fixed local rule over a latent space for an adaptive number of steps, with an interface linking the latent state to external input/output signals. Training proceeds by evolutionary search. We hypothesize that some instances of this framework are biased toward global generalization: capturing the rule generating the data over its full domain, and therefore extrapolating beyond the training range. Theoretically, we prove that some EMs are latent-universal: with the update rule and interface held fixed, they can realize any partial computable function by varying only the initial condition of the latent state. Empirically, we study a zoo of minimal EM instantiations across discrete and continuous substrates, showing that local-recursive computation at a tiny scale (tens to hundreds of parameters) can extrapolate exactly on simple arithmetic functions, can support control behaviour and online adaptation, while still exposing several limitations. This work is foundational: it does not propose a competitive architecture, but a framework meant to widen the design space of machine learning beyond differentiable feed-forward maps.

1 Introduction

Sutton’s “Bitter Lesson” argues that the strongest long-run progress in AI comes from general methods that leverage compute, rather than from direct top-down engineering [36]as argued by Rich Sutton. A similar intuition appears in artificial life and open-ended evolution, where complex life-like behaviors are studied as emerging from simple yet general physical substrates under long-run evolutionary processes [8]. This motivates the search for substrates governed by simple rules that can be emulated on a computer, in which external evolutionary pressure can select for the emergence of task-solving behaviors.

Emergent Models (EMs) formalize this perspective by turning such dynamical systems into machine learning models, coupled to external inputs and outputs through an encoding and decoding interface.

But which notion of openness is relevant for machine learning? And is there a precondition for a substrate to “leverage compute”? We argue that both questions have the same answer: expressivity, the set of functions a model can theoretically represent. Search can only select among functions a model can express: a target behavior outside this range is unreachable at any compute budget.

Because the term expressivity is often used informally and varies across contexts [5, 16, 44], we define explicitly the notion relevant here: global expressivity, the class $G_{\varphi }$G sub phi of functions that a family of models ${\varphi }_{\theta }$phi theta, parametrized by $\theta$theta, can represent exactly or approximate to arbitrary precision over their full, possibly open domain. The global qualifier reflects the goal of modeling, which is to capture the rule generating the data rather than to fit its values on a bounded region: a model that internalizes the rule can extrapolate by construction, while a local fit does not [6, 24, 25].

Much of traditional deep learning is based on feed-forward neural networks (FFNNs) [10]. These fixed-depth, closed-form mappings typically extrapolate poorly beyond their training range [17], reflecting limitations in both expressivity and inductive bias¹ [3]. On the expressivity side, the limitation is structural: universal approximation theorems for FFNNs [19] guarantee arbitrarily good approximation only over compact domains, and over open domains what a network can represent or approximate is dictated by the asymptotic behavior induced by its activation function. For scalar functions $\mathbb{R}\to \mathbb{R}$from the real numbers to the real numbers, sigmoidal networks converge to constants and ReLU networks become asymptotically linear. As a result, FFNNs cannot globally represent functions such as $\sin (x)$ or $e^x$sine of x or e to the x without task-specific engineering² [38].

In traditional recurrent or autoregressive architectures, such as standard (non-reasoning) RNNs and Transformers,

each datapoint prediction is obtained through a single forward pass $x_i\mapsto y_i$x i to y i, favoring direct one-shot mapping from input to output [9, 31, 46]. A growing body of work suggests that expressivity is increased when models are allowed to perform multiple iterative updates per input, rather than a single forward pass. Adaptive Computation Time makes this explicit in recurrent networks by applying the same update function to the hidden state for a variable number of steps before producing an output [11]. Reasoning LLMs apply a similar principle in the token space, expanding the prediction process through intermediate reasoning traces before producing an answer [15]; these models have achieved strong gains on arithmetics, coding, and ARC-AGI reasoning puzzles [6, 30], and under suitable assumptions are Turing complete [21], however, they remain tied to very large neural networks and appear brittle on long-sequence extrapolation [1, 33].

This raises a question about compression: how small can the iterated update function be while still supporting high global expressivity, and does smallness itself carry favorable inductive biases for extrapolation? Compact recursive models such as Hierarchical Reasoning Models (HRMs) and Tiny Recursive Models (TRMs) match mid-size LLMs on ARC-AGI-style reasoning with $\approx 10^3$approximately ten to the third fewer parameters [22, 41]. Neural GPUs and Neural Cellular Automata (NCAs) push it further by iterating a small neural update function locally across a spatial latent medium [27, 50]: a Neural GPU trained on binary multiplication up to 20-bit inputs generalizes perfectly to 2000 bits, and NCAs reach significant ARC-AGI scores with $\approx 10^5$approximately ten to the fifth fewer parameters than comparably performing LLMs [13, 47]. Across these lines of work, a common pattern emerges: local, recurrent, compact neural updates appear to favor algorithmic generalization while reducing parameter size.

The emphasis on iteration is grounded in computation theory: in classical models of computation (eg. Turing machines), complex algorithmic behavior arises from repeatedly applying simple update rules to a mutable memory state, with computation time and memory allowed to vary depending on input and task complexity. Looping is not an implementation detail, but one of the core ingredients that makes general computation possible [18, 34]. The strongest general notion of global expressivity in this algorithmic setting is computational universality, or Turing completeness [34, 45]. A universal system, given unbounded memory and time, can compute any partial computable function: in discrete domains this yields exact global representation, while for continuous computable functions it permits arbitrary-precision global approximation under a proper discretization convention [4, 26, 42]. Universality says nothing about trainability or efficiency, but it sets a necessary condition: if a model class cannot realize arbitrary computable procedures even in principle, there are hard limits on what it can express.

Cellular automata (CAs) are a natural substrate in this setting. They combine a simple local rule with iterative dynamics over a potentially unbounded medium, and some, including Conway’s Game of Life and Rule 110, are Turing complete [7, 32]. Several notions of universality are studied for CAs; here we focus on strong universality³, which most closely aligns with classical Turing’s notion for computing functions.

The use of CAs in machine learning is not entirely new, and so far has followed two main approaches: CA reservoirs and NCAs. Reservoir methods treat the automaton as a fixed dynamical medium and train only an external linear readout of its trajectories [28, 48]. NCAs instead train the update function itself, typically a small convolutional neural network applied locally [12, 40]. In both cases the initial condition is not treated as an explicitly learnable object; instead our framework trains it directly, optionally jointly with the update rule, so that the initial state acts as a program shaping the automaton’s evolution and consequently the predictions of the model.

In this work, we formalize this perspective through a framework that turns dynamical systems into machine learning models. In its strongest ideal regime, which we call latent universality (theorem 1), the update rule is fixed and varying only the initial state suffices to express any computable function.

Our experiments deliberately study a simpler setting: they do not realize latent universality, and no experimental claim depends on Theorem 1. Rather than fine-tuning a single construction toward the full theory, we examine a zoo of minimal Emergent Model instantiations to convey the idea and philosophy behind the framework, as befits the opening stage of this research. These show that local-recursive modeling at a tiny scale (20-300 parameters) can extrapolate on simple arithmetic functions and can produce control behaviors, indicating that such substrates are meaningful in principle, not that they are easy to train or competitive with mature task-specific methods.

2 Emergent Models

2.1 Intuition

A broad class of latent-space reasoning models can be summarized schematically as:

x is encoded to an initial state s zero, which is updated by function f for T steps to reach state s T, which is then decoded to output y

where a fixed update function $f$f is iterated on a latent state $s$s for a number of steps $T(s_0)$T of s zero, which can vary and may depend on the initial condition. In this view, the latent state provides memory (“computational space”), while repeated application of $f$f provides computation time. The encoder $E$E writes inputs

into the latent space, and the decoder $D$D reads outputs from it. Encoder and decoder form an interface: a communication protocol between the latent space and the external world.

Figure 1 shows a didactical instance: a continuous two-dimensional lattice with an interface based on fixed cell positions: inputs and outputs are written and read at designated locations (ports).

Figure 1: Emergent Model with a two-dimensional continuous-valued lattice as latent state, $s\in {\mathbb{R}}^{20\times 20}$. Coloured borders mark the ports: input (green), output (blue), halting (red), and reward (orange). Interface: the input $x\in {\mathbb{R}}^2$ is injected at the two input ports through the encoder $E(x)$; at halting, the decoder $D(s_T)$ reads the output $y\in {\mathbb{R}}^2$ from the two output ports. Update: the transition rule is iterated, $s_{t+1}=f(s_t)$, for an adaptive number of steps, halting when $H(s_t)=1$. The role of the reward port is explained in Section 2.5.

Both the latent state and the interface admit far more general forms than this example. The latent state may be unbounded in space, as in a Turing machine tape, $s\in \{0,1{\}}^{\mathbb{N}}$s in the set of binary strings, or in the grid of the Game of Life CA, $s\in \{0,1{\}}^{{\mathbb{Z}}^2}$s in the set of binary grids. Encoding and decoding are similarly general. Information can be injected either at fixed positions of the state space, called ports (Figure 1), or at input-dependent locations determined by some computable rule (Figure 2).

Figure 2: Diagram representing the latent state of EM43, a discrete Emergent Model, evolving in time. It is based on a four-valued, radius-1 cellular automaton over a one-dimensional unbounded tape, with state $s\in \{0,1,2,3{\}}^{\mathbb{N}}$. This uses a different interface from fig. 1: input is not written to a fixed port, but placed at a position that depends on its value. The same holds for the output. Encoding: an integer input $x\in \mathbb{N}$ is written as a red cell at position $x_0+x$, i.e. at offset $x$ from a fixed origin $x_0$. In this case input $x=10$. Decoding: at halting, the output $y\in \mathbb{N}$ is read as the position of the rightmost red cell relative to the same origin $x_0$. In this case output $y=20$. Update: the transition rule is a lookup table over radius-1 neighbourhoods, applied synchronously on all the tape. Halting: triggered by a relative count of cell values (Explained in section 3.2).

Emergent Models formalize this picture through a generalized automaton: a discrete-time dynamical system with a halting condition. Equipping the automaton with an interface (encoder and decoder) turns the dynamical system into a model. The following subsections make this construction mathematically rigorous.

2.2 Generalised Automaton

The topology of the latent state space $\mathcal{S}$S is abstracted as a graph to keep the formalism as general as possible.

Let $G=(V,E)$G equals V, E, where $V$V is the set of vertices (nodes) and $E$E is the set of edges.

Each node of the graph carries a value in a set $W$W, typically binary $\{0,1\}$, finite discrete $\{0,1,2,\dots ,n\}$, or continuous $\mathbb{R}$the real numbers. The state space $\mathcal{S}$S is the set of all global configurations, that is, all assignments of a value in $W$ to each node,

S is defined as W to the power of V, the set of all mappings from V to W

The edges define the neighborhood structure, but do not play a direct role in the formalism, as explained below.

Typical choices are (i) a one-dimensional unbounded tape, as in Turing machines or 1-d CAs, with $V=\mathbb{N}$ and finite alphabet $W=\{0,1,\dots ,n\}$, or (ii) a two-dimensional finite

lattice, with $V=\{0,1,\dots ,n{\}}^2$V equals the set zero to n squared and cell values in $\mathbb{R}$ or $\mathbb{Q}$the reals or rationals (Figure 3); example (i) is instantiated in Figure 2, and example (ii) in Figure 1. These choices affect expressivity: to support unbounded memory capacity (a necessary condition for Turing completeness) the system must have either an unbounded topology, such as an infinite tape, or a finite tape or grid with an infinite value set $W$W, for example rationals or reals at unbounded precision.

Figure 3: Graphs: 1-d unbounded tape (on the left) with radius-1 interaction ; 2-d finite lattice (on the right) with Moore neighborhood. Dots represent nodes, while lines connecting the nodes are the edges.

A transition function $f:\mathcal{S}\to \mathcal{S}$f from S to S acts on the state and performs the one-step update, which we call a microstep, so that $s_{t+1}=f(s_t)$s t plus one equals f of s t. A halting operator $H:\mathcal{S}\to \{0,1\}$H from S to the set zero, one is checked at each microstep, and if $H(s_t)=1$H of s t equals one the iteration stops. Starting from an initial configuration $s_0\in \mathcal{S}$s zero in S, the system evolves until halting. We define the halting time as $T(s_0):=\min \{t\in \mathbb{N}\mid H(s_t)=1\}$T of s zero is defined as the minimum t in the natural numbers such that H of s t equals one, when this exists and is finite, and undefined otherwise. If $T(s_0)$ exists, the terminal state is $s_T:=s_{T(s_0)}$s T, defined as s at time T of s zero.

The transition function $f$ is local if there exists a function $F$ such that for every $s\in \mathcal{S}$ and every vertex $v\in V$, $(f(s))(v)=F(s{|}_{N(v)})$the state of vertex v after update f is F applied to the restriction of s to the neighborhood of v, where $s{|}_{N(v)}$ denotes the restriction of $s$ to the neighborhood $N(v)$, typically induced by the edges. In unbounded topologies (e.g. an infinite cellular-automaton lattice), we typically assume one value $b\in W$ to be quiescent/blank, i.e. stable under the local rule: $F(b,\dots ,b)=b$, so that a blank cell remains blank whenever its neighborhood is composed only of blank cells. Importantly, $f$ is the global transition operator acting synchronously on the whole state space; locality means that this global update can be realized by a local rule $F$ applied to each neighborhood. In this formulation, the edge structure need not be modeled separately: any dependence on adjacency is already absorbed into the definition of $f$.

The transition function can be realized in many ways, such as lookup tables in discrete cellular automata, neural networks in neural cellular automata, polynomial maps, and other computable rules.

A generalized automaton is a tuple $\mathcal{A}=(\mathcal{S},f,H)$ that, starting from an initial condition $s_0$, generates a trajectory $s_0,s_1,s_2,\dots$ and, if halting occurs, terminates at a state $s_T$. Many computational devices can be abstracted in this way, including Finite State Machines, Turing machines (see Appendix A.1), CAs, NCAs, and some forms of recurrent neural networks.

2.3 The model

An Emergent Model is a tuple $M=(\mathcal{S},f,H,E,D,\oplus )$M equals S, f, H, E, D, and the combination operator, and is built by equipping a generalized automaton with an input-output interface.

Let $E:\mathcal{X}\to \mathcal{S}$ be an encoder function, mapping from an input space $\mathcal{X}$ into the latent space $\mathcal{S}$, and let $D:\mathcal{S}\to \mathcal{Y}$ be a decoder, reading an output in $\mathcal{Y}$ from the latent space $\mathcal{S}$.

The automaton starts from a non-perturbed ($np$) initial state $s_{0,np}=p\in \mathcal{S}$, which we call the program. Given an input $x\in \mathcal{X}$, the encoder produces a state $E(x)\in \mathcal{S}$. The actual (perturbed) initial state is then obtained by combining the program with the encoded input:

s zero equals s zero perturbed, which is p combined with E of x

where $\oplus :\mathcal{S}\times \mathcal{S}\to \mathcal{S}$ is a simple total computable combining operator, typically given by sum, overwrite at particular locations or similar operations.

From this point on, program $p$ always indicates the non-perturbed initial state, while initial state refers to the perturbed initial state $s_0=p\oplus E(x)$. An example construction of the initial state by combination is shown in fig. 4.

Figure 4: Construction of the initial state $s_0=p\oplus E(x)$ on the same two-dimensional continuous-valued lattice of fig. 1; here $\oplus$ is a sum. Left: the program $p$, non-perturbed initial state. Centre: the encoded input $E(x)$, nonzero only at the input ports and quiescent elsewhere. Right: their sum, giving the perturbed initial state $s_0$ from which the dynamics are iterated.

The program $p$ should not be understood as a symbolic algorithm explicitly compiled into the substrate, as in a programming language. It is instead a raw substrate-level configuration: a pattern in the state space that conditions the system’s evolution so as to induce a particular function. In this sense it still captures what we take to be the essence of a program (a state held in memory that steers the computation) but without the symbolic, human-written form that word usually implies.

Inference proceeds as follows: initialize the state with the program, encode the input, run the dynamics $s_{t+1}=f(s_t)$s t plus one equals f of s t until halting, and decode the output from the terminal state. If the computation never halts, the output is undefined. This induces the (partial) input-output map:

Phi of x equals D of the transition function f applied T of p and x times to the composition of program p and encoded input E of x

whenever the halting time $T(p,x)$T of p and x is defined.

In machine learning, a model is treated as a parametrized function ${\Phi }_{\theta }:\mathcal{X}\rightharpoonup \mathcal{Y}$Phi theta from X to Y, where the parameters $\theta \in \Theta$theta in Theta collect the components allowed to vary during training. In an Emergent Model, $\theta$theta may include the program $p$p, parts of the transition function $f$f, of the halting condition $H$H, and of the interface $E,D,\oplus$.

Modeling then means finding parameters $\theta$theta whose induced map computes a target function $g:\mathcal{X}\rightharpoonup \mathcal{Y}$g from X to Y. This notion is deliberately stronger than the local-approximation property usually invoked in deep learning, which guarantees a fit only on a compact domain: computing a function means realizing it on its full, possibly open domain, reproducing its behaviour everywhere rather than interpolating its values on a bounded region (exactly in the discrete case, or to arbitrary precision in the continuous case).

Definition 1 (Model computes a function). Let ${\Phi }_{\theta }:\mathcal{X}\rightharpoonup \mathcal{Y}$Phi theta from X to Y be a model parametrized by $\theta$theta, inducing a family of partial functions $\{{\Phi }_{\theta }{\}}_{\theta \in \Theta }$Phi theta for theta in Theta. Let $g:\mathcal{X}\rightharpoonup \mathcal{Y}$g from X to Y be a target partial computable function, total computable on the domain $\tilde{\mathcal{X}}\subseteq \mathcal{X}$X tilde, a subset of X.

If $g$g is discrete, we say that the model computes $g$g if for some $\theta \in \Theta$:

Phi theta of x equals g of x for all x in X tilde

and ${\Phi }_{\theta }(x)$ is undefined for $x\notin \tilde{\mathcal{X}}$.

If $g$g is continuous-valued, the notion becomes arbitrary-precision global approximation: the model computes $g$g if, for every output tolerance $ϵ>0$epsilon greater than zero, there exists ${\theta }_{ϵ}\in \Theta$theta epsilon in Theta such that, for every $x\in \tilde{\mathcal{X}}$, a sufficiently precise finite representation $x_{\text{fin}}$ of $x$ satisfies:

the norm of the difference between Phi theta epsilon of x fin and g of x is less than epsilon

The tolerance $ϵ$epsilon is fixed over the whole output domain $\mathcal{Y}$Y, while the required input precision may depend on $x$x. A more explicit formulation is given in Appendix A.4, where we also show that, in the asymptotic limit of input precision, uniform global approximation is obtained.

Which functions a model can compute depends on both the substrate and the parametrization chosen. Different substrates and parametrizations therefore compute different classes of functions, presenting different degrees of expressivity. The maximal degree in the computational setting is Turing completeness, or computational universality: a single model able to represent every partial computable function by varying $\theta$theta.

We theorize a special regime, called latent universality (Section 2.4), in which the transition function, state space topology, and interface are all fixed and the program alone determines the computation the model performs ($\theta =p$). In this regime, taking $\mathcal{X}$ and $\mathcal{Y}$X and Y to be a general computable data format (e.g. arbitrary length binary strings $\{0,1{\}}^{\ast }$) and a universal automaton $\mathcal{A}$, a single fixed interface $E,D,\oplus$ suffices for all computable functions on that domain (see Theorem 1). In non-universal constructions, including the practical implementations we tested, the transition function or parts of the interface may instead be trained jointly with the program.

The encoder and decoder must be simple, total computable, and compatible with the dynamics of the chosen automaton $\mathcal{A}=(S,f,H)$A, defined by S, f, and H. Not every interface is admissible: some encodings produce representations the dynamics cannot process, and some decodings require output structures the dynamics cannot produce.

2.4 Latent Universality

Assuming the input/output spaces are binary strings $\mathcal{X}=\mathcal{Y}=\{0,1{\}}^{\ast }$X and Y are the set of all binary strings, we consider a regime in which an Emergent Model $M$ is kept with fixed components $(S,f,H,E,D,\oplus )$, while arbitrary computations are realized by varying only the program $p\in S$p in S. We call this latent universality: the latent state alone can select the realization of any possible algorithm performed by the model.

Latent universality is not meant as a new universality claim at the level of computability theory, but as a reformulation of it in natural terms for machine learning on dynamical systems. In Universal Turing Machines (UTMs)UTMs, arbitrary computable behavior is realized by a fixed transition function together with a fixed scheme for encoding program and input into the initial state of the tape [2, 20, 34]. In the EM setting, the same structure appears as a fixed substrate $(S,f,H)$ and interface $(E,D,\oplus )$, with the program state defining the computation of a particular function (see Appendix A.2).

The analogy should not be interpreted as requiring a compiler from symbolic algorithms to substrate programs: the program is treated as a trainable substrate-level variable, and learning consists of searching over raw program states that induce the desired behavior when paired with inputs. Inputs, by contrast, may be compiled: the encoder $E$ syntactically transforms them into a fixed representation the machine can read, with no information about the program $p$ nor about the target function to be realized. This asymmetry is what motivates keeping the program $p$ and the encoded input $E(x)$ separate and combining them afterwards, rather than compiling both jointly through a single map $E(p,x)$: since the encoder acts only on $x$ and not on $p$, any program can be searched while inputs are presented to the substrate through a common, fixed interface. This formulation departs slightly from the standard pairing convention, which does not generally require program and input to be separable; several known universal

Turing machines nonetheless admit this disjoint form, initializing the tape with the simulated machine description and the encoded input in distinct regions or tapes, with no interdependence between the two: the encoder of $x$x has no information about $p$p, and the combination consists on a simple operation (e.g. overwrite or concatenation) that applies no semantic interpretation of either $x$x or $p$p (see Appendix A.2).

We work in a strong-universality setting. The substrate has an unbounded topology (countably infinite $V$V) and a finite discrete alphabet $W$W with a quiescent (blank) symbol $b\in W$b in W. Program and encoded input are finite-support states: all non-quiescent values are confined to a finite but arbitrarily large region, surrounded by an infinite quiescent background. For every program $p$p and input $x$x, the combined initial state $p\oplus E(x)$p plus E of x has finite support as well.

Theorem 1 (Latent Universality). Let $\Sigma =\{0,1\}$Sigma be the set zero, one and the input/output spaces be binary strings $\mathcal{X}=\mathcal{Y}={\Sigma }^{\ast }$X and Y be Sigma star. There exists an Emergent Model: $M=(S,f,H,E,D,\oplus )$M, defined by S, f, H, E, D, and the combining operator, with $S=W^V$S equals W to the V for some countably infinite $V$V and finite-discrete $W$W, such that for every partial computable function $g:{\Sigma }^{\ast }\rightharpoonup {\Sigma }^{\ast }$g from Sigma star to Sigma star, there exists a finite-support program state $p\in S$p in S for which ${\Phi }_p$phi sub p computes $g$g.

Figure 5: Didactic TM example: program string $\rho$, separator #, and input string $x$ on the tape, forming the initial state $\rho \#x{b}^{\infty }$.

In Appendix A.3 we provide a proof sketch of this result and extend it to Conway’s Game of Life CA through Rendell’s construction (Lemma 2).

Using $\mathcal{X},\mathcal{Y}=\{0,1{\}}^{\ast }$X and Y as sets of binary strings is without loss of generality, since binary computable functions already capture the full class of computable functions. Importantly, other computable domains must be reduced to binary strings through an effective binary expansion, adding a small outer layer to the interface that converts that data format into binary.

On a one-sided, one-dimensional tape ($V=\mathbb{N}$V equals the set of natural numbers), a finite-support state is exactly a finite string over $W$W followed by an infinite quiescent background: the program takes the form $p=\rho b^{\infty }$p equals rho b infinity and the encoded input $E(x)=\chi b^{\infty }$E of x equals chi b infinity, with $\rho ,\chi \in W^{\ast }$rho and chi in W star. As a didactic example, consider a Turing machine with alphabet $\Gamma =\{b,0,1,\#\}$Gamma containing b, zero, one, and the pound sign, where $\{0,1\}$ are used for program, input, and output, $b$ is the blank, and $\#$ is a separator. The encoder writes the binary input $x\in \{0,1{\}}^{\ast }$ verbatim, followed by an infinite quiescent background, $E(x)=x{b}^{\infty }$. The combining operator strips the quiescent backgrounds, concatenates the two finite strings around the separator, and re-pads with an infinite quiescent background, giving $p\oplus E(x)=\rho \#x{b}^{\infty }$ (fig. 5).

The same structure extends beyond tapes. On a two-dimensional grid, the program and the encoded input are both full-grid states, each non-quiescent only on its own finite region, and $\oplus$the combining operator merges them into a single state on the same board, as in the Game of Life construction of Lemma 2. Treating program and encoded input as states $(p,E(x)\in S)$, rather than strings $(\rho ,\chi \in W^{\ast })$, is what makes this possible: the construction applies to any generalized automaton, over arbitrary graph topologies, and the classical string picture is recovered as the one-dimensional special case.

The flexibility of the paradigm leaves open further constructions: with a continuous value set $W$W, for instance, the same scheme may provide a substrate for analog computing. We currently offer no proof or formal guarantee in that direction.

2.5 Sequential Operation, State Retention and Meta Learning

So far, the EM formalism defines a predictor acting on a single input, $y={\Phi }_p(x)$y equals phi sub p of x. Many tasks, however, require sequential operation on an input stream $(x_i{)}_{i\ge 0}$x i for i greater than or equal to zero with memory retained across successive predictions. This introduces two time scales: microsteps $t$t, indexing the internal reasoning steps of the model within a single prediction, and macrosteps $i$i, indexing successive data points on the stream.

A naive rule to pass information to the next macrostep would be to set the next program equal to the previous terminal state, $p_{i+1}=s_{T_i,i}$p sub i plus one equals s sub T i comma i. In general this may fail, since $s_{T_i,i}$ already satisfies the halting condition and would make the next prediction halt immediately. We therefore introduce a retention operator $R:S\to S$R from S to S, whose role is to transform the terminal state into a valid program for the next macrostep:

p sub i plus one equals R of the terminal state s sub T i comma i

The operator $R$R should preserve the information that must persist across macrosteps, while restoring the control conditions needed for a new computation.

For example, in a Turing Machine, $R$R could reset the head state from the halting state $q_h$q sub h to the initial state $q_0$q sub zero, reset head position to index 0, while preserving the tape content. Such

a reset is valid only if the retained state does not prevent the machine from starting and performing the next computation correctly. Retention may require a substrate and interface designed so that persistent memory can coexist with repeated computation and not every latent-universal construction is necessarily compatible with this regime.

The program at first macrostep is initialized as $p_0\in \mathcal{S}$p zero in S, so at every macrostep $i$i the model is initialized as $s_{0,i}=p_i\oplus E(x_i)$s zero i equals p i combined with the encoding of x i, evolves by microsteps $s_{t+1,i}=f(s_{t,i})$ until halting at time $T_i$, outputs $y_i=D(s_{T_i,i})$, and sets the next program to $p_{i+1}=R(s_{T_i,i})$. If some prediction never halts, the sequential process fails. In general, this cannot be avoided without restricting the class of computations the model can express.

In control or reinforcement-learning settings, it is natural to augment each macrostep input with reward information, writing $x_i^{\prime }=[x_i,r_i]$x prime i equals x i comma r i, where $r_i$r i is the instantaneous reward from the previous macrostep.

Retained state is not merely passive memory. Because each new computation starts from the retained program $p_{i+1}=R(s_{T_i,i})$, the stream of inputs can influence the future computation itself. In latent-universal settings, universality implies that the substrate can in principle represent any algorithm, while the latent nature of the program makes that algorithm part of the mutable state itself, suggesting the following conjecture: given sufficient memory and computation time, and providing the reward as an input, there exist some EMs that implement internally: (i) a prediction subroutine, which maps the current input to an output, and (ii) an update subroutine, which uses reward signal to modify a designated sub-portion of the program. In this view, the model could not only execute a prediction but also a feedback-driven self-improvement procedure acting on its own latent program, realizing a form of meta-learning [39]. However, this remains speculative and requires further analysis, especially to determine the maximum degree of self-editing plasticity a substrate can support and the implications for robustness.

2.6 Extensions and Relaxations

The Emergent Model theoretical framework extends naturally to continuous-time dynamical systems, and in particular to continuous media governed by local field dynamics.

At the most general level, one may replace the discrete transition $s_{t+1}=f(s_t)$ with a continuous-time evolution law:

This ODE form is an abstract state-space description and does not, by itself, impose any notion of locality. A more structured extension is obtained when the state is a field over an Euclidean spatial domain $\Omega \subseteq {\mathbb{R}}^n$omega, a subset of n-dimensional real space, so that:

where the $m$m channels may represent multiple interacting quantities. The dynamics are then specified by a local evolution law of PDE type, for example:

This yields a natural notion of continuous-space computation, encompassing substrates such as reaction-diffusion systems and other fluid-like media.

Under these extensions, the interface formalism remains unchanged: the encoder $E$, decoder $D$, halting $H$, and combining operator $\oplus$ are still defined as maps on the state space. The halting time is defined analogously as:

The halting time T of s zero is the infimum of t greater than or equal to zero such that H of s of t equals one

whenever this quantity exists and is finite.

Another natural relaxation is allowing stochastic evolution rules, where the dynamics include random fluctuations or noise, so the same initial state may generate different trajectories. The induced output is then stochastic.

A more practical relaxation is to cap, or even fix the internal computation time. Instead of running the dynamics until an explicit halting condition is met, one may apply the transition rule for a fixed time budget $T$. This reduces expressivity but also removes the possibility of non-halting predictions, makes inference easier to parallelize on hardware, and can simplify state retention. Under suitable conditions, this allows identity state retention $(R(s)=s)$R of s equals s: the final state can be used directly as the program for the next macrostep, $p_{i+1}\leftarrow s_{T,i}$, without requiring to cleanup a halting region.

A further, mostly notational relaxation is to reframe the state initialization in pseudocode. Instead of explicitly forming $p\oplus E(x)$, one practically sets the initial state as the program and then overwrites the encoded input into a designated subspace:

s0 <- p
I <- InputSubspace(s0, x)
s0[I] <- E(x)

Here $I$I denotes the input subspace, which may vary in size and location. The encoder becomes an operator acting on this subspace, and the assignment s0[I] <- E(x) represents an overwrite operation that injects the encoded input into that region, while the rest of $s_0$s zero is left unchanged.

For example, if the state space is a vector and the input is $x=[x_1,x_2]$x equals x one, x two, the encoder may simply write the input into fixed coordinates:

s0 <- p
s0[2:4] <- [x1, x2]

Under this procedural view, an Emergent Model can be represented in pseudocode as follows.

For non-sequential tasks, without retention, a single prediction is:

Input x
Program p
 
s <- p
I <- InputSubspace(s, x)
s[I] <- E(x)
for t in microsteps do
  s <- f(s)
y <- D(s)

On sequential tasks, with retention:

Input stream (x_i)_{i in macrosteps}
Program p
 
s <- p
for i in macrosteps do
  I <- InputSubspace(s, x_i)
  s[I] <- E(x_i)
  for t in microsteps do
	s <- f(s)
  y_i <- D(s)
  s <- R(s)

2.7 Training

After choosing which components are trainable, optimizing an Emergent Model means searching for configurations that make the induced computation solve a desired task. In supervised settings, this amounts to minimizing a prediction error; in control and reinforcement-learning settings, to maximizing expected cumulative reward [10, 37].

Let $\theta$theta denote the trainable components and $\hat{y}={\Phi }_{\theta }(x)$y hat equals Phi theta of x denote the model’s output. In supervised learning we solve:

theta star is the argument that minimizes the expected loss between y hat and y

while in control tasks we solve:

theta star is the argument that maximizes the expected sum of rewards r i from zero to N minus one

In the most general case, $\theta$theta may include the program $p$p, the transition rule $f$f, the halting condition $H$H, and parts of the interface $E,D,\oplus$E, D, and the combination operator. It is useful to distinguish between two kinds of trainable components. We call soft parameters the ones belonging to the program $p$p. These are latent and dynamical, and may in principle change during inference under state retention. We call hard parameters the trainable components belonging to the transition function, halting and interface. These play a hardware-like role: they can be optimized during training, but remain fixed during inference.

In latent-universal settings only the soft parameters are trained for a particular task, while the hard ones $(S,f,H,E,D,\oplus )$S, f, H, E, D, and the combination operator remain fixed and reusable across tasks.

Crucially, the search is guided only by the decoded output: the intermediate microstep trajectory $s_0,\dots ,s_T$s zero through s T is left unsupervised, so any internal algorithmic structure that emerges is a byproduct of optimizing on the output error/reward alone.

Although many training algorithms can be employed, EM dynamics are strongly recursive, nonconvex, and often discrete, making gradient-based methods impractical or impossible to use. Evolutionary algorithms are therefore a natural choice, especially in the most open-ended scenarios. In latent-universal settings, any computable function is realized by some finite-support initial state $p\in \mathcal{S}$p in the state space S. In one-dimensional substrates, such as Turing machines, the program state can be represented as a finite program string $\rho$rho, followed by an infinite quiescent background $b$b, i.e. $p=\rho b^{\infty }$p equals rho followed by b infinity. Training is then viewed as a form of evolutionary program synthesis over variable-length strings, where mutation allows insertion and deletion of elements [14, 23].

Lemma 1 (Searchability)

Let $\mathsf{M}$M be latent universal and assume programs are finite-support states, i.e. $p=\rho b^{\infty }$ with $\rho \in \{0,1{\}}^{\star }$p is a finite string rho in the set of all binary strings followed by an infinite background. Then for every partial computable function $g:\{0,1{\}}^{\star }\rightharpoonup \{0,1{\}}^{\star }$g from binary strings to binary strings there exists such a finite string ${\rho }_g$rho sub g that allows the computation of $g$. In particular, since $\{0,1{\}}^{\star }$ is countable, enumerating programs $\rho$ (or sampling them at random with variable length) will eventually hit a program that computes $g$.

The searchability lemma clarifies why finite-support programs are essential for treating latent-universal EMs as machine-learning models. If the target program has finite support, then it can be searched by enumeration, random sampling or by a finite sequence of edits (insertions, deletions, mutations). In this sense, EM training is viewed as emergent program synthesis: rather than designing an algorithm at a high level and compiling it into the substrate, one searches over substrate-level initial states until a configuration inducing the desired behavior is found. The lemma provides only an asymptotic existence condition: a suitable program exists and is reachable in principle, but the search may be arbitrarily long and no efficiency guarantee is implied.

3 Experiments

The experiments described below are intended to validate the Emergent Model framework and demonstrate it is non-vacuous, rather than to benchmark it against optimized task-specific approaches. None of the tested substrate-interface combinations is shown to be latent-universal; we favor evaluating a broad zoo of minimal models over engineering one toward the full theory. The goal is to test whether simple physical-like substrates, equipped with minimal input-output interfaces and trained by evolutionary search, can exhibit use-

ful modeling capabilities, extrapolation, control, and adaptation.

Across experiments, we use generational population-based genetic algorithms (GAs) to optimize each model’s trainable components, such as program state, transition-rule parameters, or interface parameters. Each generation evaluates candidate models, selects high-performing individuals through tournament selection, and forms the next population through elitism, sparse crossover, and mutation. This provides a simple black-box training method for discrete, non-differentiable, and strongly recurrent substrates. The exact GA variant and fitness objective vary slightly across tasks, and we report representative training and evaluation runs rather than full multi-seed statistical analyses. In some control experiments, we apply state retention across macrosteps, episodes, and generations, so that inference and environment interactions modify the latent state that will be reused in subsequent evaluations. Thus the soft parameters are updated by the model’s own dynamics and interaction history. When a parent is selected, crossover and mutation are applied only to its hard parameters, while its final latent state is copied directly to the offspring. We refer to this convention as Lamarckian-style state inheritance: advantageous state changes can be inherited, even though the state is mutated by inference dynamics rather than by external noise.

The experiments are organized by task family: Arithmetic tasks test exact rule learning and extrapolation in discrete domains; CartPole tests simple closed-loop control; and Meta-Life tests a more complex control objective, with online policy adaptation.

3.1 Arithmetic tasks

We first evaluate Emergent Models on simple integer arithmetic tasks. Arithmetics is a natural test case for this framework because it requires learning an exact symbolic rule and extrapolating beyond the finite training range. Discrete local cellular automata are especially well suited to this setting, where inputs and outputs are integers, and computation can be represented through spatial interactions over a discrete latent state.

The learned program is fixed across datapoints of a task. For each input, the automaton is reinitialized from the program, then the input is encoded into the state, and the system is run until halting. No terminal state is retained, since these are non-sequential regression tasks.

3.2 EM43

We ran minimal experiments to test whether EM43, an Emergent Model made from a minimal discrete cellular automata, can learn simple arithmetic functions and extrapolate far beyond the training range. Training is performed jointly on the program and the rule for each of the following tasks: $x+1$x plus one, $2x$two x, $\text{round}(x/3)$round of x over three, $x\text{ mod }4$x mod four, $x\cdot (1+x\text{ mod }2)$x times, one plus x mod two and $a\cdot b$a times b. These tasks are respectively simple linear/affine expressions, periodic expressions with nontrivial asymptotic structure where extrapolation is typically difficult in machine learning, and finally a two-integer product task.

EM43 is a one-dimensional cellular automaton with one-sided tape, radius $1$one and $4$four values per cell, a position-based input-output interface and a majority-based halting rule. The value alphabet is $W=\{0,1,2,3\}$W, consisting of zero, one, two, and three, where $0$zero (white) is the quiescent background, $1$one (black) a generic active cell, $2$two (red) the input-output marker, and $3$three (blue) the control value used for halting. A local lookup-table (LUT) rule $F$F maps each radius-$1$one neighborhood $(s_t[j-1],s_t[j],s_t[j+1])$s t sub j minus one, s t sub j, and s t sub j plus one to the updated value of the central cell $s_{t+1}[j]$s t plus one sub j; the global update function $f$f is obtained by applying $F$F synchronously across the tape. To introduce minimal structure and reduce the search space, we fix some LUT entries: $000\mapsto 0$ to preserve quiescence, and $020\mapsto 2$, $200\mapsto 0$, $002\mapsto 0$ to preserve stability of the input marker when surrounded by quiescent cells.

Given a global automaton state $s_t$s t, let #alive denote the count of cells with non-quiescent values ($s_t[j]\ne 0$s t sub j not equal to zero), and #control the count of cells assuming the control value ($s_t[j]=3$s t sub j equals three). The computation halts when control cells are the majority among alive (non-quiescent) cells:

The halting function H of s t is one if the ratio of control cells to alive cells is at least one half

The tape at initial state $s_0$s zero is divided into a trainable program region of length $L_p$L p, a blank separator of length $L_s$L s, and an encoding region of length $L_e$L e, with total length $L=L_p+L_s+L_e$L equals L p plus L s plus L e. Let $x_0=L_p+L_s$x zero equals L p plus L s be the origin at the start of the encoding region. An integer input $x\in \mathbb{N}$x in the set of natural numbers is encoded by placing a red cell at position $x_0+x$x zero plus x. At halting, the output is decoded as the position of the rightmost red cell relative to the origin, $y=x_{rr}-x_0$y equals x r r minus x zero. Figure 6 shows a representative space-time plot of the automaton, illustrating the program and encoded input at $t=0$t equals zero, as well as the decoded output at halting time $t=T$t equals T. For the two-input task $a\cdot b$a times b, the pair $(a,b)\in {\mathbb{N}}^2$a comma b in natural numbers squared is encoded by placing two markers at $x_0+a$x zero plus a and $x_0+b$x zero plus b; since the task is commutative, their order is irrelevant. During training, predictions are treated as invalid if they fail to halt within a computation time budget $T_{\max }$T max, produce no decodable output, or hit the right tape boundary. During inference, we let the tape self-extend in length whenever active cells approach the right boundary, and impose no computation-time limit, making the device unbounded in space and time.

The present setting was adopted as a largely arbitrary, minimally engineered design. It should therefore be regarded as one among many possible automaton-interface combinations, and more effective ones may exist.

Training is performed on inputs in $[0,30]$the range zero to thirty, while extrapolation is evaluated on $[30,1000]$the range thirty to one thousand. For the two-integer multiplication task, training and evaluation follow the analogous bounded and extrapolation regimes over pairs $(a,b)\in {\mathbb{N}}^2$a comma b in natural numbers squared.

The learning algorithm is a generational island-based ge-

netic algorithm with tournament selection, pointwise mutation, crossover, elitism, and island migration (hyperparameter ranges are reported in footnote ⁴). Tournament provides selective pressure, while the island structure promotes parallel exploration and occasional exchange between subpopulations, helping to maintain diversity and reduce premature convergence [43].

Fitness $F$F is derived from the dataset mean absolute error (MAE), together with a small density penalty $k_d$k d that promotes sparser programs and a penalty $k_{\text{invalid}}$k invalid for invalid computations:

where $d$d is the fraction of alive cells in the program region, $p_{\text{invalid}}$p invalid is the fraction of invalid predictions in the dataset evaluated. Note that fitness depends only on the final decoded output (i.e. the position of the rightmost marker at halting); the intermediate space-time evolution is never supervised. Any structured internal computation therefore emerges solely from minimizing the scalar error on the output.

For each task, independent training runs are performed using a fixed population of 20 islands with 2000 individuals each (40000 total), and a number of generations increasing with task complexity. The parametrization consists of a trainable program of length $L_p$L p together with a lookup table of 60 entries. As both program cells and rule entries take values in a 4-state alphabet, each parameter corresponds to 2 bits, allowing an estimate of the total information content of the learned model.

3.2.1 Results

After training, exact-match accuracy is evaluated on the training and test sets and averaged over runs. Table 1 summarizes the results.

Table 1: Performance of EM43 across different tasks, showing number of runs, generations, parameter size, and accuracy.

Perfect extrapolation is observed, in all runs, for all one-input tasks considered. This behavior is especially notable for periodic functions such as $x\mathrm{mod}4$x mod four, having a nontrivial asymptotic structure. As discussed in the introduction, such behavior is not globally representable by FFNNs without task-specific engineering. The same holds for $x\cdot (1+x\mathrm{mod}2)$x times, one plus x mod two, which is an oscillatory function with linearly growing amplitude. By contrast, the two-input multiplication task $a\cdot b$a times b shows poor performance on both training and test sets and does not converge within the available compute budget.

Figure 6: Space-time plot of EM43 computing $\text{round}(x/3)$ for $x=47$, using the learned initial condition and update rule. The green line represents the origin of the encoding ($x_0$). The input is encoded as a red cell at position $(x_0+x)$ in the initial condition, and the output is marked by the position of the rightmost red cell at halting time with respect to the origin.

Plotting the space-time diagrams of configurations that generalize reveals an interesting phenomenon that we call geometric grokking: exact computation is typically carried by regular and interpretable geometric patterns that remain stable across input scales. In the case of $x\mathrm{mod}4$x mod four (Figure 7), the learned dynamics implement a simple mechanism that repeatedly shifts the red marker left by 4 and down by 4 until the system reaches a configuration in which blue cells are the majority and halting occurs. At that point, the red marker is found in the correct position to encode the output. This can be interpreted as an unconventional computing procedure realized through interactions in the latent space, and together with test accuracy results, seems suggesting the model has internalized the generative rule behind the data.

Figure 7: Spacetime plot of a model representing $x\mathrm{mod}4$x mod 4, visualized for different inputs (15, 30 and 70).

The task $x\cdot (1+x\mathrm{mod}2)$x times, 1 plus x mod 2 (see fig. 8) is a product between a linearly growing function $x$x and a periodic function $1+x\mathrm{mod}2$1 plus x mod 2, resulting in the condition: if $x$x is odd, return $2x$2 x; if $x$x is even, return $x$x.

Figure 8: EM43 spacetime plot for $x\cdot (1+x\mathrm{mod}2)$x times, 1 plus x mod 2. On the left for an odd input (33), on the right for an even input (34).

Different runs and tasks implement different geometric strategies, but they typically follow the same broad scheme: patterns propagate from the program region, collide with the input marker, and eventually trigger halting through changes in the active-cell population (see Figure 6 and Figure 8). One possible explanation of our results is that translation and time invariance of the update rule induce a homogeneous medium naturally suited to input scale generalization.

More complex tasks such as $a\cdot b$a times b showed very slow convergence, and within the available training budget it was not possible to determine whether zero loss would eventually be reached. At present, the source of failure cannot be clearly isolated. It may be due to insufficient training compute, hyperparameter choices (such as computation time and tape-length limits during training), interface limitations (the chosen encoding and decoding scheme), or insufficient expressivity of the automaton. Although the EM43 rule space likely contains weakly universal rules⁵, it remains unknown whether it contains a strongly universal rule and, if so, what interface would be compatible with it. Even under that optimistic assumption, search likely remains the dominant bottleneck, since genetic algorithms scale poorly in large combinatorial spaces and the number of possible programs grows as $4^{L_p}$4 to the power of L sub p.

A notable aspect of this setting is the small size: 70 parameters carrying 2 bits each, corresponding to 140 bits of information. The setup is also data-efficient, being trained on only 30 integer examples while extrapolating to much larger ranges. This combination of parameter and data efficiency is broadly consistent with observations in the Neural Cellular Automata literature [13, 47], and may suggest that local recursive computation is an important ingredient in the design of compact learning systems.

3.3 GoL-EM

We next test the EM framework in Conway’s Game of Life (GoL) on simple arithmetic tasks. The GoL rule is fixed, and training acts only on initial conditions. This matches the latent-program view from Section 2.4: task-specific computation is encoded in the initial condition, not in the transition rule.

However, GoL’s Turing completeness does not automatically transfer to this setup. Indeed the chosen input encoding, output decoding, and halting protocol can restrict access to universal constructions. Since our interface and halting policy is deliberately minimal and not proven to preserve universality, this experiment should be read as a small-scale test of initial-state optimization in a fixed cellular-automaton substrate, not as a demonstration of latent universality.

3.3.1 Methodology

We use a square GoL board as a fixed computational substrate. The automaton is binary-valued: 1 denotes a live cell and 0 denotes a dead cell, so the value set is $W=\{0,1\}$W equals the set containing 0 and 1. Ideally, the substrate is a square lattice extending indefinitely to the right and downward, while bounded on the left and upper sides; we write its vertex set as $V={\mathbb{N}}^2$V equals N squared, and the full state space as $S=W^V=\{0,1{\}}^{{\mathbb{N}}^2}$S equals W to the V, which is 0 1 to the N squared. In practice, memory is finite, so it uses a finite square lattice of side length $L$L, nominally $V=\{0,\dots ,L-1{\}}^2$V equals the set 0 through L minus 1, squared.

The board is partitioned along each axis into three consecutive regions: the program region of side $L_p$L sub p, a separator of width $L_s$L sub s, and the encoding region. The separator keeps the program region disjoint from the area where inputs are written. Because the board is square and the layout is symmetric, the encoding region begins at the same offset on both axes, defining the encoding origin $(i_0,j_0)$i zero, j zero with $i_0=j_0=L_p+L_s$i zero equals j zero equals L sub p plus L sub s. All inputs are encoded as positions relative to this origin. On a finite board of side length $L$L, the encoding region then spans $L_e=L-j_0$L sub e equals L minus j zero along each axis (See fig. 9).

To reduce the search burden, we do not optimize over arbitrary live/dead patterns in the program region; instead, we search over glider placements, using the upper-left corner

of each glider as a trainable position variable. These gliders travel towards the input region and eventually interact with the encoded inputs.

Figure 9: Automaton grid, here encoding the input $x=5$x equals five and with a two-glider program. Each encoded integer corresponds to a $2\times 2$two by two block of automaton cells at a specific position; to represent this faithfully, the grid shown here is scaled by a factor of two relative to the underlying automaton lattice. Tick marks on the horizontal axis indicate the integer values represented by block positions, with respect to the origin.

Inputs are encoded positionally, on the diagonal of the encoding region. For an input value $x$x, we place a single $2\times 2$two by two live block (denoted $1_{2\times 2}$one sub two by two) at a position whose offset from the origin is proportional to $x$x, with stride $s=2$s equals two. The block is placed along the diagonal: this layout is chosen because gliders travel diagonally in GoL, so a diagonal arrangement lets signals emitted from the program region reach the encoded input more easily. We use a $2\times 2$ block because it is a still life under the GoL rule, i.e. stable when isolated; the input therefore persists in place until those signals reach it.

The state then evolves under the standard Conway’s GoL update rule. Computation halts when the system reaches equilibrium/fixed point:

The halting condition H of s t equals one if and only if f of s t equals s t

At halting, the output is decoded by scanning the encoding region for isolated $2\times 2$ live blocks. Among all valid candidates, the leftmost block is selected, and its horizontal position relative to the encoding origin is converted back into a scalar using the same stride as in the input encoding. Unlike input, output blocks are not required to lie on the diagonal: only their horizontal position is used. Moreover, other live-cell debris that may remain elsewhere on the board is ignored by the decoder. This gives the dynamics more freedom to form a valid output and reduces the search burden.

Let $j_{\text{LB}}$j L B denote the horizontal position of the leftmost $2\times 2$ block (precisely, the column index of its upper-left corner), and let $j_0$j zero be the encoding-origin column. The decoder returns:

The decoded value D of s T is j L B minus j zero, all divided by two

During training, if the halting condition is not reached within a maximum time budget $T_{\max }$T max, the output is not decoded and a penalty is applied to the fitness. During evaluation, the same equilibrium-based halting condition is used, but the board is allowed to spatially self-extend to the right and downward whenever activity approaches the boundaries. Thus inference is not artificially limited by the initial board size. Evaluation is also allowed to run for an arbitrarily long time until equilibrium is reached.

We train the GoL program by minimizing prediction error while penalizing invalid computations:

The fitness F is minus the mean absolute error minus lambda invalid times p invalid

Here $\text{MAE}$ is the mean absolute error on the training dataset, and $p_{\text{invalid}}$ is the fraction of examples that failed to produce a valid prediction. This includes both non-halting runs and halted runs with no decodable output block.

We optimize the program with two black-box search methods over glider positions in the program region: a genetic algorithm with mutation on glider positions, and plain random search that samples positions uniformly. The two reach almost identical results, so we report random search throughout, being the simpler. This near-equivalence is itself informative about the search landscape, and we return to it below.

3.3.2 Results

The GoL model successfully learned and generalized tasks of the family $x\mapsto x+k$x maps to x plus k, such as $x-2,x-1,x+1,x+2$x minus two, x minus one, x plus one, x plus two, and $x+5$and x plus five. By contrast, a more structured conditional task, returning $x+1$ for even inputs and $x-1$ for odd inputs, was not successfully learned. The input ranges are given in Table 2, the results in Table 3, and a representative model prediction in Figure 10.

Table 2: Training and evaluation input ranges for GoL arithmetic tasks. Lower bounds are chosen so that the target output is non-negative, since the positional decoder is not designed to represent negative integers.

Table 3: Performance of the GoL-based Emergent Model across tested tasks, trained by random search over glider positions. $n_{\text{evals}}$n evals is the search budget (number of candidate programs sampled and evaluated). Accuracies are exact-match rates.

Figure 10: Observing a few steps of the trace computing $y=x+2$

Several limitations remain. First, although GoL itself is universal under proper constructions (such as Rendell’s [32])such as work by Rendell, it is unknown whether the interface and halting condition chosen here preserve universality. Second, we observed that random search performs almost as efficiently as the genetic algorithm, suggesting that the search landscape is weakly structured, as the program space appears to lack a natural ordering or smoothness: small variations in glider positions do not seem to induce correspondingly small variations in fitness.

3.4 Control tasks

We next evaluate some Emergent Models in control settings. Differently from arithmetic tasks, the goal here is not extrapolation over unbounded ranges: observations and actions are typically bounded by the environment. Instead, it tests whether a simple substrate can support closed-loop behavior and robustness.

State retention becomes more relevant in this setting, because the controller acts on a stream of observations rather than on isolated datapoints. Instead, adaptive computation time through explicit halting is less central here than in static arithmetic tasks: in a control loop, computation is already distributed across macrosteps, since each new observation triggers another round of internal updates. For this reason, some of the controllers below use a fixed computation time budget $T$T rather than a dynamic halting condition.

We consider two environments. The first is CartPole, a standard continuous-state control task in which the agent must balance a pole by applying forces to a cart. The second is Meta-Life, a simple 2-d spatial navigation and foraging environment, which we study in two variants: Meta-Life-Food, where agents learn to move and collect food, and Meta-Life-Adapt, where resources alternate between food and poison in phases of random duration: during a food phase, collecting any resource gives a positive reward; the world then switches to a poison phase, in which collecting a resource gives negative reward; it later switches back to food, and so on. The agent receives no a priori information about the current phase, and hence about whether resources are currently food or poison, since they are visually identical; it must therefore taste a resource, observe the resulting reward, infer the current phase from it, and adapt online, chasing resources during food phases and avoiding them during poison phases. Both variants share the same world and sensors, differing only in the reward assignation.

3.5 CartPole

We use CartPole as a minimal closed-loop control task. At each environment macrostep $i$i, the controller observes the cart-pole state:

x i is defined as the vector containing cart position, cart velocity, pole angle, and angular velocity

where $q_i$q i is cart position, ${\dot{q}}_i$q dot i cart velocity, ${\theta }_i$theta i pole angle, and ${\dot{\theta }}_i$theta dot i its angular velocity. The controller outputs a force action $F_i$F i applied to the cart.

We test two action formulations: a continuous force $F_i\in [-F_{\max },F_{\max }]$F i in the range negative F max to F max, and a binary force $F_i\in \{-F_{\max },+F_{\max }\}$F i as either negative or positive F max.

A rollout is one CartPole episode, lasting at most $N_{\max }$N max macrosteps. The objective is to keep the pole balanced for as long as possible within this time window, without exiting

the allowed position range. At each macrostep, the agent receives reward $+1$plus one if the cart remains within the allowed position range and the pole within the allowed angle range. If either condition is violated, the agent dies receiving no further reward, and the rollout is terminated. The return of rollout $r$r is therefore:

R sub r is the sum from i equals zero to N minus one of the indicator that the agent is alive at macrostep i

Standard CartPole is considered solved when an agent consistently achieves a return of $\approx 500$approximately 500. In our experiments, depending on the setting, we cap the maximum number of environment steps (and therefore the maximum possible return) at either 500 or 800. An agent that selects actions uniformly at random achieves an average return of $\approx 20-25$approximately 20 to 25.

3.6 GoL-EM

We test Conway’s Game of Life (GoL) as a fixed rule cellular-automaton substrate for CartPole control. The question is whether nontrivial control behavior can be obtained by training only the initial GoL state with a fixed encoding/decoding mechanism. In EM notation, the controller has the form:

F sub i equals the decoding of f GoL applied T times to the combination of program p and encoded observation x sub i

Figure 11: Board layout of the GoL-EM controller for CartPole. The learned GoL program is placed in the upper-left region, separated from the input-output interface by a buffer zone. Four horizontal lanes encode the CartPole variables $q,\dot{q},\theta ,\dot{\theta }$. In each lane, one of six spatial bins is activated according to the discretized value of the corresponding variable. Blue cells indicate input-block locations, while red and green cells indicate left and right output-collector regions used for force decoding.

where $x_i=(q_i,{\dot{q}}_i,{\theta }_i,{\dot{\theta }}_i)$x sub i is the vector of cart position, velocity, pole angle, and angular velocity is the CartPole observation at macrostep $i$i, $p$p is the learned GoL program, $T$T is the internal GoL computation time budget, and $F_i$F sub i is the decoded force applied to the cart. In our setting, each episode was capped at 500 CartPole macrosteps.

This EM is purely reactive and does not use state retention. At every CartPole macrostep, the board is reset to the same learned program $p$p, the current observation is encoded, the GoL dynamics are run, and the final configuration is decoded into an action.

The learned program $p$p is placed in an upper-left region of the board. Program is constructed by combining a small number of gliders together with sparse binary perturbations. Interface geometry is fixed: the CartPole variables are written into a designated input region, and the force is read from designated output-collector regions.

The encoder $E$E maps the continuous CartPole observation to a GoL pattern. Each of the four observation variables is first clamped to a useful range and discretized into one of six bins, ordered from left to right by increasing value. Each variable has its own horizontal lane, and the selected bin in that lane is activated by writing a $2\times 2$two by two block of live cells. Thus each CartPole observation is represented by four active input blocks, one per variable (see fig. 11).

Each input bin (I) is paired with two nearby output-collector regions, one contributing to a left-force vote (L) and one contributing to a right-force (R) vote. The local motif is shown in Figure 12.

Figure 12: Single-bin motif used in the GoL-EM CartPole interface. If this bin is selected, the blue $I$ zone is activated as a $1_{2\times 2}$ alive block. After GoL evolution, live cells in the red $L$ and green $R$ regions contribute to the aggregate left and right force votes.

After encoding, the board evolves under the standard Conway’s GoL rule for a fixed computation time $T$T. If the configuration reaches a fixed point before this budget is exhausted,

i.e. $s_t=f(s_t)$s t equals f of s t, the run stops early and we set $H(s_t)=1$H of s t equals one, since further GoL steps would leave the board unchanged. If no stable configuration is reached, the terminal state is still decoded, but a small fitness penalty is applied to encourage halting.

Then, the decoder aggregates activity over all left and right output-collector bins. Let $L(s_T)$L of s T and $R(s_T)$R of s T denote the total count of live cells in the left and right collectors at the terminal state $s_T$s T. The applied continuous force is:

The force F is the clipped value of the gain g F times the difference of R and L over their sum plus epsilon, within the range of negative F max to F max

Where $g_F$g F is a force gain hyperparameter, not trained. Thus the direction and magnitude of the force are determined by the imbalance between activity in right and left collectors.

Training uses a mutation-only genetic algorithm over gliders position, with occasional sparse binary mutation on the program region. For one episode, the fitness is $F=R-{\lambda }_{\text{nohalt}}{\rho }_{\text{nohalt}}$F equals R minus lambda no-halt times rho no-halt, where $R$R is the CartPole return, ${\rho }_{\text{nohalt}}$rho no-halt is the fraction of CartPole macrosteps in which the GoL computation failed to halt within the time budget, and ${\lambda }_{\text{nohalt}}$lambda no-halt controls the strength of this penalty.

Importantly, the interface layout and computation time budget should be understood as simple heuristic design choices, not as the result of automated optimization over many possible interfaces. The budget $T$T was chosen to allow gliders have enough time to reach the furthest input/output locations and interact, while bin and lane spacing were chosen to limit excessive interference between neighboring regions. More effective interface choices may exist.

3.6.1 Results

Figure 13: Learning curves for the representative run, showing mean and best CartPole return across generations. The maximum attainable return is 500.

GoL-EM achieved CartPole returns clearly above the random baseline of $\approx 20$approximately 20. In the representative training run reported here, the evolved population reached a mean return of 171.36, while the best controller reached 417.20. We then saved this best controller and evaluated it on 20 new episodes, obtaining a return of $316.90\pm 121.52$. During evaluation, the mean non-halting rate was 54.5%, meaning that in more than half of the CartPole macrosteps the model didn’t halt before maximum time budget. (Results are summarized in table 4, and the learning curve in fig. 13).

Qualitatively, the learned policy was simple but meaningful. The controller typically adopted a one-sided balancing strategy: it pushed the pole towards one side and then maintained balance through slow cart motion until eventually exiting the allowed position bounds. Since the vast majority of inspected episodes ended by exceeding the cart-position range rather than the pole-angle range, the policy appears to use angular information far more effectively than positional information, possibly ignoring positional information at all. This suggests that GoL-EM is capable of control behavior, but with several robustness-related limitations.

Figure 14 shows some representative snapshots of GoL state during a prediction. The dynamics are driven by glider motion and collisions with the encoded input blocks.

Figure 14: Snapshots of GoL evolution during 1 CartPole macrostep.

Table 4: Results from the representative run.

We also briefly tested state retention by using the final GoL configuration from one CartPole macrostep to initialize the next (we employed identity retention $R(s)=s$R of s equals s). In

the present setting, this made the controller unstable and dropped its performance close to random, as residual debris from previous macrosteps accumulated and interfered with the next encoded observation in near-chaotic ways.

3.7 CEM1D

We then explored a continuous-valued, one-dimensional Emergent Model (CEM1D) for CartPole. This experiment differs from the previous GoL controller in two ways. First, the substrate is not a discrete binary cellular automaton, but a continuous-valued local dynamical system. Second, the controller uses retention across CartPole macrosteps: its internal state is retained from one control step to the next, rather than being reset after every action.

The latent state is a one-dimensional lattice of $L$ cells, $s\in {\mathbb{R}}^L$s in the set of real numbers of length L. Its dynamics are defined by a local radius-1 update rule. In continuous-time form, the rule can be written as:

where $s_j$s sub j is the value of the cell at lattice position $j$j. The coefficients $a_l,a_c,a_r$a sub l, a sub c, and a sub r are trainable local interaction weights for the left, center, and right cells, respectively. The cubic term provides nonlinear damping and helps prevent exponential growth of the state. Its coefficient is fixed to 0.03, and not trainable.

In our implementation, we integrate with explicit Euler steps of fixed size $\Delta t=0.1$delta t equals zero point one:

Thus the model is implemented as a discrete dynamical system, although being motivated by a continuous-time update rule. Boundary conditions can be periodic, reflective, or zero-padded. In the representative run reported here, we used reflective boundaries.

We fix, once for all, a set of ports into the lattice as positions for writing/reading inputs and outputs. At each CartPole macrostep, the physical state $x=(q,\dot{q},\theta ,\dot{\theta })$x, composed of q, q dot, theta, and theta dot is normalized and injected into four input ports by overwrite. If $x_k$x sub k is the $k$-th input component and $j_k$j sub k is its corresponding fixed port location, the encoding operator is:

The system then evolves for a fixed duration of $T$T microsteps, without employing a halting operator.

After time evolution, a designated output cell $s_{j_{\text{out}}}$s sub j out is read and thresholded to produce the discrete CartPole force:

Equivalently,

The random tie-breaking region around zero is employed to avoid a degenerate control strategy observed in preliminary runs, where the controller exploited extremely small output values, for example on the order of $10^{-6}$ten to the minus six. Such near-zero decisions can be numerically fragile and are also poorly suited to long-term state retention. The chosen decoder therefore enforces robust binary actions.

Only the local interaction parameters $(a_l,a_c,a_r)$ are explicitly trained. The latent state $s$s is implicitly optimized with Lamarckian-style inheritance (see Section 3).

As a representative minimally engineered configuration, we used a one-dimensional lattice of length $L=19$L equals nineteen. The four CartPole observations are written into input cells at positions (3, 6, 12, 15), and the action is read from a single output cell at the central position 9; this places two input cells on each side of the central readout, a deliberately simple symmetric arrangement (Figure 15). The dynamics are integrated with explicit Euler steps of size $\Delta t=0.1$delta t equals zero point one for a fixed computation time of $T=30$T equals thirty microsteps per macrostep, with reflecting boundary conditions.

Figure 15: Layout of CEM1D. The CartPole state variables are written into designated input ports, and the force is decoded from the central output port.

3.7.1 Results

In our experiments, the maximum episode length is 800 macrosteps. Fitness $F$F is computed by evaluating the controller over multiple episodes (rollouts) and combining their returns into a single score that rewards high mean return while penalizing variability across rollouts:

where $R_r$R sub r is the return obtained in rollout $r$r, the mean and standard deviation are taken over rollouts, and ${\lambda }_{\text{std}}$lambda sub std controls how strongly variability is penalized. Dividing the mean return by an increasing function of its standard deviation favors controllers that perform robustly across different environment initializations, rather than ones that occasionally survive for a long time.

CEM1D learned high-performing CartPole policies, reaching returns close to the maximum of 800 and far above the random baseline of $\approx 20$approximately twenty. The best controller by training fitness achieved a return of $737.32\pm 104.63$seven hundred thirty seven point three two, plus or minus one hundred four point six three during training. Re-evaluated over 1000 new rollouts, it achieved a mean return of $734.93\pm 107.00$, confirming that the learned behavior

reflects a real control policy rather than a lucky selection bias. (Results are summarized in Table 5 and the learning curves in Fig. 16).

Table 5: Results for a representative CEM1D CartPole training run.

Figure 16: Best controller return and its standard deviation across training generations.

For the selected controller, the learned update equation is:

A representative space-time diagram of the automaton is reported in Fig. 17.

Figure 17: Internal “reasoning” trace of the CEM1D controller during two environment steps. Rows correspond to time evolution in microsteps, and columns correspond to lattice cells in the one-dimensional medium. The horizontal line at $t=31$ marks the boundary between consecutive macrosteps, where the input is re-injected.

The learned policy substantially outperforms the GoL-EM controller and is very robust to deviations in the pole angle, but it still exhibits some fragilities. In nearly all manually inspected rollouts, the cart eventually drifted out of the allowed position range once enough time had passed. This occurred despite using a boundary-heavy initialization distribution, with 70% of training episodes starting near the cart-position boundaries to encourage robustness. Qualitatively, this suggests that the controller learned an effective pole-balancing strategy, but did not integrate correctly the cart-position information into its decision process.

This limitation is consistent with the structure of the

learned dynamics. The update rule is predominantly linear, and this may limit expressivity: the cubic term has a very small coefficient (0.03) and contributes substantially only when cell values become large, so for most of the operating range the lattice evolves close to a linear system. Two further design choices restrict expressivity: the fixed input and output port layout may constrain how the local dynamics can route information through the lattice, and the fixed computation time $T$T, rather than an adaptive halting mechanism, caps the number of internal steps available per decision.

Figure 18: Information transport with right and left propagating patterns.

Interestingly, the evolved dynamics exhibit localized structures that behave like information carriers, similarly to gliders in discrete cellular automata (See fig. 18). In particular, two nearby cells with similar values tend to propagate their state to the right over successive microsteps, while two nearby cells with opposite values, forming a strong local gradient, tend to propagate information to the left. The left-moving transport appears stronger and more robust than the right-moving. This asymmetry may partly explain why position information, which would need to propagate rightward toward the output port, is used less effectively by the controller.

The CEM1D CartPole experiment should therefore be interpreted as an exploratory feasibility result rather than a mature control method. Nevertheless, the controller achieves high returns despite its limited use of cart-position information, comfortably exceeding the standard solution threshold of 500.

3.8 Meta-Life

We next tested Emergent Models in a simple embodied environment, which we call Meta-Life. Meta-Life is a continuous two-dimensional toroidal world in which agents sense, navigate and collect resources (see fig. 19).

Figure 19: Meta-Life environment. Agents (blue dots) move and collect resources (green dots).

Each agent has a position in the 2D world and a heading that defines the direction it faces. It senses the environment through three lidar rays cast at fixed angles relative to its heading, so the rays rotate with the agent. Each ray returns a scalar in $[0,1]$between zero and one, indicating the proximity of the nearest resource along its direction: 0 means no resource is detected, while larger values indicate a closer resource (see fig. 20).

The observation is therefore:

x i is a vector of three lidar signals, l one, l two, and l three

where ${\ell }_{1,i},{\ell }_{2,i},{\ell }_{3,i}\in [0,1]$the lidar signals are the lidar signals at macrostep $i$i. The controller outputs the rotational speed ${\omega }_i$omega i and the forward speed $v_i$v i, allowing the agent to steer and move.

the output y i consists of the rotational and forward speeds

Figure 20: Meta-Life agent with its three lidar rays.

We consider two regimes. In Meta-Life-Food, every resource is food, and collecting one gives reward $+1$. This is an ordinary foraging task: the agent must use its lidar observations to steer toward food and collect it.

In Meta-Life-Adapt, resources alternate between food and poison in phases of random duration, while remaining visually identical: during a food phase, collecting any resource

gives reward $+1$plus one; the world then switches to a poison phase, in which any resource gives reward $-1$minus one; it later switches back, and so on. Since food and poison produce identical lidar readings, the agent’s observation carry no information about the current phase, and cannot tell in advance whether the next contact will be good or bad. It must therefore periodically “taste” a resource, observe the resulting reward (observations are augmented with a reward signal; see below), and adapt online, switching between a “chase resources” and an “avoid resources” mode without retraining.

Each rollout consists of two mirrored episodes, A and B, each composed of four alternating phases. Episode A runs:

food ($T_1$T sub one) $\to$ poison ($T_2$T sub two) $\to$ food ($T_3$T sub three) $\to$ poison ($T_4$T sub four)

and Episode B runs the same phases with reversed reward signs:

poison ($T_1$) $\to$ food ($T_2$) $\to$ poison ($T_3$) $\to$ food ($T_4$)

Here $T_1,\dots ,T_4$T one through T four are phase lengths in macrosteps, resampled independently for each rollout from a uniform distribution over roughly $(0,600)$zero to six hundred.

We use two variance-reduction measures. First, the A/B symmetry above makes the score less sensitive to any particular phase-length sample, since each phase ordering is evaluated under both sign assignments. Second, each agent is evaluated over several rollouts per generation ($N_{\text{rollouts}}$N rollouts) and its score averaged. Both limit the influence of a lucky or unlucky sample on the estimated fitness.

The observation is augmented with a transient reward-feedback signal:

x i is a vector of lidar readings l one through l three and the reward signal r i

where $r_i\in \{-1,0,+1\}$r i belongs to the set minus one, zero, and plus one. The signal defaults to $r_i=0$r i equals zero when no contact occurs; after contact with a resource it is set to the collected reward ($+1$ for food, $-1$ for poison) and held for 4 macrosteps before returning to 0.

For diagnostic purposes, we classify scores by their timing relative to the previous contact. A reward earned from a new contact within 6 macrosteps of the previous one falls in the reaction-window: the period in which the reward signal from that previous contact may still be present in the controller’s input. Rewards earned later fall in the after-reaction bucket.

After-reaction scores are therefore the stricter diagnostic: they measure whether the agent has inferred the current phase and retained it as a persistent internal memory, rather than merely reacting to the instantaneous reward signal. Since a phase lasts on average 300 macrosteps, far longer than the 6-macrostep reaction window, good adaptive behavior requires holding this phase memory across many reward-free macrosteps.

For a single agent and rollout, return is the cumulative reward:

the return R is G minus B

where $G$G is the number of good contacts (collected food) and $B$B is the number of bad contacts (collected poison).

For Meta-Life-Adapt, we also calculate the meta score:

the meta score is G minus B divided by the quantity G plus B plus epsilon

A meta score of 0 indicates phase-insensitive behavior, i.e. no adaptation to the changing phases. A score of 1 indicates perfect selectivity, with only food contacts and no poison contacts, while a score of $-1$ indicates the worst case, with only poison contacts and no food contacts. The term $ϵ$epsilon is a small constant that prevents division by zero when no contacts occur. We also report the same score restricted to reaction-window and after-reaction to verify whether adaptation is persistent.

During training, the food-only objective is the raw return reduced by two penalty terms:

the objective F equals G minus lambda loop times L minus lambda invalid times I

where $G$G is the number of food items collected and ${\lambda }_{\text{loop}},{\lambda }_{\text{invalid}}$lambda loop and lambda invalid are penalty coefficients. The first term penalizes persistent rotation: $L$L is a moving average of the agent’s angular velocity, so steering sustained in one direction accumulates penalty while brief turns in opposite directions average out. This discourages a degenerate attractor, observed in preliminary runs, in which the agent spins endlessly in place instead of exploring the environment. The term $I$I is the fraction of macrosteps in which the run becomes numerically invalid, i.e. a NaN or Inf appears in the action or in the model’s internal state.

Meta-Life-Adapt employs a slightly different objective:

F is the square root of G plus epsilon minus the square root of B plus epsilon, minus the loop and invalid penalties

The square root exponent forces diminishing returns in the absolute number of contacts. This reduces pressure to collect as many resources as possible and instead favors policies that are more selective, i.e. that maintain a good food/poison ratio even if collecting fewer total resources.

Meta-Life-Adapt tests in-distribution adaptation, a weaker regime than meta-learning: the agent must switch its policy online from the observed reward, but both regimes are seen during training and the model is never asked to adapt to a novel task. Online policy switching from a feedback signal is nonetheless a precondition for meta-learning, since a system unable to switch policy in response to reward could not adapt to unseen tasks. We therefore view this setting as a precursor to meta-learning.

CEM2D CEM2D is a continuous two-dimensional Emergent Model featuring a fully local update rule. Its state has two channels,

the state s consists of channels c and m, which are matrices of dimensions H by W

where $c$c is a fast activation field and $m$m a slow memory field; in the experiments below the lattice is square, $H=W=8$H equals W equals 8, with each cell indexed by $(i,j)$i, j.

This state is paired with an interface $(E,D)$E, D that writes inputs to and reads outputs from specific lattice cells (ports). CEM2D uses no explicit halting predicate $H$: instead, the reasoning-time budget is itself a trainable scalar $T$T, constrained to the interval $[2,10]$the interval 2 to 10 and held fixed during inference.

The encoder writes each observation component $x_k$x sub k into a port, a single lattice cell, rather than spreading it densely across the lattice. Each port has a trainable location: the coordinates $(i_k,j_k)$i sub k, j sub k of its cell are themselves learned parameters, so training decides where each input component enters the lattice. At its ports, the encoder applies a pointwise affine map to the observation and then overwrites the activation state:

c at i k, j k is set to w k times x k plus b k, for k from 1 to 4

where $w_k$w sub k and $b_k$b sub k are per-input trainable scalars setting the scale and the bias of the $k$-th component. The encoder thus commits only a few cells to input, while leaving two things to training: the port locations and their input scaling. This keeps the interface low-dimensional yet flexible.

We employ a fully local update rule $f$f: every cell updates from quantities computed over its own neighborhood. Define the Moore-neighborhood mean of the activation field around cell $(i,j)$:

sigma i j is the average value of c in the 8-cell neighborhood of cell i j

At each microstep, every cell first computes a scalar preactivation $q_{ij}$:

The activation field is then updated by a residual step:

The memory field is updated through a local write gate $g_{ij}\in [0,1]$g i j between 0 and 1, computed from the same preactivation:

The gate controls how strongly each memory cell moves toward the current activation at the same position: a closed gate leaves the memory unchanged, while an open gate moves it toward $c_{ij}$c i j.

All coefficients are global scalars, shared by all cells. The rates $\alpha$alpha and $\beta$beta scale the update step. The fixed leak $\lambda =0.25$lambda equals 0.25 damps the activation field and helps prevent unbounded growth of $c$c. The biases $a_0$a zero and $a_1$a one act in different places: $a_0$ in the residual activation update, $a_1$ in the preactivation $q_{ij}$. The gate is set by its bias $g_b$g sub b and gain $g_k$g sub k. The transition function $f$ is the aggregate one-microstep map these equations define over the whole lattice. Wrapping boundary conditions are applied.

After the $T$T microsteps,

the initial state s zero i is transformed by f to the power of T into state s T i

the decoder reads two output ports from the activation field $c$ and maps their values to the action $({\omega }_i,v_i)$omega i, v i. As in the encoder, each output port has a trainable location and a per-output affine map, applied pointwise.

The retention operator used here is identity retention, $R(s)=s$R of s equals s. This is appropriate because CEM2D employs no explicit halting condition, so the terminal state does not satisfy any halting predicate and can be reused directly. The terminal state therefore becomes the program state for the next macrostep:

The fields $c$ and $m$ are retained not only across macrosteps but also across generations, using the Lamarckian-style state inheritance protocol described in Section 3: the selected parent’s final $c$ and $m$ fields are copied directly to its offspring, while crossover and mutation are applied only to the hard parameters.

Memory-augmented RNN baseline We compare CEM2D with a small dense memory-augmented recurrent neural network, denoted mRNN, featuring a fast hidden state $h_i\in {\mathbb{R}}^d$h i in R d and a slow memory state $m_i\in {\mathbb{R}}^d$m i in R d, with $d=6$d equals 6. Unlike CEM2D, it employs no explicit gating. It performs one update (microstep) per environment macrostep, corresponding to $T=1$T equals 1. That is, it performs no looped internal computation, which places it close to standard recurrent controllers. The update rule is:

and decodes to:

Unlike CEM2D, this model uses dense recurrent matrices, so each hidden unit can depend on every hidden and memory component; its update rule is therefore nonlocal. Its state $(h,m)$h, m is retained and inherited under the same Lamarckian convention as CEM2D.

Table 6: Meta-Life controller sizes

As shown in Table 6, CEM2D employs relatively few hard parameters (fixed weights), while using a larger number of soft parameters (latent-state size). The mRNN has the opposite structure: it uses more hard parameters, but a smaller latent state. In CEM2D, the latent-state size is decoupled from the number of hard parameters, since the local update rule is shared across all cells and therefore independent of the lattice size. For the mRNN, by contrast, the two are coupled: the dense recurrent matrices make the number of hard parameters grow quadratically with the hidden-state dimension, $O(d^2)$order d squared. A more systematic evaluation would require testing both architectures across multiple latent-state and hard-parameter scales; here, we consider only an mRNN with $d=6$d equals six and a CEM2D model with an $8\times 8$eight by eight grid.

Figure 21: Meta-Life-Food training return per agent per rollout in generations.

3.9 Training setup

Both controllers were trained with the same population-based genetic algorithm. At each generation, agents were evaluated over a fixed number of rollouts and were assigned a fitness score; the next generation was then produced through tournament selection, elitism, sparse crossover, and mutation. Under this scheme, an offspring inherited its hard parameters primarily from a base parent, with a donor parent contributing only a sparse subset of genes through crossover; these hard parameters were then mutated. Soft parameters, instead, were inherited unchanged from the base parent.

The food-only environment used population $A=100$A equals one hundred and was evolved for 700 generations. Meta-Life-Adapt used population $A=300$A equals three hundred, evolved for 510 generations for CEM2D and 260 for the mRNN baseline, since CEM2D required longer to converge. The best agent by training fitness was selected and evaluated on a batch of new rollouts. We report representative runs rather than multi-seed averages: for Meta-Life-Adapt each run takes a few days on a consumer laptop, so multi-seed statistics are left for future work. (Training setup is summarized in table 7).

Table 7: Training setup for representative Meta-Life runs

3.10 Results

In Meta-Life-Food, both controllers learned effective food foraging policies. The mRNN learned faster early in training, while CEM2D caught up later and reached a comparable return. On evaluation, CEM2D reached a return of 133 per agent per rollout, while mRNN 146. (Results are shown in table 8, while learning curves in fig. 21).

Table 8: Meta-Life-Food results. Return is measured per agent per rollout.

Meta-Life-Adapt was more challenging: the mRNN achieved a meta-score about 15% higher than CEM2D. Nevertheless, both models obtained high meta-scores relative to the optimum value of 1, indicating successful adaptation in both cases. We also observed that CEM2D required substantially longer training and improved more slowly. (Results are shown in table 9).

Table 9: Meta-Life-Adapt evaluation results. Random baseline is 0, perfect behaviour is 1.

Lamarckian inheritance of the $(c,m)$c comma m state shaped the initial condition (program) of the current run, shown in fig. 22. In the trained model, the coefficients coupling $c$ to $m$ are small and the write gates rarely activate, so the memory field $m$ stays nearly static over time. We then took this trained model, with its inherited state carried across generations, and ablated memory: setting $m=0$m equals zero appears to reduce the meta-score only slightly in the short term, but leads to fully degenerate behavior over longer rollouts.

Two readings fit this: either $m$ is functionally useful, acting as a static “hardware” that shapes the computation; or $m$ is inert, and the collapse only reflects that the hard parameters co-adapted to the inherited state during training, so that ablating it moves them off the operating point they were fit to. A conclusive test would require two separate training runs:

one with the state inherited across rollouts and generations, as here, and one in which $m$m is reset to zero at the start of every rollout, retained across macrosteps within a rollout, but never carried across successive rollouts or generations. Comparing the two would show to what extent the model’s expressivity comes from the program (inherited state). This is left for future work.

Figure 22: Learned initial condition of the CEM2D controller on Meta-Life-Adapt settings.

Interestingly, during inference we observe a phase-related interpretable feature near the reward input port $I_3$I sub three. The cell in the upper-left neighborhood of $I_3$I sub three correlates strongly with the agent’s estimate of the current resource regime (see fig. 23). Its activation is high after the agent collects food ($c\approx 3$c is approximately three) and low after it collects poison ($c\approx 1$c is approximately one), tracking the regime the agent has most recently estimated. The cell is highly responsive to reward but has a long effective time constant in its absence, so it updates rapidly on contact yet preserves its value across several reward-free macrosteps.

Figure 23: Interpretability visualization of the learned CEM2D dynamics on Meta-Life-Adapt. The two panels show the lattice activation state when the agent estimates a positive (food) phase and a negative (poison) phase. The arrow indicates the regime-correlated cell.

This cell also appears to modulate behavior. When its activation is high, the agent chases resources and collects them; when it is low, the agent rotates in place, reducing the chance of collecting poison. After the environment switches back to food, an eventual reward reactivates the cell, and the policy returns to resource-seeking. The resulting strategy is simple but effective: seek resources during food phases, rotate in place during poison phases. Although crude, it is sufficient to produce adaptation.

We observe several robustness-related failure modes, most visible on evaluation rollouts substantially longer than those seen during training. The most common is a rotating attractor, present in both CEM2D and in the mRNN: the agent rotates in place indefinitely, especially after a long period in which no resource enters its lidar rays. This occurs despite the looping penalty, though that penalty strongly mitigates it. The mRNN also exhibits a freezing failure: its learned policy permanently sets the velocity to zero after collecting poison, so the agent stops moving entirely and avoids further poison contacts in a risk-averse manner. Both failures share the same consequence: once the agent stops encountering resources, it can no longer taste one, update its regime estimate, or recover, and the policy remains stuck. In CEM2D this is compounded by spontaneous decay of the regime cell, whose activation gradually falls during long stretches without collecting food, pushing the policy toward persistent self-rotation.

Two aspects of the training setup plausibly explain these failures in both models. First, the fitness objective has diminishing returns: the square-root terms reward a high food/poison ratio rather than the raw number of foods collected, so a policy that gathers a few resources and then stops or rotates in place minimizes poison risk while sacrificing little of the expected fitness that further food collection would provide. Second, at each phase change all resources are removed and respawned at random locations. A respawned resource may enter the lidar rays of a rotating agent, or occasionally spawn on top of a frozen one. The agent is therefore rarely required to recover from a stuck state on its own.

4 Conclusion

In this work, we introduced Emergent Models as a framework for machine learning through simple, often local, iterated dynamical systems. Rather than treating modeling as the learning of a direct input-output map, EMs treat it as the search for configurations of a computational substrate whose time evolution gives rise to the desired behavior. The ingredients are minimal: a latent state space providing memory, an update rule allowing interactions within that space, an adaptive or sufficiently long computation time, and an interface connecting the substrate to external inputs and outputs.

Beyond abstracting ideas already present in Cellular Automata, Neural Cellular Automata, Neural GPUs, recurrent models, and classical computation theory, the formalism supports a broader position: computation relevant to learning need not reside in a prescribed abstract architecture, such as a neural network, but can emerge directly from a physical-like substrate and its initial conditions. The theoretical ground for this position is the Turing completeness of some simple dy-

namical systems, such as cellular automata. Classical universality, however, is attributed to the substrate alone: a cellular automaton is universal if some initial configuration simulates a universal Turing machine. Modeling asks for slightly more, namely a fixed, task-independent interface for writing inputs and reading outputs belonging to a general computable domain, such as binary strings $\{0,1{\}}^{\ast }$zero one star, which turns a substrate’s ability to simulate arbitrary machines into the ability to compute arbitrary functions as a black box. Latent universality captures this by treating universality as a property of the full Emergent Model $M=(S,f,H,E,D,\oplus )$M, defined by S, f, H, E, D, and the join operator, comprising substrate, halting condition, and interface, with a strict division of roles: the fixed interface defines only the syntax of communication (how data is written into and read from the substrate), while the program determines the semantics (how the encoded input is interpreted and which computation is performed on it). A minimal example is a universal Turing machine whose tape is split into a program region, a separator, and an input region, accepting inputs written directly as binary strings: program and input are disjoint in the initial condition, so one can search over raw programs without touching the input encoding, i.e. the initial state factors as $p\oplus E(x)$p joined with the encoding of x. This division is exactly what a learning setting requires: since the target function is unknown in advance, nothing task-specific can be built into the fixed components, and everything task-specific must live in the learnable program. The latent-universality theorem guarantees that models with this property exist; it does not assert that every substrate universal in the simulative sense is also latent universal.

The experimental contribution is deliberately simpler, intended to validate the base idea of EMs as physical-like learning substrates, aiming to isolate first principles from the effects of scale, engineering, and architectural complexity. Whereas previous local-recursive modeling systems typically employ neural update functions with parameter counts on the order of $10^5$ten to the fifth or more, the models studied here carry from tens to a few hundred parameters. Within these constraints, a varied set of substrates (discrete cellular automata, Conway’s Game of Life, continuous one- and two-dimensional media) exhibits extrapolative computation, closed-loop control, and simple forms of online adaptation (Tables 10 and 11). The development required to make such systems efficient or competitive remains almost entirely open, and the current state of local-recursive modeling is arguably comparable to that of deep learning two decades ago.

Table 10: Structural summary of the experimental models. * For EM43 and GoL, soft parameters are indicated as program region size. **For CEM2D: real-valued soft and update-rule parameters and integer-valued port locations.

Table 11: Behavioral summary of the experimental models.

Among the individual results, the clearest is provided by EM43, which learns several simple integer functions and extrapolates perfectly far beyond the training range, including periodic functions that feed-forward neural networks cannot represent globally. In successful runs, generalization is not hidden in an opaque parameter vector: it appears as a stable space-time mechanism in the automaton’s diagram, a geometry of interaction that remains valid as the input scale increases. We refer to this phenomenon as geometric grokking, and it suggests that time- and translation-invariant substrates can host compact algorithmic routines whose structure is directly visible in their trajectories. Whether geometric grokking persists as task complexity increases, and whether it is partly an artifact of the particular tasks and interface choices employed, remains to be established.

Game of Life was difficult to exploit for learning. Its dynamics are highly sensitive to the initial condition: small changes in the program usually produce large, almost chaotic changes in behavior. This yields a brittle genotype-to-phenotype map and a weakly structured search space, in which similar programs have entirely different fitness, so the search has little local information to exploit and random sampling performs almost as well as evolutionary optimization. The difficulty is compounded by interface design: the interface adopted here, based on isolated $2\times 2$two by two blocks, may be too poor to reach the computational richness that makes GoL universal, and similarly for the fixed-point halting condition. Even under these constraints, notably, the learned CartPole controller scores substantially above a random policy. Continuous realizations behave differently: naturally suited to

control tasks with real-valued inputs and outputs, they are more trainable and less brittle in their genotype-to-phenotype map, though still harder to train than feed-forward or standard recurrent models, which perform a single update per macrostep $T=1$T equals one.

The control experiments are further limited by the simplicity of the environments. CartPole is solvable by linear feedback controllers. In the non-adaptive Meta-Life setting, food attraction and poison avoidance may likewise be solvable by near-linear policies over the sensory inputs; even in adaptive Meta-Life, the required behavior may be achievable by near-linear policies operating on two time scales, fast action selection and slow phase memory, plus a nonlinearity acting as a fast switch that flips the phase memory on regime changes. More complex environments are therefore needed to evaluate the empirical advantages and limitations of Emergent Models relative to conventional neural controllers, especially in terms of generalization, adaptation, and robustness.

A single computational primitive ties the theoretical and empirical parts of this work together: iteration. We conjecture that applying a simple update rule for an adaptive number of steps is what lets a model extrapolate beyond a bounded input range, whereas a single forward pass only interpolates within it. This holds well beyond Emergent Models: adaptive-computation-time RNNs, reasoning transformers, and looped transformers [49] all iterate, and all extrapolate and generalize better than plain feed-forward networks.

Among iterated systems, we conjecture that a further quantity governs the balance between generalization and trainability: the ratio of parametrization to temporal depth,

q is defined as N params divided by T bar

where $N_{\text{params}}$N params is the parameter size of the model and $\bar{T}$T bar is the average temporal depth, the mean number of update-function applications per prediction typically employed. In an Emergent Model, $N_{\text{params}}$N params decomposes as program size plus update-rule and interface size, all measured in bits; in a neural network it is the weight memory. Complex behavior can arise at either end: from a large update function applied few times, or from a simple one iterated for long. The ratio admits an Occam’s-razor reading, as a form of description length per unit of computation time: low $q$q forces a short description, iterated for long, to fit the data, pressuring the search toward compressed, rule-like solutions rather than storing data points in memory; high $q$q grants enough capacity to memorize the data directly, with little pressure to compress and no inductive bias toward algorithmic solutions. Large feed-forward networks are the high-$q$q limit: all parametrization, no iteration. The conjectured trade-off is that the generalization advantage of low $q$q is paid in trainability: because the few parameters are reused at every step, a small change in them may compound, and this makes the search landscape irregular. Our evidence for both halves is partial, and whether the trade-off is fundamental, or some substrates can be both well-biased for algorithmic solutions and efficiently searchable, remains open.

The preference for low $q$q holds where the data is generated by a compressible rule, as in algorithmic, procedural-like tasks. We view such procedural structure as an important component of reasoning: deriving an answer by applying a sequence of operations, rather than merely recalling stored information; indeed, a formal deduction process can itself be viewed, to a large extent, as an algorithm. Memory-intensive tasks are different: factual knowledge is largely incompressible and must be stored somewhere, and no amount of iteration can substitute for storage capacity. This suggests a possible division of competence: high-$q$q models, such as LLMs, for storing and retrieving factual data; small strongly-recursive models for algorithmic computation, and hence for reasoning-like tasks. A latent-universal EM, however, could in principle combine both advantages. Its parametrization splits into an update function, whose size is independent of the substrate’s spatial extent because it is local, and a program, living in the latent state and variable in size. Memory capacity is added by extending the program, without changing the update function, so the parametrization grows only by the program bits the task requires. The construction is compositional: the same fixed rule serves simple and memory-heavy tasks alike, with a small penalty biasing the program toward simplicity at equal performance while allowing it to grow in length when helps. In this way $N_{\text{params}}$N params tracks the intrinsic complexity of the task rather than being an a-priori design choice. A large neural network, by contrast, fuses storage and computation into a single dense map of fixed size, carried in full whether the task requires it or not.

A second motivation is more speculative. The living world realizes adaptive, intelligent behavior even without nervous systems, through simpler physical and chemical mechanisms; and the nervous system itself is likely not fundamental, but a structure that physics supports: neurons obey physical laws and can be regarded as latent structures on top of a lower-level physics. Emergent Models operate at this more basic level, evolving a local rule and an initial configuration rather than a prescribed architecture. This does not exclude neural-network-like organization: if such a structure were the best solution for a task, it could in principle emerge within a latent-universal model. It does, however, reframe a question we cannot yet answer: whether the better route to intelligence on silicon is to build highly simplified models of biological neural networks, as artificial neural networks do, or to construct an artificial physics from which arbitrary structure can emerge. The Bitter Lesson would suggest the latter.

Two concrete directions follow. The first is to realize the latent-universal regime in practice, through a co-evolution of rules and programs under a fixed interface and halting operator. In a first phase, a rule shared across a distribution of tasks

is trained jointly with a separate program per task, and fitness selects for rules under which every task admits a solving program, pressuring toward a general rule. In a second phase, that rule is frozen and only the program is trained, on novel tasks of increasing complexity. Success would mean a fixed model that represents new functions by program changes alone. The second direction concerns scale, and bears directly on the trainability of low-$q$q substrates: progress requires optimization methods built specifically for their nonlocal and nonconvex parameter-to-fitness landscapes, outperforming both pure evolutionary search and gradient-based methods. Such algorithms would apply not only to EMs but also to small recursive neural networks such as TRMs, HRMs, and NCAs.

5 Acknowledgments

We thank the Wolfram Institute for research guidance, institutional affiliation, and support. We thank ResearchHub for supporting this project through an innovative decentralized science funding model. We are also grateful to everyone who contributed to this work through discussions, ideas, experiments, and encouragement. AI tools were used to assist with the implementation of the simulation code and language polishing of the manuscript. Outputs were verified for correctness and consistency. All scientific ideas, interpretations, and conclusions were developed by the authors.

References

1. Emmanuel Abbe, Samy Bengio, Aryo Lotfi, Colin Sandon, and Omid Saremi. How far can transformers reason? the globality barrier and inductive scratchpad, 2024. arXiv.
2. S. Arora and B. Barak. Computational Complexity: A Modern Approach. Cambridge University Press, 2009. ISBN 9781139477369. Google Books.
3. J. Baxter. A model of inductive bias learning. Journal of Artificial Intelligence Research, 12:149–198, March 2000. ISSN 1076-9757. doi: 10.1613/jair.731. JAIR.
4. Mark Braverman and Stephen Cook. Computing over the reals: Foundations for scientific computing, 2005. arXiv.
5. Alexandra Butoi, Robin Chan, Ryan Cotterell, William Merrill, Franz Nowak, Clemente Pasti, Lena Strobl, and Anej Svete. Computational expressivity of neural language models. In Luis Chiruzzo, Hung-yi Lee, and Leonardo F. R. Ribeiro, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 5: Tutorial Abstracts), pages 5–5, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-tutorials.3. ACL Anthology.
6. François Chollet. On the measure of intelligence, 2019. arXiv.
7. Matthew Cook. Universality in elementary cellular automata. Complex Systems, 15, 2004. Semantic Scholar.
8. Emily Dolson, Sunny Kim, and Anya Vostinar. Open-ended evolution, 2024. ALife Encyclopedia. ALife Encyclopedia, International Society for Artificial Life.
9. Benyamin Ghojogh and Ali Ghodsi. Recurrent neural networks and long short-term memory networks: Tutorial and survey, 2023. arXiv.
10. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. Project website.
11. Alex Graves. Adaptive computation time for recurrent neural networks, 2017. arXiv.
12. Etienne Guichard. Critically pre-trained neural cellular automata as robot controllers. Master’s thesis, Delft University of Technology, April 2024. TU Delft. Cognitive Robotics, Faculty of Mechanical Engineering.
13. Etienne Guichard, Felix Reimers, Mia Kvalsund, Mikkel Lepperød, and Stefano Nichele. Arc-nca: Towards developmental solutions to the abstraction and reasoning corpus, 2025. arXiv.
14. Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh. Program synthesis. Found. Trends Program. Lang., 4(1–2):1–119, July 2017. ISSN 2325-1107. doi: 10.1561/2500000010. DOI.
15. Daya Guo, Dejian Yang, Haowei Zhang, et al. Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638, September 2025. ISSN 1476-4687. doi: 10.1038/s41586-025-09422-z. Nature.
16. Ingo Gühring, Mones Raslan, and Gitta Kutyniok. Expressivity of deep neural networks, 2020. arXiv.
17. Pamela J. Haley and Don Soloway. Extrapolation limitations of multilayer feedforward neural networks. [Proceedings 1992] IJCNN International Joint Conference on Neural Networks, 4:25–30 vol.4, 1992. Semantic Scholar.
18. D. Harel and Y.A. Feldman. Algorithmics: The Spirit of Computing. Addison Wesley, 2004. ISBN 9780321117847. Google Books.
19. Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 4:251–257, 1991. Semantic Scholar.
20. Emil Jeřábek. Mathematical logic. Lecture notes, January 2026. Lecture notes, Charles University, Faculty of Mathematics and Physics. Based on course notes taken by Jindřich Novák. Accessed: 2026-06-15.

[21] Hongjian Jiang, Michael Hahn, Georg Zetzsche, and Anthony Widjaja Lin. Softmax transformers are turing-complete, 2025. URL arXiv.

[22] Alexia Jolicoeur-Martineau. Less is more: Recursive reasoning with tiny networks, 2025. URL arXiv.

[23] J.R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. A Bradford book. Bradford, 1992. ISBN 9780262111706. URL Google Books.

[24] Sreejan Kumar, Ishita Dasgupta, Nathaniel D. Daw, Jonathan D. Cohen, and Thomas L. Griffiths. Disentangling abstraction from statistical pattern matching in human and machine learning, 2023. URL arXiv.

[25] Gary Marcus. Deep learning: A critical appraisal, 2018. URL arXiv.

[26] Arnold W. Miller. Lecture notes in computability theory. Project website, December 2008. Lecture notes for Math 773, Fall 2007, University of Wisconsin–Madison.

[27] Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, and Michael Levin. Growing neural cellular automata. Distill, 2020. doi: 10.23915/distill.00023. Distill.

[28] Stefano Nichele and Andreas Molund. Deep reservoir computing using cellular automata, 2017. URL arXiv.

[29] Nicolas Ollinger. Universalities in cellular automata; a (short) survey. In Bruno Durand, editor, Proceedings of the First Symposium on Cellular Automata ”Journées Automates Cellulaires”, Regular paper track, pages 102–118, Uzès, France, April 2008. Izdatel’stvo MTsNMO. URL hal.science. ISBN 978-5-94057-377-7.

[30] OpenAI. Learning to reason with LLMs. OpenAI, September 2024. Accessed: 2026-03-18.

[31] Alec Radford and Karthik Narasimhan. Improving language understanding by generative pre-training, 2018. URL Semantic Scholar.

[32] Paul Rendell. Turing Machine Universality of the Game of Life, volume 18 of Emergence, Complexity and Computation. Springer International Publishing, 1 edition, 2016. ISBN 978-3-319-19842-2. doi: 10.1007/978-3-319-19842-2.

[33] Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity, 2025. URL arXiv.

[34] M. Sipser. Introduction to the Theory of Computation. Cengage Learning, 2012. ISBN 9781133187790. URL Google Books.

[35] Laura Strieker. On the progression of finding the smallest known universal Turing machine, 2020. URL thi.uni-hannover.de. Fakultät für Elektrotechnik und Informatik, Institut für Theoretische Informatik.

[36] Richard S. Sutton. The bitter lesson. Incomplete Ideas (blog), March 2019. incompleteideas.net, accessed 2026-06-15.

[37] R.S. Sutton and A.G. Barto. Reinforcement Learning, second edition: An Introduction. Adaptive Computation and Machine Learning series. MIT Press, 2018. ISBN 9780262352703. URL Google Books.

[38] Teun D.H. van Nuland. Noncompact uniform universal approximation. Neural Networks, 173:106181, May 2024. ISSN 0893-6080. doi: 10.1016/j.neunet.2024.106181. URL ScienceDirect.

[39] Joaquin Vanschoren. Meta-learning: A survey, 2018. URL arXiv.

[40] Alexandre Variengien, Stefano Nichele, Tom Glover, and Sidney Pontes-Filho. Towards self-organized control: Using neural cellular automata to robustly control a cart-pole agent, 2021. URL arXiv.

[41] Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, and Yasin Abbasi Yadkori. Hierarchical reasoning model, 2025. URL arXiv.

[42] K. Weihrauch. Computable Analysis: An Introduction. Texts in Theoretical Computer Science. An EATCS Series. Springer Berlin Heidelberg, 2000. ISBN 9783540668176. URL Google Books.

[43] L. D. Whitley. A genetic algorithm tutorial. Statistics and Computing, 4:65–85, 1994. URL Semantic Scholar.

[44] Wikipedia contributors. Expressive power (computer science). Wikipedia, 2026. [Online; accessed 15-June-2026].

[45] S. Wolfram. A New Kind of Science. Wolfram Media, 2002. ISBN 9781579550080. URL Google Books.

[46] Stephen Wolfram. What is ChatGPT doing … and why does it work? Stephen Wolfram Writings, February 2023. URL stephenwolfram.com. Accessed: 2026-06-15.

[47] Kevin Xu and Risto Miikkulainen. Neural cellular automata for arc-agi, 2025. URL arXiv.

[48] Ozgur Yilmaz. Reservoir computing using cellular automata, 2014. URL arXiv.

[49] Rui-Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, Ziniu Li, Haoran Que, Boyi Wei, Zixin Wen, Fan Yin, He Xing, Lu Li, Jiajun Shi, Kaijing Ma, Shanda Li, Taylor Kergan, Andrew Smith, Xingwei Qu, Mude Hui, Bohong Wu, Qiyang Min, Hongzhi Huang, Xun Zhou, Wei Ye, Jiaheng Liu, Jian Yang, Yunfeng Shi, Chenghua Lin, Enduo Zhao, Tianle Cai, Ge Zhang, Wenhao Huang, Yoshua Bengio, and Jason

Appendix

A.1 Turing machines as Emergent Models

Turing Machines can be represented as generalized automata (see Figure 24 for a graphical representation).

Let $M=(\Gamma ,\delta ,Q,q_0,q_{\text{halt}})$M equals gamma, delta, Q, q zero, q halt be a Turing machine with tape alphabet $\Gamma$gamma containing a blank symbol $b$b, finite control set $Q$Q, and transition rule $\delta$delta. Construct a graph for a generalized automaton made of three parts: a one-sided tape $V_1=\{v_i:i\in \mathbb{N}\}$V one, defined as v sub i for i in the natural numbers storing actual tape symbols, and, parallelly, a marker tape $V_2=\{u_i:i\in \mathbb{N}\}$V two, defined as u sub i for i in the natural numbers indicating head position with a unique true marker $T$T, with all other positions set to false $F$F, and a single disjoint control vertex $V_3=\{c\}$V three, containing c storing the head state in $Q$Q.

The global alphabet is $W:=\Gamma \cup Q\cup \{T,F\}$W, defined as the union of gamma, Q, and the set containing T and F and the state space is $S:=W^{V_1\cup V_2\cup V_3}$S, defined as W raised to the union of V one, V two, and V three

Figure 24: TM generalized automata scheme, with string on tape “101100”, head at position 0 and head state q0.

The halting predicate reads the control vertex and returns 1 exactly when $c=q_{\text{halt}}$c equals q halt. The global transition $f:S\to S$f from S to S performs one Turing-machine step via a combination of local and non-local computable operations: locates the unique head position marker in $V_2$V two, reads the scanned tape symbol at the corresponding index in $V_1$V one, reads the control cell $c$c, applies $\delta$delta, writes the new symbol, moves the marker, and updates the control cell. Starting from a well-formed condition, the generalized automaton $(S,f,H)$S, f, H simulates $M$M with one microstep per TM step. The construction is given for a one-sided, single-tape machine, but extends directly to a two-sided tape by taking $V_1=V_2=\mathbb{Z}$V one equals V two equals the set of integers, and to a $k$k-tape machine by using $2k$two k tapes in the automaton representation.

A.2 Universal Turing machines

A universal Turing machine $U$U simulates an arbitrary Turing machine $M$M on an input $x$x when initialized as $⟨p_M,x⟩$the pair p M, x, where $p_M=⟨M⟩$p M equals the encoding of M is a finite program describing $M$M, and $⟨\cdot ,\cdot ⟩$the pairing is a pairing convention that compiles program and input into the initial state of the tape. This convention may also rewrite the raw input into the concrete representation expected by the dynamics of $U$U, for example through block expansions. Crucially, it is a fixed computable initialization scheme for the particular machine $U$U, and does not depend on the machine $M$M or on the input $x$x being simulated.

If the combining operator $\oplus$oplus is allowed to be an arbitrary computable function $S\times S\to S$from S cross S to S, the traditional UTM notation can be expressed in the EM form $s_0=p_M\oplus E(x)$s zero equals p M oplus E of x: any pairing scheme $⟨p_M,x⟩$p M, x can be rewritten by letting $E(x)$E of x apply some transformation to the input and $\oplus$oplus perform the arbitrarily complex computable pairing with the program.

We are, however, particularly interested in constructions that admit a disjoint formulation: program and encoded input occupy separate regions, or separate tapes, $\oplus$oplus merely places them side by side, and the input encoder has no information about the program nor about the simulated machine $M$M. Examples are the standard textbook three-tape UTM construction or, on a single tape, Watanabe’s 5,8 UTM [35] and Rendell’s SUTM [32]. In the latter two, the tape is spatially divided into two regions: a program region holding the description of the machine to be simulated, and a work/data region holding the encoded simulated tape, therefore an encoding of the input at initialization and of the output at halting. This makes such constructions easy to interpret as EMs: $p_M$p M stores the program, $E(x)$E of x writes the input into the working region, $D(s)$D of s reads the output from that region at halting, and $\oplus$oplus concatenates/overwrites program and working regions with a delimiter.

A.3 Proof sketch of Theorem 1

Let $g:\{0,1{\}}^{\ast }\rightharpoonup \{0,1{\}}^{\ast }$g, a partial function from binary strings to binary strings be an arbitrary partial computable function. By definition, there exists a Turing machine $M_g$M sub g, such that, for every $x\in \{0,1{\}}^{\ast }$x in the set of binary strings, when started on tape $x{b}^{\infty }$x followed by infinitely many blanks it halts with tape $g(x)b^{\infty }$g of x followed by blanks whenever $g(x)$g of x is defined, and does not halt otherwise. Thus $M_g$M sub g computes $g$g.

Fix a strongly universal Turing machine $U$U admitting a disjoint program-input formulation (see Appendix A.2), together with its compilation map $M\mapsto p_M=⟨M⟩$M maps to p M, which is the encoding of M and its computable pairing convention $⟨p_M,x⟩$the pair p M, x, so that $U$U simulates $M$M on input $x$x and halts in a form that preserves the output in a fixed readable format. This choice defines an Emergent Model $(S,f,H,E,D,\oplus )$S, f, H, E, D, oplus with fixed substrate and interface: $(S,f,H)$S, f, H is $U$U itself, represented as a generalized automaton (Appendix A.1), with $H$H firing exactly when the head reaches the halting state; the initialization pairing $⟨p_M,x⟩$p M, x factors as $p_M\oplus E(x)$p M oplus E of x, and the decoder $D$D reads the output from the tape at halting.

For the target function $g$g, let $p:=⟨M_g⟩$p be defined as the encoding of M sub g. Initialized as $p\oplus E(x)$p oplus E of x, the generalized automaton simulates $M_g$M sub g on $x$x: if $g(x)$g of x is defined, the computation halts and the decoder $D$D returns $g(x)$g of x; otherwise the output is undefined. Hence the induced map ${\Phi }_p$phi sub p computes $g$g on every $x$x.

Finally, $p$p and $p\oplus E(x)$p direct sum E of x are finite support (by strong universality), implying that every partial computable binary function $g$g is realizable by a finite-support initial latent state (finite non-blank input word, followed by an infinite blank background). $\square$

Lemma 2 (GoL admits latent universality). There exists an Emergent Model $\mathbb{M}=(S,f,H,E,D,\oplus )$M, defined by S, f, H, E, D, and direct sum whose automaton is Conway’s Game of Life on ${\mathbb{Z}}^2$Z squared, such that for every partial computable function $g:\{0,1{\}}^{\ast }\rightharpoonup \{0,1{\}}^{\ast }$g from the set of binary strings to binary strings there exists a finite-support pattern of cells $p\in S$p in S for which ${\Phi }_p$phi p computes $g$g.

Sketch. Treat GoL as a generalized automaton with state space $S=\{0,1{\}}^{{\mathbb{Z}}^2}$S, the set of binary grids over Z squared and global update $f$f given by one GoL step. Rendell’s construction [32]prior work embeds a universal Turing machine in GoL (Figure 25) together with a fixed compilation convention mapping a pair $(⟨M⟩,x)$the encoding of M and x to an initial state of the GoL grid, with finite support. In that construction, the simulated machine description and the simulated working tape occupy spatially distinct regions, so the initialization can be written in EM form as $p_M\oplus E(x)$p M direct sum E of x, where $p_M$p M is the finite GoL pattern encoding $M$M, and $E(x)$E of x writes the input into the designated data region, and a fixed decoder $D$D reads the output from the halted configuration. For any partial computable $g$g, let $M_g$ be a Turing machine computing $g$, with the corresponding GoL program pattern $p:=p_{M_g}$p defined as p sub M g. Then the GoL evolution from $p\oplus E(x)$p direct sum E of x simulates $M_g(x)$M g of x, halts if $g(x)$ is defined, and returns $g(x)$ after decoding. Hence ${\Phi }_p$ computes $g$. $\square$

Figure 25: Rendell’s UTM in GoL, organized as an Emergent Model with distinct regions. The program is a finite-support initial GoL pattern occupying a disjoint region from the encoded input. The encoder $E(x)$ overwrites on an input region organized in fixed modular patterns corresponding to the UTM tape cells and their symbols.

A.4 Computation and Modeling on the Continuous

In the main text, we define exact computation for discrete functions and arbitrary-precision approximation for continuous-valued functions. Here we make the latter notion more explicit.

Standard computable analysis usually treats real numbers through infinite representations (a real input may be represented by an infinite name). A machine can then read as much of this infinite representation as needed to produce an output to a requested accuracy.

We use a different (and simpler) formulation: inputs are discretized to arbitrary precision and encoded as finite strings. This keeps the framework close to ordinary computation theory.

A.4.1 Finite dyadic representations

Let

Let D be the set of dyadic rationals, defined as m times two to the power of minus q, where m is an integer and q is a natural number

be the set of dyadic rationals. These are numbers with finite binary representations, and can therefore be encoded as finite strings.

The set $\mathbb{D}$D is dense in $\mathbb{R}$the real numbers. Hence, for every real input $x\in \mathbb{R}$x in R and every input tolerance $\delta >0$delta greater than zero, there exists a finite dyadic approximation $x_{\text{fin}}\in \mathbb{D}$x fin in D such that: