AUSTRALIAN NATIONAL UNIVERSITY
DOCTORAL THESIS IN COMPUTER SCIENCE

michaeltimothybennett.com

This dissertation is an account of research that began March 2019. It is comprised of 13 chapters, based on 13 of my papers, written under 13 advisors and completed on the 13th of May, 2025.

The work presented in this thesis is that of the candidate alone, except where indicated by due literature reference and acknowledgements in the text. It has not been submitted in whole or in part for any other degree at this or any other university.

Michael Timothy Bennett, May 2025

ACKNOWLEDGEMENTS

This work was mostly funded by my personal savings account. RIP. This work was partly funded by a Fundaçao para a Ciência e a Tecnologia (FCT) grant under the reference PTDC/FER-FIL/4802/2020, JST (JPMJMS2033), and an Australian Government Research Training Program (RTP) Scholarship.

I’d like to thank the 13 people who have advised me during my various attempts at research at the ANU, both during my masters and during my PhD¹: Sean Welsh, Anna Ciaunica, Yoshihiro Maruyama, Colin Klein, Sylvie Thiebaux, Marcus Hutter, Marcus Hegland, Michael Barnsley, Elizabeth Williams, Ehsan Nabavi, Uwe R. Zimmer, Badri Vellambi and Samuel Allen Alexander. I’d particularly like to thank Yoshi, Sean and Anna. I would not be here without you. To Yoshi, who became my primary supervisor two years into my PhD: You saw something in me and my half-complete project, which I can only imagine must have sounded mad. If you hadn’t co-authored that first journal article with me, my academic career might have been over before it began. To Sean and Anna who have worked so tirelessly to get me over the finish line, without your support I might never have finished! My thesis has benefited immensely from your inputs, and your support. You have made a huge difference! On top of that I must note that Sean has done all this in his spare time as an independent researcher. Sean you have been incredibly generous with your time and feedback, and this thesis would be much less polished without your oversight. I will never forget it!

I also want to thank the AGI Society and its members. The 2023 and 2024 AGI conferences were the highlights of my PhD. The encouragement, awards and sense of belonging I felt there quite profoundly changed my life! I’d also like to acknowledge the many others who have helped me, but there are too many. To Ricard Solé, Lenore and Manuel Blum, Karl Friston, Peter Watts, Noel Hinton, Vincent Abbott, Lucas Scott, Simon Strauss, Elija Perrier, Paul McMahon, Tim Wicks, Seth Lazar and the many others who were so generous with either encouragement or feedback: Thank you!

Finally to Ashitha Ganapathy: You have been with me through the highs and lows of all of this. My maniacal obsessions with new topics that extended the length of this thesis by years. My moments of despair, forgetfulness and stubbornness. We even wrote our first paper together. You have listened to every part of this thesis more times than I can count. I don’t know what I would have done without you. More than anyone, credit for getting me this far goes to you. From the bottom of my heart, thank you. This thesis is your achievement as well.

SECTIONS ALREADY PUBLISHED

To validate my progress I have continuously published throughout my PhD. Key published include optimal learning² (chapters 6 and 7), and my arguments regarding meaning³ (chapter 10), causality⁴ (chapters 9 and 12) which links consciousness to intelligence, the fermi paradox⁵ (chapter 11), complexity⁶ (chapter 7), the artificial scientist⁷ (chapter 13) and abstraction layers⁸ (chapters 4, 5, 6 and 8). I published The Mirror Symbol hypothesis, which informs many of the results in meaning, in an IEEE journal⁹ (chapter 10). My argument regarding the hard problem¹⁰ (chapters 12 and 13) and my more recent survey of AGI¹¹ (chapter 3) are currently under review, but the former was accepted to and presented at both ASSC27 and MoC5. My paper on systems as a stack is of central importance to this thesis, and is forthcoming¹² (chapters 4, 5, 8, 10 and 11). I co-authored and published a precursor to that paper at an IEEE cybernetics conference¹³. My paper on Computable Artificial General Intelligence¹⁴ was important but it has been under review with IEEE Transactions on Emerging Topics in Computational Intelligence for 3 years. Fortunately, I was able to publish the key result of that paper at AGI-23 and 24 instead. I’ve written 21 papers in total. I expect 19 of those will have passed peer review by the time this thesis is out. My other papers are also cited but are not particularly important for this thesis. So this thesis is comprised of 13 chapters, based mostly on 13 of my papers, written under 13 advisors and completed on the 13th of May, 2025. Though many of these results were published in stand alone papers, they were all written in service of the vision I present here.

DISCLAIMER

This thesis is a draft that is under review. I have already published most of the results in peer reviewed books and journals, but the thesis itself may still change based on reviewer feedback. Take it with a grain of salt.

Please send questions, feedback, hate and fan mail to michael.bennett@anu.edu.au.

[blank page]

Contents

• I. FOREWORD AND CHAPTER SUMMARIES

• II. SOME PHILOSOPHY

• III. WHAT THE F*CK IS AGI?

• IV. WOW, EVERYTHING IS COMPUTER

• V. TURTLES ALL THE WAY DOWN

• VI. MASTER, WHAT IS MY PURPOSE?

• VII. WEAK

• VIII. STACKISM

• IX. LETS GET PSYCHOPHYSICAL

• X. LANGUAGE CANCER

• XI. WHY IS ANYTHING ALIVE?

• XII. Why Is Anything Conscious?

• XIII. How to Build Conscious Machines

• Appendix A: Technical Appendix

• Bibliography

I. FOREWORD AND CHAPTER SUMMARIES

Humans overlook subtractive solutions. We refuse to reduce. Engineers cobble together bits of code into webs so monstrous the errors cannot be found. On a more human scale governments add laws with reckless abandon, but how often do you see them repeal the old? This bias for expansion over contraction is well documented across the spectrum of human endeavour¹⁵. For scientific and philosophical pursuits, I suspect our tendency to overlook subtractive solutions has made many problems more difficult than they need to be. When we encounter data we cannot explain within the confines of existing theory, an additive solution would be to construct more and more convoluted theories to reconcile the old theory with the new data. However, we do not always need to reconcile the new with the old. We just need to explain what is, and sometimes that means throwing out preconceptions. For example Milton Friedman proposed simple monetary models instead of complex cyclical models, informing monetary policy that allows us to avoid repeating the great depression. I am interested in broad reaching questions which are similarly burdened by precedent. How can we build a conscious machine? Why is anything conscious? Alive? What is life? Is complexity an illusion? Are biological systems more intelligent than artificial intelligence? Why? In search of answers I have published a number of papers¹⁶ in peer reviewed books and journals. I wrote these papers not as disconnected works but as interconnected parts of a larger vision, culminating in this thesis.

Overall this thesis is about how to build a conscious machine. I don’t actually have a conscious machine, because that seemed like overkill for a thesis. What I do have is an explanation of what consciousness is and how it came about. There remain one or two unanswered questions. I also have some proofs and experimental results showing how to ‘adapt’ as efficiently as possible, which is useful for building artificial superintelligence.

There are a few other results too. I’ve given explanations of the origins of life, language, the Fermi paradox, causality, an alternative to Ockham’s Razor, the optimal way to structure control within a company or other organisation, and instructions on how to give a computer cancer. They’ve mostly been published and, strange as it is, they all tie in to a coherent vision. They weren’t conceived in isolation, but as parts of a whole.

What follows now is a summary of the whole thesis. The purpose of this summary is to give you a narrative overview of what I am doing and why, before I get into the weeds. As such, it uses terms like causal-identity and task without formally defining them. These terms are formally defined later in the thesis main body, but here and now they are to be read intuitively. Brevity is the only virtue to which this chapter aspires.

II-III. LITERATURE REVIEWS

Chapters II and III are literature reviews.

Chapter II surveys philosophy and neuroscience. What is a conscious entity? To build one, I must know. Philosophy, psychology and neuroscience all provide insight. However the matter is far from settled. I must take concrete positions on disputed issues within these fields before I can say how to build a conscious machine. Hence I survey some relevant concepts and disputes, combining the introductory sections of my publications on enactive and ethical AI¹⁷, communication¹⁸ and consciousness¹⁹. Topics covered include the mind body problem, functionalism, theories of consciousness, self organisation, the free energy principle, enactivism, epistemology, semiotics, structuralism, post-structuralism and theories of meaning.

Chapter III deals with AGI, which is the foundation of this thesis. It is a survey from on of my earliest publications²⁰, updated to reflect more recent developments²¹. I begin by discussing several definitions of intelligence and AGI. I end up framing intelligence as adaptation²², and AGI as that which adapts generally. For the purposes of benchmarking, I define AGI is an artificial scientist. I take inspiration from Sutton’s ‘Bitter Lesson’²³, which is that throwing compute at a wall consistently beats human ingenuity. With sufficient resources any general approach to optimisation can eventually attain an arbitrary level of skill. Two have consistently scaled: search and approximation. I discuss strengths, weaknesses and examples of each. Hybrids of search and approximation are best. I discuss some hybrids including Hyperon²⁴, AERA²⁵ and NARS²⁶. I introduce the concept of meta-approaches that can be applied to search, approximation or hybrids. One example of a meta-approach is the maximisation of scale and available resources (scale-maxing), in accord with Sutton’s bitter lesson. Another is simplicity maximisation (simp-maxing) based on Ockham’s Razor. I evaluate the strengths and weaknesses of these approaches. They allow us to speculate about how a superintelligence might behave. However, simplicity is a matter of interpretation. It is subjective, and so these claims are also subjective. In this thesis I propose an alternative meta-approach that is optimal. Overall the meta-approaches I discuss in this thesis are to maximise the simplicity of form (simp-maxing), to maximise the scale (scale-maxing) and to maximise the weakness of constraints on function (w-maxing). This latter one is my proposal.

IV. WOW, EVERYTHING IS COMPUTER

This chapter explains why complexity is subjective and what can be done to formalise objective performance. The key result is a concept I call computational dualism, described in my publication of the same name²⁷. I begin by pointing out that the very idea of a software intelligence is broken. The behaviour of software is determined by the hardware on which it runs. It interprets the environment for the software, and the software for the environment. I use the term ‘computational dualism’ to describe theories that treat ‘minds’ as disembodied entities that interact with the environment through an interpreter. I conclude that to make claims regarding the objective behaviour of an intelligence, we must avoid computational dualism.

I propose a solution, which I published earlier in several of my papers²⁸. To avoid computational dualism, it might be tempting to think we just need to focus on the hardware. However this would repeat the same mistake. Computer systems are organised into “abstraction layers”. Higher abstraction layers run in lower abstraction layers. For example Python is interpreted by a C program. I argue the abstraction layers do not end at hardware, and that hardware is interpreted by physical laws just as software is interpreted by hardware.

Taken to its logical conclusion, everything is a stack of abstraction layers²⁹. Software is a state of hardware. A human is a state of organs which are states of cells. If the mind is $f_3$f three, the body or hardware³⁰ is $f_2$f two and the local environment $f_1$f one, then The Stack is $f_1(f_2(f_3))$f one of f two of f three. Perhaps The Stack has a lowest layer like an ‘underlying physics’ $f_0$f zero, meaning The Stack is $f_0(f_1(f_2(f_3)))$³¹. However we have no way of knowing. The Stack might go on forever. To make claims that hold regardless, I conclude that I need a formalism that holds in every possible world. I propose one. It is a formal definition of environment, which is the foundation of what I call Stack Theory³². It is what is common to all environments and ‘underlying physics’. It equates time with difference, and difference with a state of the environment. This lets me formalise declarative programs in terms of difference, to integrate pancomputationalism³³. I then argue everything must fall within the scope of what this formalism can describe. Yes, my formalism is still an abstraction. However, some claims are so weak they are true of everything.

V. TURTLES ALL THE WAY DOWN

Chapter V is about embodiment. Each body is an abstraction layer. When I do something with my body like raise my arm, I change the possibilities for what happens next. I impose a constraint on the world. In this sense, a body speaks a formal language³⁴. This embodied language is ontological, meaning a statement is rather than refers to something. Every physical thing is an abstraction layer that speaks a formal language, not just living bodies. A computer speaks a formal language of hardware states. The universe speaks a formal language of physics³⁵. This idea is once again from my publications on abstraction layers³⁶. I show how Stack Theory expresses an embodied formal language of declarative programs. Those programs are the vocabulary of the language. Using this, the body makes statements that have truth values. In an embodied formal language, something is physically ‘said’ by the environment. This is the language of physical laws. If I was omniscient the environment would have one state at a time, because time is difference³⁷. That state would determine what is true at the present time. The grammar of the language comes from the fact that states of the environment are in this sense mutually exclusive, and some programs in a vocabulary can never be expressed together. Everything that exists is a statement made in an environment’s embodied formal language, and which statements are true depends on the state. However from my subjective perspective within my environment, I cannot know what the physical state is. I am a statement, and I exist for as long as the environment expresses me. When a statement is made, it constrains the space of what else can happen. Each statement has an extension. Intuitively, my extension is like the ‘many worlds’ in which I exist.

Each statement implies another higher abstraction layer. The extension of a statement forms a vocabulary of the layer above. In this way, every statement the environment makes creates an abstraction layer. The outputs of the level below form the vocabulary of the level above. We go up a level of abstraction by looking at second order effects of a body we started with. An abstraction layer is like a smaller environment defined in the context of a larger environment. A ‘small world’ defined inside a ‘big world’³⁸. It has its own formal language that is equivalent to a subset of the things the bigger environment can say. Each statement the environment makes is a body, and each body has an extension and thus its own, more restricted embodied formal language in which further statements can be expressed. Layer upon nested layer of abstraction.

VI. MASTER, WHAT IS MY PURPOSE?

Chapter VI is about purpose. These results were published earlier in my papers on abstraction layers³⁹ and consciousness⁴⁰. The result is a formal definition of an embodied tasks, inference and stacks. In earlier chapters, I defined bodies or hardware as embodied formal languages that express statements. I can choose any statement the body can make and call it an input. The possible outputs are the extension of the input. So a body can be seen as a computational system that maps inputs to outputs. I can single out a subset of those possible outputs and call them correct. I call a set of inputs and outputs a task. This is a way of formalising a arbitrary notions of correctness, or what ought to be. According to Hume’s Guillotine, I cannot derive what ought to be from what is so I need a universal, cosmic ought from which to derive all others. I argue this comes from time. Change is foundational. Statements are destroyed as states change. A body is a statement that lasts for as long as the environment expresses it. Subjectively, we can interpret the process of creation and destruction as statements ‘moving’ relative to one another. Statements that persist are those that move away from circumstances in which they are destroyed. As the environment transitions from one state to another, this eliminates that which doesn’t seek to preserve its existence. This is like natural selection, but applied to every aspect of the environment. It creates an incentive I call the cosmic ought. What I argue here is that everything that the environment expresses is a statement of what ought to be, and the rest that which ought not.

Each statement implies a narrower abstraction layer than the one in which it was expressed, like a window or small world within a big world. As we go to higher in a stack, the ought gets more specific. For example, an environment could be an abstraction layer. Lifeforms would then be statements made in that abstraction layer, growing ever more specific with each additional layer of abstraction. A lifeform might be considered ‘fit’ if it continues to exist, so the set of all fit outputs for a fit organism could be its extension. However organisms are often unfit. Such ambiguously ‘fit’ organisms would be statements whose extension contains unfit as well as fit behaviour. Were it to engage in unfit behaviour it would still exist, just not in any condition to maintain homeostatic and reproductive goals. It is that distinction between ‘fit’ and not that a task formalises, by pointing out the set of outputs considered ‘correct’ and the inputs in which being correct actually matters. Hence I formalise goal directed behaviour in a stack of tasks⁴¹.

VII. WEAK

Chapter VIII is about intelligence. The key results were published in my papers on ‘weak’ hypotheses⁴². The result is a theory of optimal learning. I propose a meta-approach I call w-maxing, and an upper bound on intelligent behaviour based upon it. I formally prove and demonstrate experimentally that w-maxing is optimal, and simp-maxing is not⁴³.

If we take a Darwinian point of view then intelligence is long-term adaptation⁴⁴ that facilitates short-term adaptation⁴⁵. Without intelligence, an organism would need to have all knowledge hard coded from birth. With intelligence, it can adapt during its lifetime to survive in more circumstances than without intelligence⁴⁶. To represent this in my formalism I describe an organism by what it does, rather than is⁴⁷. What it does is a task. I explain how the task an organism does can be subdivided, by choosing subsets of inputs and outputs I call child tasks. Tasks thus exist in a generational hierarchy. An organism’s past is a child task of its future task. A task implies a set of policies that constrain an organism’s behaviour to the task definition. An organism embodies a ‘fit’ policy if it is constrained to fit behaviour. The process of learning is inferring a policy from the past that ensures future behaviour is fit. Intuitively, a policy is like a tool. A tool can complete more than one task. A hammer can be either a weapon or a paper weight. A weaker policy is a tool that completes more tasks. The weakest policies complete the largest number of tasks. I prove that, among all policies, the weakest policies are the most likely to generalise, maximising efficiency of adaptation. I call this the meta-approach of w-maxing⁴⁸${}^{,}$⁴⁹.

I go on to compare w-maxing and simp-maxing⁵⁰. I prove that we can w-max without simp-maxing. I support this claim with experiments comparing the two meta-approaches. I have them attempt to learn binary multiplication and addition. The w-maxing system outperforms the simp-maxing system by $110-500\%$one hundred ten to five hundred percent. The fewer examples one has to learn from, the greater the advantage in choosing weak policies. This all goes to show an optimal agent does not need to optimise for simpler models. I then prove that the objectively optimal agent is one that embodies the weakest policies for a task, providing an upper bound on embodied intelligence.

VIII. STACKISM

This chapter brings together my papers on complexity⁵¹ and abstraction⁵². I explain why simplicity of form has anything to do with function. In theory there could be no correlation, but in practice there is⁵³. My result is proofs explaining this correlation, and this explains why biological systems seem to adapt more efficiently than AI. I begin by proving that at the lowest level of abstraction, all policies are equally simple. There is no such thing as objective complexity. Then I argue bodies must use finite vocabularies, because of the Bekenstein bound^{54 55}. I show that there exist abstraction layers in which simple statements are weaker. Because vocabularies are finite, an abstraction layer in which weak statements take simple forms will be able to express more weak policies than an abstraction layer where weak policies do not take simple forms. This means complexity is an illusion perpetrated by abstraction layers. To maximise adaptability given finite resources, it is necessary⁵⁶ for abstraction layers to express weaker constraints using simpler forms. In other words, maximising the weakness of constraints on function (w-maxing) will cause simplicity of form to be maximised (simp-maxing), but simp-maxing may not cause w-maxing. Natural selection prefers bodies that can express policies that are more versatile. This forces a correlation between weakness and simplicity. Since we are products of natural selection, our languages reflect this.

Next I explore how systems do this. Biological systems seem to do a better job than AI of building versatile abstraction layers. To understand why, I look at how systems vary along dimensions of abstraction, delegation and distribution. I argue systems which delegate control to lower levels of abstraction are more adaptable. I illustrate this point using examples from biological, computational, human organisational, military and economic systems. Using Stack Theory I prove that adaptability at higher levels of abstraction requires adaptability at lower levels of abstraction. I call this The Law of the Stack. I argue biological systems are more ‘intelligent’ than because they delegate control to lower levels of abstraction. To put it provocatively, artificial intelligence is like an inflexible bureaucracy that only adapts top down. By adapting at lower levels of abstraction, biological systems can ensure weak constraints take simple forms at higher level⁵⁷. I argue this is why naturally occurring systems enforce a correlation between simplicity of form and weak constraints on function. This is why there is a correlation between simplicity of form and the weakness of constraints on function.

IX. LETS GET PSYCHOPHYSICAL

Chapter IX is about how there are objects and properties. This is brings together my work on causality^{58 59} and consciousness⁶⁰. The key result is the formalisation of causal-identites explaining how systems learn cause and effect, the Psychophysical Principle of Causality explaining why systems learn the objects and properties they do based on w-maxing, and the formalisation of selves that will inform the later theories of consciousness and meaning.

Normally to describe causality I would start with a set of variables representing objects and their properties, and then experiment to figure out if changing one variable changes another. However this only works if I already have the world divided up into variables, which my formalism doesn’t yet have. Fortunately, we have attraction and repulsion from physical states. Valence, which is a causal relation. Hence, I can flip the problem and learn the objects instead. We have proofs of optimal adaptability, and any system that adapts optimally must correctly identify cause and effect. I show that by w-maxing in response to attraction and repulsion from environmental states⁶¹, a system embodies policies that classify causes of valence. I call these policies causal-identities. They are prelinguistic classifiers. Weaker causal-identities classify more commonly encountered causes of valence. This explains why and how a contentless environment is divided up into objects and properties. I call this The Psychophysical Principle of Causality⁶². I identify two preconditions for a system to construct a causal-identity for an object: incentive and scale. First there must be an incentive, for example the object is relevant to survival. Second, the system must be able to embody the causal-identity⁶³.

To survive, I must be able to tell the difference between what I have caused, and what I did not. This implies the construction of causal-identities for one’s self. I introduce ‘orders’ of causal-identity for self, and show that if the scale and incentive preconditions are met they will be constructed. 1st-order-self classifies my interventions. A 2nd-order-self is my prediction of your prediction of my 1st-order-self. This is needed for theory of mind, or to herd and capture prey. Finally, a 3rd-order-self permits one to predict one’s own 2nd-order-selves, which is needed to predict social environments and complex narratives.

X. LANGUAGE CANCER

Chapter X is about language and cancer. This integrates my first paper, in which I proposed The Mirror Symbol Hypothesis⁶⁴, with my subsequent papers on symbol emergence⁶⁵ and the formalisation of Gricean pragmatics⁶⁶. The results are the formalisation of how meaning is communicated, of how norms are formed and how this relates to cancer, and a refutation of the Orthogonality Thesis.

I show how 2ND-order-selves are necessary for communication as described by Grice (1957; 1969)Grice⁶⁷. Grice argued that if I am speaking to you, my meaning is what I intend. You have understood me if you infer my intended meaning. A 2ND-order-self lets me predict what you think I think. I can use that to predict what you will think I intend. Hence I can anticipate what I need to express to bias your inference toward my intended meaning. Conversely, if I want to know what you mean, I can abduct that from my prediction of your prediction of my prediction of you.

This explains the emergence of norms. Organisms that can communicate can co-operate. Now that we know how, it is easy to see how language would evolve. I formalise protosymbols and preferences to connect causal-identities to established semiotic theory. I explain how co-operation facilitates social predation, and how sufficient predictive accuracy in repeated interactions incentivises honesty. I argue members of a species have similar preferences, and thus efficiency dictates an organism use its own preferences to predict others⁶⁸⁶⁹. Finally, I relate normativity to cancer. In an ecosystem computation is distributed and concurrent. Different organisms act upon one another at the same time, forming collectives. When they constrain one another in service of a goal, they form a collective informational structure with an identity. Davies and Levin (2011; 2021)Davies and Levin have argued cancer is what happens when a cell becomes isolated from the informational structure of its collective⁷⁰. I formalise this in Stack Theory. I use explain normativity as collective identity. I show that when no policy weak enough to be shared by the members of the collective, identity is lost. As such, parts of the collective will act in a manner analogous to cancer. The Law of the Stack shows systems should be as under-specified and loosely constrained as possible while still meeting their functional requirements. Cooperation and the emergence of norms depends on delegation of control to low enough levels of abstraction. I then use this to explore AI safety and refute the strong Orthogonality Thesis.

XI. WHY IS ANYTHING ALIVE?

In this chapter I ask what drives the emergence of life in an ostensibly indifferent universe? Why is it that life is complex, when complex forms are less likely to exist? In answering these questions I respond to criticisms of Pancomputational Enactivism which allege my theory does not formalise cognition in a manner which aligns with the Free Energy Principle, and accounts for a boundary⁷¹. I argue that a rock persists by simp-maxing alone, and that causes it to persist because simpler forms tend express weaker constraints. On the other hand, a system that self-repairs does the opposite. It w-maxes at the expense of simp-maxing. This is only possible in a stable environment. When the underlying stack is static, weak constraints do not need to take simple forms. A slime mold is more fragile than a rock in general, but in the context of earth’s environment it is more adaptable in the sense that it can do more, to spread and multiply. Systems like this can optimise for adaptability within the constraints of higher levels of abstraction. I then relate this to The Law of Increasing Functional Information^{72 73}, which I translate into Stack Theory and subsequently prove. Finally, I explain the Fermi Paradox using the incentive and scale preconditions for causal-identities. Intelligent systems might be all around us, but we do not recognise them as intelligent because we cannot construct a rationale for their behaviour. They fall outside the scale and incentive preconditions afforded by the human stack. This integrates my papers on abstraction layers⁷⁴, complexity⁷⁵ and most importantly my early paper on The Fermi Paradox⁷⁶. This serves to further illuminate how and why we divide our subjective worlds up into the objects and properties that we do. I do all of this in order to lay the foundations for chapter XII.

XII. WHY IS ANYTHING CONSCIOUS?

Chapter XII addresses the hard problem. I describe what is consciousness, and why is anything conscious. I published these results earlier in my papers on consciousness⁷⁷. First, Higher Order Thought theories argue that we are conscious of higher order meta-representations of lower order conscious states like ‘the smell of coffee’ or ‘the colour red’, but they don’t explain where the latter come from. To understand higher order consciousness we must explain how lower order local states of consciousness arise.

I begin by examining valence. At the most basic level we have ‘one-dimensional’ valence. How a cell is attracted or repelled, for example. Such a system cannot learn a causal-identity for any object. However, when we have two cells we now have a richer vocabulary. We can express more. If we scale up the system, we can have many parts which are being attracted or repelled by the state at any given time: a ‘tapestry of valence’. A vast orchestra of cells playing a symphony of valence. Every state of the environment would evoke such a symphony, which can be reduced to causal-identities for those aspects of the environment which cause valence. This is the point of The Psychophysical Principle of Causality. An organism learns causal-identities from valence alone. They form an abstraction layer. Causal-identities can be categorical variables like hunger and thirst, which at the higher level of abstraction have the same ‘one-dimensional’ valence, but have a fundamentally different qualities because they are different tapestries of valence at the lower level of abstraction. An organism does not have a lookup table of ordered causal-identities, and does not choose to use a policy to interpret inputs. It embodies causal-identities as policies and is impelled by valence to act accordingly. A tapestry of valence does not have the luxury of separating a representation from its estimated utility. Reward is not a label applied after the fact. Interpretation and value judgement are one and the same. I call this integrated representation and value judgement. This is counterintuitive from a computer science point of view, where we are used to dealing in key-value pairs for neat databases. However from an evolutionary perspective such a separation of description from valuation is implausible.

Consciousness is something an organism does, rather than is. It is being impelled by a hierarchy of causal-identities. I argue phenomenal consciousness begins with a 1ST-order-self. A 1ST-order-self accompanies every intervention an organism makes and so, having a character, it answers Nagel’s famous question of ‘what it is like’ to be an organism. Causal-identities become qualia. A philosophical zombie has access but not phenomenal consciousness. The contents of access consciousness are those available for communication. I argue that means access consciousness requires a 2ND-order-self, because that is what is required to communicate meaning in the pragmatic sense as humans do. Communicating requires reasoning about interventions. Hence, it also requires a 1ST-order-self. A philosophical zombie that behaves exactly like a human is therefore impossible. Intelligent behaviour at a human level requires a 1ST and 2ND-order-selves. Efficiency demands the delegated computational architecture of biological self-organisation with persistent structure that supports a tapestry of valence. Intelligence adaptability, so there is no way to achieve human-level intelligence without consciousness. Increasing intelligence is reflected in increase scale that facilitates the construction of causal-identities. I conclude the chapter by I describing the stages a conscious organism, from rocks to humans, as intelligence increases.

XIII. HOW TO BUILD CONSCIOUS MACHINES

Chapter XIII is about how to engineer conscious machines. It integrates my papers on the artificial scientist⁷⁸, and consciousness⁷⁹. The key result is a description of the features necessary and sufficient to build a conscious machine, the proposal of an unresolved problem I call The Temporal Gap, and two options describing strategies we might take to build a conscious machine, or to avoid building a conscious machine respectively.

I begin by discussing existing theories of conscious machines and AGI. I argue that, because intelligence begets consciousness and consciousness requires intelligence, these are one and the same. I frame Stack Theory and subsequently Pancomputational Enactivism as bottom up frameworks. I argue they should be used to improve rather than supplant existing theories that focus on top-down implementation of conscious or intelligent systems. I subsequently enumerate the features of an artificial scientist that should in turn lead to a conscious machine.

I examine the shortcomings of conventional computing hardware in contrast to biological polycomputers, and argue there are several features we must build into our systems if we want them to be as adaptive as a human scientist, and thus conscious. I identify a problem I call The Temporal Gap, which is that it is unclear whether a conscious state is at a point in time⁸⁰, or can be smeared across time⁸¹. Machines that satisfy the former definition are conscious according to the latter definition⁸². This has profound implications not just for what sort of machines can be conscious, but for our understanding of human subjective experience. There does not appear to be any way to conclusively resolve The Temporal Gap. However I argue that if we want to build a conscious machine we should assume consciousness is at a point in time, and design a machine accordingly. If we wish to avoid building a conscious machine, we should assume consciousness is smeared across time and avoid building potentially conscious machines accordingly.

Finally, I conclude the thesis by summarising the many and varied results.

[blank page]

II. SOME PHILOSOPHY

I MUST KNOW WHAT IT IS I WANT TO BUILD before I can build it. I want to build a mind, so that means I have to take concrete positions on disputed issues within philosophy of mind, psychology, cognitive science and neuroscience. The following is a survey of some relevant material from those fields. It is based on the introductory sections of my publications on enactive and ethical AI⁸³, communication⁸⁴ and consciousness⁸⁵. Topics covered include the mind body problem, functionalism, the “hard problem” of consciousness, various theories of consciousness, self-organisation and the free energy principle, enactivism, epistemology, semiotics, structuralism, post-structuralism and theories of meaning. Though this is a very broad ranging survey, I try to tie these concepts together into a coherent, sequential story from beginning to end.

A BRIEF HISTORY OF THE MIND BODY PROBLEM

There is public knowledge, and private knowledge⁸⁶. When I see, smell, touch, hear or taste an object, I am said to be directly observing that object. However, I cannot directly observe someone else’s experience. I can see evidence that they might be experiencing pain, for example. I could run a test and directly observe C-fibre stimulation in their brain, but that is not the same thing as directly observing their experience of pain. One’s own subjective experiences are “private” knowledge. To say something is “public” information is to say it is at least possible for more than one person to observe the event. A private event is never observable by more than one person. Even if I somehow built a computer to “read” someone’s mind, store the information, and then “write” that subjective experience into another’s mind, how could I know the experience is truly the same? When a scientific experiment is run, it is to test whether one publicly observable event reliably follows another publicly observable event. One reason it is so difficult to study the mind is because the things for which we are testing are not publicly observable⁸⁷. This brings us to the “mind-body” problem.

“What is a mind” is a loaded question, because it seems to suggest a mind is a publicly observable object⁸⁸. We know that minds are had by particular things. So instead we could ask “what does it mean when we say something has a mind?”. We know the things we can observe that have minds also have physical substance. Objects with physical substance are spatially extended, meaning that for each moment in time that they exist they must occupy space. No other physical object may occupy that same space at that same time.

However, when people speak of minds and mentality they often talk like these are not part of their physical form. For example, there is mental and physical illness. This hints at something like a mental substance. Something non-physical. However, mental and physical phenomena clearly have a causal relationship. A mind causes the body to act, and that the body causes the mind to experience what it does.

EARLY 1600s - SUBSTANCE DUALISM

The idea that there exist distinct mental and physical substances is called substance dualism. It was most famously argued by

Descartes, 16th century French philosopher and namesake of “Cartesian Dualism”. He sought to describe the union of immaterial mind and material body. His position is unsurprising, given prevailing beliefs in the 16th century. What is surprising is that his arguments were compelling enough for us to be mentioning them four centuries later.

According to Descartes mental substance does not occupy space. Mental events are not spatially extended. Presumably this is how a mind can be inside a body without making it explode. Descartes thought mental substance interacts with physical substance through the pineal gland, which acts as a sort of interpreter⁸⁹. An interpreter is like an abstraction layer in a computer. It takes one sort of thing and turns it into another. This idea that the mental and physical causally interact is called interactionism. In the case of Cartesian Dualism, the mental and physical directly interact through the pineal gland. He speculated fluids called “animal spirits” act upon the gland, causing it to move, which causes the conscious states of the mind. The mind then acts directly upon the gland, causing it to move and affect the animal spirits.

This argument has problems. I don’t need to enumerate them. Cartesian Dualism hasn’t aged well, but somehow it is still here. The reason I mention it is because I will later argue that dualism seems to have been baked into computer science⁹⁰. The idea that AI is a software “mind” running on a hardware “body” echoes Cartesian Dualism. Software is just a state of hardware, and yet many still seem to treat software as something that interacts with the world through hardware. I call this computational dualism⁹¹.

MID 1600s - PREESTABLISHED HARMONY

Following Descartes, others asked why mental substance should affect the physical through only the pineal gland, and not elsewhere? Why this inconsistency? Either mental substance affects physical substance, in which case the mental is a sort of physical substance, or it affects nothing physical. In order to preserve substance dualism (likely for religious reasons), some philosophers argued the latter, holding that causal interactions between mental and physical are an illusion perpetrated by god. Leibniz argued that mental and physical processes are set in motion by god in preestablished harmony so that they look like they interact, but never do. Like clocks synchronized by a clockmaker. The practicalities of quantum communication

are strangely reminiscent of this idea⁹². Malebranche was another philosopher who proposed yet another alternative to interactionism. He argued the physical can affect the mental only indirectly, through the intervention of god⁹³. Each time you will your body to move, god intervenes in the physical world to move your body as you wish⁹⁴. Any time your body is affected by something in the physical realm, god affects your mind. In occasionalism, god causes all interactions between the mental and physical by intervening constantly to create the illusion of interaction, whereas in Leibniz’s preestablished harmony god intervenes only once, to synchronize the mental and physical worlds so that they appear to causally interact. Either way, there is still an interpreter (god, rather than the pineal gland). I mention all this because the idea of just moving the interpreter or abstraction layer is a central theme of this thesis. It’ll come back a lot.

LATE 1600s - NEUTRAL MONISM

Both Leibniz and Malebranche denied there are any direct causal interactions between mental and physical, invoking god as a means of indirect influence. Spinoza was yet another who denied direct causal interaction, but circumvented the need for divine intervention by arguing that both mental and physical are mere aspects of a third, unobserved substance that is neither mental nor physical. In other words, reality is neither mental nor physical. This position is now called neutral monism. There is a secret third thing. Physical and mental are just aspects of this secret third thing. This idea will come up later when I formalise abstraction layers.

1800s - EPIPHENOMENALISM

Much of the difficulty in understanding the apparent two way causal interaction between mental and physical stems from the assumption that our perception of mental activity as causing physical activity is accurate. What if instead I just consider a one way causal relation? By this I mean that the mental has no causal effect upon the physical, but physical events cause mental events. We might believe we act upon the physical, but this belief is an illusion. For example, I might think I chose to get up and get a glass of water, but every aspect of my decision was determined by physical processes in my body. My mental processes are the effect, not the cause, of physical processes. This is epiphenomenalism. It was proposed by Thomas Huxley, who argued neural events in the brain are the physical events that

cause mental events. However mental events don’t actually do anything. Epiphenomenalism is a means of preserving dualism, but it leaves me wondering why anything would evolve to have consciousness? From an evolutionary perspective, epiphenomenalism seems a bit pointless. The alternative is materialism, or physicalism in contemporary terms. That’s the idea that mental events are just part of the physical world. It seems a lot more compelling, because it means we can come up with an evolutionary explanation for mental events.

NOW - PHYSICALISM

Physicalism comes in two flavours: reductive and non-reductive. The reductive physicalists think we will be able to reduce mental events to non-mental physical events. The non-reductive physicalists believe we will not be able to do that. They hold that certain physical processes have mental properties which are irreducible, meaning we can’t break them down into anything simpler and so we can’t reduce them to non-mental physical parts. This position is basically that “qualia” are fundamental building blocks of reality. This still requires mental causal efficacy, in that mental events cause other mental and physical events. Mental events must supervene on the physical, meaning two objects that are physically identical must be mentally identical. I am a reductive physicalist. Perhaps what might be called a Hobbesian Stackist⁹⁵. The main point of this thesis is to explain how. I’m now going to steel-man the non-reductive physicalists for the sake of argument.

Psychoneural identity theory is one example of reductive physicalism. It holds that the mind is the brain. Feelings, sensations and thoughts may be reduced to neural activity in the brain, or more generally to a specific physical event. Each “identity” equates publicly observable physical event with a private mental event (they are one and the same thing). One objection to psychoneural identity theory is this: if a mental event like pain is a particular neural event like C-fibre stimulation, then why is it that the same mental event can be caused by entirely different physical events? If I experience pain when my C-fibres are stimulated, and an animal appears to experience pain but has no C-fibres, does that mean it is not experiencing pain? It seems unlikely. Instead, it would seem a mental event like pain can be “realised” by any number of physical events. This is called multiple realisability.

BEHAVIOURALISM AND FUNCTIONALISM

We need to say how systems behave in order to describe mental events. Behaviouralism is the idea that one can equate mental events with outwardly observable behaviour. By observable behaviour, I mean inputs and outputs. Behaviour would be a set input-output pairs. This is one way around multiple realisability. However it mostly depends on how we define input and output. These depend on the level of abstraction. If an input is something so vague as an intuitive human definition of “pain” then yes it would appear an octopus in pain is experiencing pain like a human. If the inputs are as specific as C nerve fibre stimulation then the octopus does not have C nerve fibres and so cannot experience pain. There are many possible processes which map $I$I to $O$O exactly as $f$f does, but are not all the same thing.

This brings us to the Chinese Room. Much debated, but a good example of multiple realisability. Imagine I sit in a room. I don’t speak Chinese. Through one door I am passed a note written in Chinese. I pull out my laptop, get Google to translate the note, then pass a response back out the door. Someone outside the room then starts to believe I speak Chinese. Likewise, just because something behaves as if it has a mind, does not mean it does. Behaviouralism discounts mental activity in favour of observable behaviour. It reduces meaning to inputs and outputs. The obvious problem with behaviouralism is that there is more to the story. I think, I know I think, and I can do so without giving an output. Machine functionalism⁹⁶ tries to resolve this kind of problem by adding a causal intermediary between inputs and outputs. This causal intermediary is an interpreter like a Turing machine. It maps inputs $I$I to outputs $O$O. Given $⟨I,O⟩$the pair I, O and a function $f:I\to O$f from I to O, machine functionalism says there are many different “causal intermediaries” equivalent to $f$f. The trick is working out which Turing machine is most likely to have caused the behaviour.

For a reductive account of the mind to be convincing, it must deal with private first person behaviours (e.g. understanding meaning)⁹⁷, and show why these behaviours arise. The problem one faces then is arguing “is this behaviour really what I experience”? And we’re back to the public-private knowledge debate. To get around the public-private distinction, I argue we have to step outside the universe and look in. The only way to do that is to establish axioms that hold in every universe. That is the approach I will take in chapter 5. For now, I’ll delve further into background.

CONTEMPORARY EXPLANANDUM AND HARD PROBLEMS

Contemporary theories frame consciousness⁹⁸ as having two aspects: functional and phenomenal⁹⁹. Functional means the behaviour of consciousness, however it might be realised. Anything which might be explained by natural selection. Some equate this with “access” consciousness¹⁰⁰, which is the contents one can consciously “access” for reasoning and report¹⁰¹. I will point out some inconsistencies in how access consciousness is typically understood. I’ll argue access consciousness is merely part of functional consciousness.

The other aspect of consciousness is phenomenal¹⁰². This is the subjective experience of having “global” and “local” states of consciousness. A global state, for example, is being awake or asleep. Local states or “qualia” are the specific experiences of how coffee smells in the morning, or how wet grass feels underfoot. These are hard to define in rigorous terms. By function, I mean anything that serves reproductive and homeostatic¹⁰³ goals. That serves any goal really, but later I will make the argument that all goals stem from persistence and survival. So for now just take it as that. This information processing results in behaviour natural selection deems to be fit. Some of this information we are consciously aware of, as I am aware of the words I am writing on the page at this very moment. However most of the information processing in our bodies goes on “in the dark”. We are unaware of it, as I am unaware of whether my muscles have decided to atrophy because I have spent too long sitting in this chair writing. Why doesn’t all the information processing go on in the dark? Why do I have conscious access to some information, and not other information? Why is there phenomenal consciousness if we can just do everything in the dark?

Some speculate¹⁰⁴ we might take phenomenally conscious being like a human, and make a “zombie” of it. The zombie is a clone that has all the function of the original, but not phenomenal consciousness. From the outside it looks and acts the same, but inside it is dead. If zombies are possible, then that means phenomenal consciousness has no function. If zombies can exist then there is no evolutionary explanation for phenomenal consciousness. Supposing this is true, some have asked why is there something it is like¹⁰⁵ to be me, instead of nothing? Why do I subjectively experience some events, when it seems possible¹⁰⁶ for information processing to occur without any subject experiencing it? So to reiterate in the simplest possible terms, the functional aspect of consciousness is everything

we can explain as a consequence of evolutionary processes, and the phenomenal part is the rest¹⁰⁷. The question is whether there is such a thing as phenomenal consciousness distinct from function. My plan is to explain is to explain phenomenal consciousness as something functional, which will kill the distinction between the two.

THIS IS THE SO CALLED HARD PROBLEM OF CONSCIOUSNESS. Some interpret it as demanding a reductive explanation for local states. However, phenomenal consciousness is arguably an easy problem. Sensory processing may explain the character of qualia, and the “subject” of subjective experience may be explained in causal terms. A representation of the self lets an organism identify the effect of its actions¹⁰⁸, and is necessary for accurate inference in many circumstances¹⁰⁹. In other words natural selection demands there exist a self to be subject to sensations. What some have called phenomenal consciousness is really the function of consciousness in the first person¹¹⁰, What has been called “functional” is the very same thing from the third person perspective¹¹¹. However, that doesn’t explain why the same behaviour couldn’t come about¹¹² without consciousness. Some have separated phenomenal consciousness into first person functional and “hard” consciousness¹¹³. In that sense, hard consciousness is whatever remains unexplained by function. For the sake of this thesis and the associated papers, I interpret the hard problem as demanding an explanation of why a world in which a zombie is possible is inconceivable¹¹⁴. I’ll address the hard problem by describing how consciousness¹¹⁵ follows from evolutionary processes¹¹⁶, which follow from the very fact of existence. I describe a formalism that applies to every conceivable environment, and show that a zombie is impossible according to that formalism.

LEVELS OF CONSCIOUSNESS

Beyond the phenomenal and functional aspects, there are levels. Morin proposed four levels¹¹⁷, which I describe below. I will later make it 6 levels:

1. Unconsciousness: the absence of consciousness, including sensorimotor information processing.
2. Consciousness: a minimal level of consciousness in which one has subjective experience of local states. Phenomenal and access consciousness both begin here.
3. Self Awareness: this is where there is a distinction between public and private knowledge. One now has an inner monologue, and a concept of self. Importantly, this is where self knowledge becomes possible. It is also, according to Morin, where symbolic representations come into the picture. I interpret this as akin to the “meta-representations” in higher order theories, and I will later show that access consciousness must be equated with self awareness, because it is not possible without it¹¹⁸.
4. Meta Self Awareness: the logical conclusion of self awareness if we simply “scale up” the reflective aspect, so that one’s reflection contains a reflection. It is where one becomes aware that one is aware.

CONTEMPORARY EXPLANANS

Functional and phenomenal are broad categories that leave a lot of questions unanswered. There are several dominant theories which seek to explain how the phenomenal and functional aspects of consciousness come about.

HIGHER ORDER THOUGHT THEORIES

Higher order thought theories (HOTs)H O Ts¹¹⁹ seek to explain why we have conscious access to some information and not all. HOTsH O Ts characterise the contents of access consciousness as higher order “meta-representations” derived from lower order mental states. One can be aware of the higher order representations, but not the lower order states. This implicitly divides consciousness into higher order abstractions and lower order senses. The higher order representations are of a world divided neatly into concepts like “chair” and

“sit”. Grounded, multi-modal symbols. The lower order mental states are more primitive parts of which we cannot be aware, because our awareness is constructed from them. While the focus of HOTs is on access consciousness, they have also been used to shed light on the character of local states¹²⁰. The theory I put forward in this thesis tacitly embraces HOTs, although its origins lie with AI rather than neuroscience¹²¹. Furthermore, I define conscious access in very different terms, and point out flaws in how HOTs define access.

GLOBAL WORKSPACE THEORIES

Like HOTs, global workspace theories (GWTs)G W T s explain why some information processing goes on “in the dark”¹²². The focus is on access, not qualia. GWTs can be understood using a stage analogy. The content of which we are conscious is whatever is happening on the stage. The events on the stage are globally broadcast to all the unconscious processes which observe the stage and make use of the globally broadcast information. One gains access to sensory information when it is broadcast to different parts of the brain, in particular the prefrontal cortex. GWTs differ from HOTs in that they hold that it is the broadcast of information, rather than the composition of information to form meta-representations, that distinguishes conscious from unconscious content. GWTs explain why one might be conscious of a particular local state at a particular time, but unlike HOTs they provide little insight into why two local states might differ in character. Because of this, GWTs are often understood as providing insight into conscious access rather than qualia¹²³. They address questions like attention and working memory. GWTs like the Conscious Turing Machine seek to address the hard problem¹²⁴ by treating the phenomenal as first person functional. I take a similar stance and our theories seem to be compatible, although I take a different position on what consciousness is and how it functions.

AN ASIDE ON REENTRY

Reentry refers to the bidirectional exchange of signals between brain areas¹²⁵. It is thought to play a role in synchronized firing of neurons, allowing information to be integrated¹²⁶, forming patterns within patterns. Higher levels of activity. Some associate consciousness with top down signalling resulting from this¹²⁷.

INTEGRATED INFORMATION THEORY

Unlike GWTs and HOTs which begin with information processing and focus foremost on access, Integrated Information Theory (IIT) begins with the phenomenal, from first principles regarding the character of qualia¹²⁸. From those axioms necessary preconditions for consciousness are derived, and then it is claimed that satisfying these preconditions is sufficient to instantiate consciousness¹²⁹. This is all formalised in mathematical terms. IIT speaks of a “cause-effect structure” and the “causal power of a system to influence itself”. Global states of consciousness are associated with the quantity $\Phi$phi, that indicates the “maximum irreducible integrated information generated by a system”. If $\Phi$phi is non-zero, then the system is supposed to be conscious. The process of reentry is thought to play a key role in integrating information. Local states are then shapes in a high-dimensional space implied by the aforementioned cause-effect structure. IIT is comprehensive, the downside of which is that there exist more potential points of failure. Also, it doesn’t answer the question I want answered. It makes consciousness primary and physics secondary. I want the opposite. I want to know why anything is conscious, and I want that reason to be in terms of the physical world. The idea that consciousness is primary is fascinating, but I am more interested in the alternative. The theory I propose in this thesis is also from first principles, and also formalises consciousness in mathematical terms. However I will make the environment primary. From physics to phenomenology, as opposed to from phenomenology to physics. We do not arrive at the same conclusions, but there are complementary ideas.

AN ASIDE ON SELF ORGANISATION AND NATURALISM

Wʜᴇɴ I sᴀʏ ᴀ sʏsᴛᴇᴍ sᴇʟғ-ᴏʀɢᴀɴɪsᴇs, I mean its parts interact to produce a coherent pattern or whole. Intuitively, think of a drone swarm with no central controller. Distributed computation. The internet. Self-organization¹³⁰ is more typically defined as the spontaneous emergence of order from interactions¹³¹. The notion applies to physics¹³², biology¹³³ neuroscience¹³⁴ and of course computer science. A typical assignment for a distributed systems programming class is to write a program that interacts with copies of itself in a simulated environment, and the various copies must co-operate to achieve a goal without electing a central controller. For example, these programs might be nodes in a simulated network, and be tasked with delivering messages to specific addresses in the network without any prior knowledge of where nodes are. That is a great example of engineered self-organisation.

Sᴇʟғ-ᴏʀɢᴀɴɪsᴀᴛɪᴏɴ ɪs ɪᴍᴘᴏʀᴛᴀɴᴛ in biology because biological systems distribute and delegate control down to the level of cells and proteins. Supposing life did not begin with a centralised controller, the only possible means of organisation is self-organisation. They form a multiscale competency architecture where cells form organs, which form organisms which form ecosystems¹³⁵. In other words, a self-organising system made up of self-organising systems. To be self-organising, a system must act to occupy only a subset of possible states. A system which does not seek some states over others merely exists, rather than self-organises. More intuitively, a self-organising system will break down in some states, so it must act to remain out of those states. It must “resist a natural tendency to disorder”¹³⁶. It must optimise, or at least satisfice to a survival level. To do this, a self-organising system must predict future states in order to remain within the set of acceptable states. When I talk about an organism, I mean a biological self-organising system motivated to act in a manner deemed fit by natural selection. A conscious human is a self-organising system. So is a snowflake. Self-organisation is only part of the picture, but one well suited to naturalist explanations¹³⁷ that treat the phenomenal as something that must be functional.

FREE ENERGY

Pʀᴇᴅɪᴄᴛɪᴠᴇ ᴄᴏᴅɪɴɢ explains human perception as the result of predicting the causes of sensory signals. It frames cognition as optimisation, and thus self-organisation. Minimising expected prediction

error cost, or equivalently maximising expected utility or reward. Active inference builds upon this idea to frame cognition not only as optimising one’s internal state or “model” to correctly predict the surrounding environment, but the surrounding environment to match one’s model¹³⁸. This allows for the possibility of experimentation, to falsify one’s hypotheses. In this context, “free energy” is a bound on prediction error. By minimising free energy, one can minimise prediction error, and so active inference seeks to minimise free energy.

This is called the free energy principle. It is formalised as variational Bayesian inference¹³⁹, an approach borrowed from machine learning. It is claimed that a system which minimises free energy is optimal, making the most accurate predictions possible. Overall you can think of these ideas as a reformulation of control systems for the purpose of understanding life. To explain consciousness as a consequence of free energy minimisation is to explain it as an adaptation. A functional adaptation. Solms¹⁴⁰Solms presented such an explanation, in which one’s self is defined by a Markov blanket, in which one’s internal state is conditionally independent of the outside world. Qualia are predictions. Inward facing “interoceptive” predictions about one’s body are how one feels. Outward facing “exteroceptive” predictions are the phenomenal characteristics of the surrounding world. One has conscious access to the most probable predictions.

REAFFERENCE

One remaining noteworthy account of consciousness is that of Merker¹⁴¹Merker. Merker sought to explain subjective experience through the emergence of a subject. The ability to discern the consequences of actions logically necessitates the existence of an integrated and egocentric representation of the world from the subject’s perspective. I can’t know that “I” caused something, unless I have a representation of “I”¹⁴². For example, if a fly sits upon my shoulder and I move, this may indicate a threat to the fly. Natural selection demands the fly be able to discern the difference between the world moving because I moved, and the world moving because it moved. This ability is called “reafferance”¹⁴³. In vertebrates this capacity is supported by integrated structures in the mid-brain and in insects, the central cortex¹⁴⁴. Proponents of reafference as an explanation of consciousness argue it is where the minimal requirements for something we might call “subjective experience” are found. The theory I present largely

agrees, though for quite different reasons¹⁴⁵. That we arrived at the same conclusion from two different points of origin lends it credence. However my explanation of the emergence of causality also accounts for why organisms divide the world up into the particular objects we do. In other words, it links causality to relevance and symbol grounding.

LIQUID AND SOLID BRAINS

It should be noted here that reafference requires a degree of centralisation. It serves to integrate and unify information for navigation and other purposes. Brains, like those in humans, are solid. The neurons remain in place, and support a bioelectric network. Information is passed synchronously through this network. Timing and direct access to information is important. All of this, in service of the ability to predict and adapt.

However, brains are not the only thing that predict and adapt. Ant colonies can solve shortest path problems. Each and has a brain, sure, but the ant colony doesn’t and it seems far more intelligent than any individual ant. Ricard SoléRicard Solay proposed two classes of brain to understand this¹⁴⁶. First are solid brains with persistent structure. Second are liquid brains without any persistent structure or network and which does not require centralisation. Information in a liquid brain is always A liquid brain is asynchronous, spread across time and space, and cannot support something like a bioelectric network. This distinction will become important in the final chapters of this thesis. For now, just note that a human population is a liquid brain, and human has a solid brain.

RELEVANCE AND ENACTIVISM

Relevance realisation is the formation of a cognitive language in which inference can take place. For example, active inference describes how an organism models the world using mathematical tools like variational Bayes. In predictive coding a self-organising system assigns higher weight to more relevant aspects of the world, treating relevant aspects of the world as more precise¹⁴⁷. However, where do these aspects come from? How and why is the world divided up into particular objects? Before one can model the world and predict, one must have a language for doing so. I don’t mean a spoken language like human speech. I mean the circuitry of cognition. A vocabulary of primitive structures of which more abstract machinery can be constructed. That vocabulary determines which problems are hard, and which are easy. So before one can engage in active inference or predictive coding, one must first tackle the problem of relevance realisation, turning semantics into syntax. The organism learns a world or language relevant to its motivations¹⁴⁸.

Relevance realisation requires the organism be embodied. Yet where does the body begin and end? There is now a great deal of evidence to suggest that mental processes extend beyond the brain, into the immune system¹⁴⁹ and even the environment an organism inhabits¹⁵⁰. Intuitively, my language of cognition is not constrained to my body. I can use a pencil to write reminders on a piece of paper, and extend my memory into the surrounding environment. I am embedded in a particular environment through which my cognition is extended, and if you take me out of that environment my cognitive capabilities change. Finally, different people may interact to enact cognition, co-creating this text in co-operation with the environment, which I affect and am affected by in turn. Such distributed processing takes place not just between people, but within them. What we call human intelligence is the collective or swarm intelligence of cells¹⁵¹. This blurs the line between organism and environment, but it means we can dispense with the idea of an interpreter¹⁵². This is called enactive cognition. Intuitively, a human simplifies the world into abstract objects like “chair” and “pen”. We don’t think about details, just whole objects. We reduce the big world of all details to a small world of things which impact our survival¹⁵³. Such concepts have emerged from the interaction between humans and our environment. They are “co-created”¹⁵⁴.

PANCOMPUTATIONALISM

Computationalism, computational cognitivism or computational theory is the idea that mental processes are computational processes. From this point of view, artificial intelligence is the engineering branch of philosophy of mind. It is an attempt to formalise the systems that support mental processes and thus recreate them. This is hard to reconcile with enactivism because it presupposes some form of interpreter between the organism and its environment. It makes a firm distinction between the organism and its environment, where enactivism blurs the line between the two. In contrast, pancomputationalism is the idea that everything is computation, not just mental processes. Pancomputationalism is trivially true given a weak notion of computation¹⁵⁵. More importantly, it does not require we make any distinction between the organism and its environment, so it leaves room to formalise enactivism.

Over the course of this thesis I will formalise enactivism in terms that are compatible with functionalist, computational ideas regarding the mind. To do so I will formalise the stack in which boundaries or interpreters are formed, rather than presupposing them. Unfortunately, notions like enactivism and relevance realisation and often considered to be at odds with computation¹⁵⁶. Of course, that depends on what we consider computation to be. Some might consider computation to be just that which occurs in a human made computational system like an Apple Silicon M4 processor that uses ARM system architecture. Others might consider it to be more more abstract and general notion of Turing computation, in the sense of any machine which mimics the operations of a Turing Machine. Piccinini divides computation up into abstract and concrete sorts¹⁵⁷. Abstract is whatever we interpret it to be, much like a mathematical symbol. Concrete is that which is physically manifest in the environment. It is this latter variety I’ll formalise, in order to describe the possible worlds that might exist.

EPISTEMOLOGY

To argue such a position is justified we must also consider how it is one might come to know anything. At the beginning of this chapter I spoke of the difference between explanans (explanation) and explanandum (the thing to be explained). Given an explanandum, there may be many equally plausible explanations. This is

particularly relevant when we are considering explanations of private knowledge, that cannot be easily verified by experiment. There are so many theories of consciousness for exactly this reason. Hence, when we consider theories of consciousness we need a means of evaluating explanations, to decide which is most plausible.

OCKHAM’S RAZOR

Something more complex is more difficult to understand or predict. Ockham’s Razor amounts to the idea that simpler explanations are more likely to hold true¹⁵⁸, or are more likely to generalise. Yet simplicity is a measure of form, not function. As a subjective measure of how difficult something is to understand, complexity makes perfect sense. As a measure of something objective, namely how likely something is to hold true in future interactions with an objective reality (an environment of which one’s self is part), it makes little sense. Nevertheless, empirically it is the case that simpler explanations usually hold up better under scrutiny. One could perhaps interpret this as suggesting the environment is the product of one’s perception, or that there is something else going on. The important thing is that subjective perception of simplicity does correlate with empirical veracity. As part of this thesis, I explain why this is the case. I show this correlation is due to causal confounding¹⁵⁹. In the context of the mind body problem, SmartSmart used Ockham’s Razor to argue in favour of the mind-brain identity theory. He argued it is implausible that consciousness is non-physical while all aspects of human sensation are physical¹⁶⁰. Why should we think neural activity cause mental activity, when the simpler explanation is that neural activity is mental activity?

PRINCIPLE OF INFERENCE TO THE BEST EXPLANATION

Not all hypotheses are equally “good”. If one explains why it rains, and another explains why it rains and why the sun rises, then the latter is a “better” explanation. It explains something that otherwise would not be explained. The Principle of Inference to the Best Explanation is merely that one should prefer “better” hypotheses¹⁶¹. Of course, such a principle is not without its critics¹⁶². One obvious problem is that we might construct an infinite number of hypotheses which are equally “good” according to the criterion given above, including some which are implausibly convoluted and specific. Yet the principle is still worth mentioning as, like Ockham’s Razor, it can help identify useful explanations.

STRUCTURALIST BRAINS IN VATS

Structuralism is the idea that words and ideas are not intelligible as isolated items, but become so through their interrelations. Those interrelations are the “structure” that structuralism refers to. For example, semiotics is the study of symbols. Saussure’s semiotics defines a symbol as a sign, for example a sound or visual pattern like the word “cat”, and a thing which is signified, called a referent. For the word “cat” the referent would typically be a cat, but of course that can change depending on context. According to Saussure, signs gain their meaning from their interrelations with other signs. Put another way, meaning is the difference between signs. Structuralism became extremely influential over the course of the 20th century, until the rise of a counter-movement called post-structuralism. For our purposes a notable post-structuralist was Derrida¹⁶³, who argued that any structural description that seeks to fully encapsulate the semantics, truth or unmediated pure experience of something will be deferred or incomplete. He coined the term “différance” to describe this combination of difference and deferral. Structuralism and post-structuralism are particularly relevant given the recent success of language models. A language model like GPT-4 is optimised to learn the structure of language through their signs alone. The results have been impressive, lending credence to structuralism. However, just because a language model writes like a human does not mean it has understood the aforementioned semantics, truth or unmediated pure experience a human might have. I mention post-structuralism because my discussions with post-structuralists have proven useful. To answer my questions I’ll take a primarily structuralist approach, but one that seeks to acknowledge and formalise Derrida’s post-structural critique. If we want to know if a machine is truly intelligent, and has conscious experience like a human, then we must first answer what those things are for a human. We cannot begin at the level of any one concept. We must formalise the space of everything conceivable.

Over the course of this thesis I’ll discuss computational dualism, a criticism I published concerning software ‘intelligence’¹⁶⁴. A brain in a vat can know only what it is fed by its senses. It has no idea what is objectively true. Likewise, a computer program can know only what it is fed through hardware. The “meaning” of code is entirely determined by the hardware on which we run it. If a computer program is a model of the world, then its accuracy depends on the interpretation of it. In other words if we’re to know what a

computer program knows, then the conventional distinction between software and hardware is going to have to be abandoned. We’re going to have to avoid computational dualism, and to do that we need to formalise the space of all conceivable environments and see what holds in all of them. I’ll argue every conceivable environment has at least one state. The power set of states is the set of every possible difference. There is no difference within a state, only between states, and differences are the programs of which aspects of the environment are formed. Intuitively, it doesn’t matter what states are because we assume nothing about them. After all a human can only interact with aspects of his environment. If he were to try and pinpoint what an aspect is made of, the answer would be deferred to other aspects. This is analogous to the treatment of foundational concepts in structuralism and post-structuralism. Any answer that sought to fully encapsulate the semantics, truth or unmediated pure experience of a state would, in the language of post structuralism, be always already delayed, deferred or incomplete. This does not render such attempts vacuous, but does speak to their inherent contingency and conditionality¹⁶⁵. From there I take a firmly naturalist approach to explaining what might or must exist in every conceivable environment, working from first principles with pragmatic assumptions of natural selection and self-organisation.

PRAGMATICS AND THE ORIGINS OF OUGHT

The philosopher Hume famously showed a statement describing what ought to be cannot be derived from a statement of what is. This dissociation of value from description is named “Hume’s Guillotine”¹⁶⁶. To take a naturalist approach to explaining everything, I need to dissolve Hume’s Guillotine by showing where an original ought comes from, to get natural selection. Finally, I’ll take a moment to describe an alternative to Saussure’s structuralist semiotics. Note that Saussure’s symbols were dyadic, meaning they contained two parts. A sign, and a referent. In contrast the semiotics of Peirce defines a symbol as triadic. A Peircean symbol as a sign, a referent and an interpretant. The interpretant is the effect of the sign upon the person who interprets it. For example, if I see the word “cat” and feel hungry, this has implications for what I’ll do next. Such a pragmatic, consequence oriented account of symbols is useful for a naturalist account of meaning. I’ll dispense with “is” by arguing the very fact of continued existence constitutes an ought from which purpose and behaviour follow. As part of that I’ll formalise meaningful communication in terms of pragmatics, namely Gricean theories of mean-

ing¹⁶⁷. Grice held that the meaning of an utterance¹⁶⁸, is whatever the speaker intends the listener hold in their mind as a consequence of listening. Likewise, the listener has understood the meaning of an utterance if they come to hold in their mind approximately what the speaker intended. As an unintended consequence of following the thread from first principles to try and explain consciousness, the formalisation I’ll present just happens to align with both Peircean semiotics and Gricean pragmatics, unifying the two.

This chapter brought together many ideas. The rest of this thesis tells a more straightforward story, from beginning to end. The next chapter is yet another survey, but it tells a nice story about a bitter lesson.

[blank page]

III. WHAT THE F*CK IS AGI?

CONTEMPORARY AI SYSTEMS ARE NARROW^{169 170}, brittle, and proficient only within stable environments. Artificial General Intelligence (AGI) represents the pinnacle of artificial intelligence research: a machine that learns and adapts with the ferocity of a human mind¹⁷¹.

Many peg AGI to human-level performance across a broad range of tasks¹⁷². I myself have done this. It is a cozy, intuitive benchmark. It is also anthropocentric and so vague it’s practically a Rorschach test. This definition is insufficient, but arguably necessary. Human intelligence has many aspects. Some have emphasised autonomy, agency, and a balance of exploration in search of knowledge against exploitation of that knowledge¹⁷³. AGI is not a passive observer of the world but part of it. As Pearl puts it, a truly intelligent agent must surmount a ‘ladder of causality’¹⁷⁴. It must discriminate between events it has caused and events it merely observes. It must evaluate counterfactuals and imagine entirely alternative paths to the same end. Certainly these are all necessary for AGI. At a higher level, Goertzel (2023)Goertzel¹⁷⁵ has described AGI as a system tackling complex goals in broad, unpredictable environments. However we must then decide what is “complex”? What’s “unpredictable”?

HutterMarcus Hutter ¹⁷⁶ sought to answer such questions with a universal problem-solving model, weighted by complexity. Legg and HutterLegg and Hutter ¹⁷⁷ later framed this idea as ‘the ability to achieve goals across a wide range of environments’. It’s crisp and formal, but incomputable and entirely subjective ¹⁷⁸. CholletFrançois Chollet ¹⁷⁹ argued AGI is that which maximises ‘g-factor’. G-factor is an idea from psychology. It is how much information one requires to acquire a skill. By some accounts, this is what an IQ test is supposed to measure. However Chollet’s formal measure of intelligence is not in any meaningful way different from Legg-Hutter intelligence. It still uses complexity to assess difficulty, suffering the same pitfalls of subjectivity and incomputability. Both Legg-Hutter intelligence and Chollet’s measure treat goals as something that can be separated from intelligence. This implicitly endorses the orthogonality thesis. In AI safety the orthogonality thesis is that intelligence can be separated from final goals, and any goal can be pursued by an advanced intelligence ¹⁸⁰. As far as I can see the alternative is to treat intelligence as embodied, and since embodiment conveys a bias towards some goals over others this links intelligence to goals. I argue this refutes the orthogonality thesis later, and in a related paper ¹⁸¹. For now, what is significant about this is that it sets the foundation to frame intelligence as adaptation. Pei WangPei Wang argued in favor of this definition ¹⁸². Wang combines various definitions to arrive at ‘intelligence is adaptation with insufficient resources’. I agree with this definition. I quibble about a few details, in order to formalise it.

I give two testable definitions. The first is a quantifiable definition of intelligence ¹⁸³. It is the ability to complete a wide range of tasks: a nod to Legg-Hutter intelligence, but most closely aligned with Wang’s definition. It deals in systems as a whole. If system $A$A can complete a superset of the tasks system $B$B can, then $A$A is more adaptable. This says intelligence is contextual, and that there is no intelligence absent a goal. This measures both sample and energy efficiency. If intelligence is adaptation then AGI should be that which adapts generally.

Hence I define AGIA G I as an artificial scientist. Others have proposed this¹⁸⁴, I just formalise it¹⁸⁵. This is a high bar in terms of adaptability. Scientist is a job description. The test is can an AI do this job? Not just solving problems we hand it, but generating new hypotheses, experiments, and making real breakthroughs without relying on a human for direction. It must even give lectures, podcast interviews, apply for grants and flatter donors. An artificial scientist must be capable of autonomously making scientific progress, like a human. It must balance exploration and exploitation of knowledge. It must allocate resources. It must be able to achieve goals in a wider range of complex environments. It must identify cause and effect. It must construct plausible hypotheses and design experiments to test them. In short, an artificial scientist must satisfy all of the definitions above.

This isn’t just about AGI for AGI’sA G I for A G I’s sake. It is a stepping stone to the core of this thesis: how to build a conscious machine. An AGIA G I that discovers isn’t just clever; it’s got the kind of mental horsepower that might hint at awareness. I will explain consciousness as a consequence of function. The next sections will dig into the tech while calling out the gaps still holding us back.

EVERYTHING IS A BITTER LESSON

Having defined what this thing is I now need to say how anyone hopes to get there. Rich Sutton ¹⁸⁶Rich Sutton argues the history of AI has taught one ‘bitter lesson’. In chess, early systems encoding grandmaster strategies were eclipsed by brute-force search algorithms as computational power grew¹⁸⁷. In NLP, meticulously designed linguistic rules gave way to deep learning models trained on sprawling corpora, exemplified by the transformer architecture¹⁸⁸. Sutton’s insight was that the relentless march of compute trumps human ingenuity¹⁸⁹.

To solve a problem I can hand-craft clever solutions to problems, or I can apply general methods like search or approximation and just optimise for what I want¹⁹⁰. If resources are not a consideration, then general methods will eventually beat any approach that relies on human-crafted knowledge or structures. AI started to be of practical use because hardware improved to the point where AI could be applied at scale, not because anything significant changed with the algorithms. The Bitter Lesson gives you The Scaling Hypothesis. The Scaling Hypothesis asserts that by amplifying the size of AI models, the volume of training data, and the computational power deployed, we’ll eventually rival or surpass human capabilities. The Scaling Hypothesis has surged in prominence, fueled by the striking achievements of large-scale models across diverse domains. For example, OpenAI’s GPT-3, boasting 175 billion parameters, showcased remarkable proficiency in generating human-like text, executing tasks with minimal prompting, and even hinting at basic reasoning¹⁹¹.

Likewise, DeepMind’s AlphaFold 2 harnessed vast computational resources and biological datasets to revolutionize protein structure prediction, solving a decades-old challenge in biology¹⁹². These breakthroughs demonstrate that scaling does get results, at least to an extent. Empirical support for the scaling hypothesis is bolstered by scaling laws, which reveal predictable performance gains as model size, data, and compute increase. Kaplan et al. ¹⁹³Kaplan and colleagues demonstrated that in natural language processing (NLP), larger models consistently improve in performance. This hints at a systematic relationship between scale and capability¹⁹³. Advocates argue that as models grow and ingest more diverse data, they approximate a deeper, more general understanding of the world.

There are critics. I count myself among them. While scaling might eventually work, the word “eventually” is doing a lot of work. There are diminishing returns. Beyond a certain threshold, additional parameters yield only incremental gains, suggesting a ceiling to this approach. In language models, performance gains taper off as size increases. Marginal improvements don’t justify exponential resource costs. This plateau challenges the notion that scale alone can bridge the gap to AGI. The environmental toll of training behemoth models is staggering, with carbon emissions rivalling those of small industries¹⁹⁴. This is exacerbated by the fact that scaled models excel in their training domains but often falter beyond them. Large language models generate fluent text yet stumble on tasks demanding deep reasoning or contextual nuance¹⁹⁵. Some suggest this neural networks are fundamentally incapable of reasoning or causal understanding¹⁹⁶. I don’t know about that. A full human brain integrated with a human body is quite spectacular. A chunk of human brain sitting on a counter-top tends to be rather ghoulish and unimpressive. I do know these systems are sample inefficient, meaning they need a lot of data or many ‘examples’ to learn from. That is a criticism I find compelling. Adaptability is about dealing with edge cases, not rote learning. A system that needs a data centre to learn tic-tac-toe isn’t intelligent: It’s a whale beached on silicon. Finally, scaling assumes you know what you want and can measure it. That is quite the assumption. We can mimic human behaviour, but is that really what we want? To replicate ourselves?

The Scaling Hypothesis is potent. Yet it is not a silver bullet. Empirical success must be weighed against diminishing returns, theoretical gaps, and ethical trade-offs. To understand how we can do this, we must examine what exactly it is that we’re scaling. The typical ML and AI concepts like supervised learning, reinforcement learning, inference, reasoning, planning and so on aren’t useful because an artificial scientist must able to do all of it. Instead I will take my cue from Sutton’s bitter lesson and speak only of the means by which these things are achieved. These means are the search and approximation. This is not the only way to think about this, so I then discuss hybrids. Hybrids are those systems which do not fall neatly into the buckets of search and approximation. Finally, I discuss meta-approaches, which are frameworks through which search, approximation and hybrid systems can be understood. Meta-approaches give us a quantifiable answer to ‘what is intelligence’ that other systems can optimise for.

BASIC TOOLS

SEARCH

SEARCH IS THE HISTORICAL WORKHORSE OF AI¹⁹⁷. I include any symbolic reasoning and planning in this bucket. In its most basic form search involves representing a problem space and solution criteria. Then every nook and cranny of the problem space is explored and tested until until a solution is found. Rooted in the foundational era of computation, search-based methods embody the belief that intelligence can be distilled into systematic exploration of well-defined possibilities. This section dissects search-based AI. I discuss operational principles, its strength in structured domains, its limitations in the face of complexity, and its fit within the broader quest for AGI. It is a precision instrument. At its core, search-based AI is about exhaustive exploration. Whether it’s planning a route, solving a puzzle, or proving a theorem, search involves a representation of the problem’s state space (often as a graph or tree). A search algorithm then systematically traverses it, evaluating paths against a defined goal. This is the essence of algorithms like breadth-first search (BFS), which explore all nodes at the current depth before moving deeper. Depth-first search (DFS) dives deep into one path before backtracking. A more sophisticated method called A*¹⁹⁸A star employs a ‘heuristic’ to guide the search toward promising areas. A heuristic is like a rangefinder, and A*A star searches the nodes that the heuristic says are closer to the goal first. A canonical example of all this is SatPlan¹⁹⁹, which transforms planning problems into Boolean satisfiability (SAT) instances, solvable via logic-based search. SatPlan and its ilk have excelled in domains like logistics scheduling and automated reasoning where the problem can be fully specified. By this I mean states, actions, and goals can be laid out clearly. Search thrives in these environments, where the solution is a matter of finding the optimal path through a labyrinth of possibilities.

Search has its advantages. Here are a few:

• OPTIMALITY: When properly configured (e.g. with an admissible heuristic in A*A star), search algorithms guarantee the discovery of the optimal solution, provided one exists. This is invaluable in domains where precision is non-negotiable, such as automated theorem proving²⁰⁰ or mission-critical planning in aerospace.
• INTERPRETABILITY: The process is transparent. Each step can be traced and understood. That makes search-based systems easier to debug, verify, and trust than their approximated counterparts.

• STRUCTURE EXPLOITATION: Search excels in problems with well-defined structures, where the state space, though potentially vast, is navigable through clever pruning and heuristic guidance. This makes it a go-to for tasks like game playing (e.g. chess engines pre-AlphaGo) and pathfinding in robotics.

These strengths have cemented search as a cornerstone of AI, particularly in environments where correctness and transparency are paramount.

HOWEVER, SEARCH ALSO HAS DRAWBACKS:

• COMBINATORIAL EXPLOSION: The primary curse of search is its scalability. For problems with large state spaces, the number of possible paths grows exponentially, a phenomenon known as the combinatorial explosion. Even with heuristics, search can become computationally intractable for all but the most carefully constrained problems. In chess the state space is approximately $10^{46}$ten to the forty sixth nodes. This is too large for brute-force exploration without aggressive pruning. Prior or contextual knowledge can be used constrain the search space and mitigate this problem.
• SEQUENTIAL NATURE: Search algorithms are sequential, making them ill-suited for modern parallel hardware like GPUs, which thrive on matrix operations and batch processing. This puts search at a severe disadvantage compared to approximation-based methods, which can leverage massive parallelism to accelerate learning and inference. Concurrent and distributed search algorithms exist, but have not yet matured into user friendly and scalable libraries²⁰¹.
• RIGIDITY IN PROBLEM FRAMING: Search demands a pristine problem definition. This means explicit states, transitions, and goals. Real-world problems are often riddled with uncertainty. Search falters in these environments, requiring human intervention to massage the problem into a tractable form. This reliance on human pre-processing is a far cry from the autonomous adaptability we seek in AGI. However this ceases to be a significant problem if search can be made more efficient and scalable.

In its current form, search-based AI is a perfectionist that thrives in controlled, sterile environments but wilts when faced with the chaos of reality.

Search has a few notches in its belt:

• SATPLAN: By converting planning problems into SAT instances, SatPlan has solved complex logistics and scheduling tasks with precision²⁰². However, its reliance on well-defined constraints limits its applicability to more fluid, real-world scenarios.
• CHESS ENGINES (E.G. DEEP BLUE): Chess engines like Deep Blue²⁰³ relied on search algorithms augmented with evaluation functions to defeat world champions.
• PATHFINDING ALGORITHMS: $A^{\ast }$A star and its variants remain the gold standard for navigation in robotics and video games²⁰⁴, efficiently plotting optimal routes in static environments. But again, their effectiveness diminishes with increased uncertainty and dimensionality.

These examples underscore search’s prowess in structured domains while highlighting its limitations in more complex, adaptive settings.

APPROXIMATION

By approximation I mean curve fitting. I mean all those artificial intelligence techniques that address complex problems by approximating underlying functions, distributions, or decision surfaces, rather than relying on exhaustive computation or exact solutions. Unlike search, approximation-based approaches excel in environments with high dimensionality and noise. Computer vision and natural language processing systems depend heavily on approximation. This section briefly examines its defining characteristics, advantages and limitations. At its core, approximation-based AI optimises a model reflect patterns in data so it can be used to make predictions about other data generated by the same source. In its simplest form this would be like writing down and averaging someone’s score in a game so you can predict what they will get in future. There is something that generated data (the player and games), and you train a model (by taking the average) until it reflects some aspect of the generator. I can train a model to classify data, answering ‘which thing generated this data?’. I can also train a model generate new data.

Typically a parameterized model such as a neural network approximates a target function by minimizing a loss function over a training dataset. Mathematically, given an input space $(X)$X and output space $(Y)$Y, the goal is to find a function $f_{\theta }:X\to Y$f theta from X to Y, parameterized by $\theta$theta, that closely matches the true mapping $f^{\ast }$f star, even when $f^{\ast }$ is unknown or intractable. The error is typically quantified via a loss function $L(f_{\theta }(x),y)$L of f theta of x and y, and optimization techniques like gradient descent adjust $\theta$ to minimize this loss over a dataset $D=\{(x_i,y_i){\}}_{i=1}^N$D, defined as a set of x i, y i pairs from i equals one to N. The ascendancy of deep learning, a subset of approximation-based AI, has been particularly notable. Deep neural networks leverage multiple layers of interconnected nodes to learn hierarchical feature representations, enabling them to tackle tasks with unprecedented accuracy. For instance, convolutional neural networks (CNNs) have redefined computer vision²⁰⁵, while transformer architectures have revolutionized natural language processing²⁰⁶. Approximation-based end-to-end reinforcement learning has shown promise in game playing and robotics²⁰⁷. Approximation is ideally suited to scenarios where we can trade accuracy and reliability for scalability and practicality.

Approximation has advantages over search:

• scalability: These methods efficiently process large-scale, high-dimensional data. For example, convolutional neural networks can

classify millions of images by learning compact feature representations, bypassing the need for exhaustive hand-crafted rules.

• ROBUSTNESS TO UNCERTAINTY: By modeling data distributions probabilistically or incorporating regularization, approximation-based models can generalize from noisy or incomplete inputs. Techniques like dropout in neural networks²⁰⁸ or Bayesian methods enhance this resilience, making them suitable for applications like speech recognition in variable acoustic conditions.

• FLEXIBILITY AND AUTOMATION: Search often requires a domain-specific heuristic. Approximation is cheaper, which means it can learn directly from data. This is ideal for problems where the relationship between inputs and outputs is highly non-linear or poorly understood. It can minimise the need for human-engineered features. This adaptability has fueled its adoption in fields from genomics to finance, with minimal reconfiguration.

Scalability in particular has led to widespread adoption, as you might expect given the bitter lesson. Some examples:

• CONVOLUTIONAL NEURAL NETWORKS (CNNS): CNNs exploit spatial locality and parameter sharing to achieve state-of-the-art performance in visual tasks²⁰⁹.

• TRANSFORMERS: Transformers rely on self-attention mechanisms to model long-range dependencies in sequences²¹⁰. Models like BERT²¹¹ and GPT-3²¹² have set benchmarks in natural language understanding and generation, leveraging massive datasets (e.g. GPT-3 was trained on 45TB of text) to approximate linguistic structures.

• DEEP REINFORCEMENT LEARNING: Deep Q-Networks (DQN)²¹³ combine neural networks with Q-learning to approximate value functions, achieving human-level performance in Atari games. Similarly, Proximal Policy Optimization²¹⁴ has advanced policy approximation in continuous control tasks.

These examples highlight the ability of approximation-based AI to address diverse challenges with tailored architectures.

Despite recent success, approximation is not a panacea:

• UNRELIABILITY: Approximation is only approximate. Stochastic. It is unreliable by design²¹⁵. This makes it difficult to apply to problems where failure cannot be tolerated. This is why search is used for applications like maps and directions. Directions that ‘approximate’ a route through a river are not useful.
• INTERPRETABILITY: The complexity of models like deep neural networks, often with millions of parameters (e.g. GPT-3 has 175 billion), renders them opaque. This “black box” nature complicates understanding of decision rationales, a critical issue in domains requiring accountability, such as medical diagnostics or legal systems. Efforts like LIME²¹⁶ and SHAP²¹⁷ provide post-hoc explanations, but these are often approximations themselves and lack the rigor of causal insight.
• SAMPLE INEFFICIENCY: High performance hinges on access to large, labeled datasets. For instance, training ResNet-50 on ImageNet requires 1.28 million labeled images, while GPT-3’s training consumed computational resources equivalent to thousands of GPU days²¹⁸. In data-scarce domains, such as rare disease diagnosis, this dependency limits applicability and risks overfitting, where $f_{\theta }$f theta fits noise rather than signal (bias-variance trade-off). In other words, approximation is maladaptive. Techniques like transfer learning can mitigate costs, but performance still drops sharply outside the training distribution²¹⁹.
• COMPUTATIONAL COST: The training of approximation-based models incurs substantial energy and infrastructure demands. For example, Strubell et al. (2019)Strubell and colleagues²²⁰ estimate that training a single transformer model emits carbon equivalent to 626,000 miles of car travel, raising sustainability concerns.

These drawbacks underscore the trade-offs inherent in approximation, necessitating careful consideration of context and resource constraints.

HYBRIDS

Hybrids are those systems which do not fit neatly into the search or approximation buckets. Biological self-organising systems learn and adapt, but they are not clearly a case of just search or approximation. Hybrid approaches are inherently more general because I can pick choose any general approach for any occasion. I can fuse search and approximation, or something else. By combining complementary strengths, hybrid systems offer a tantalizing path toward AGI, promising robustness where monolithic approaches falter²²¹. Perhaps no single AI paradigm holds the key to AGI. Search excels at precision. Approximation thrives on raw data and uncertainty. Hybrid systems bridge these gaps, blending precision with flexibility, logic with learning. The goal? Synergy. Emulate humanity’s versatility, tackling everything from sensory processing to scientific discovery.

Hybrids take many forms. AlphaGo²²² is the simplest example of how approximation and search can complement one another. This hybrid crushed Go’s world champion in a testament to blending search and approximation²²³. Search allowed it to plan sequences of moves that conformed to the rules of Go, while approximation allowed it to figure out which sequences of moves were most likely to win. Hybrids can also take the opposite approach. Neuro-symbolic hybrids tackle the symbol grounding problem by linking raw data to abstract concepts²²⁴. Think neural nets mapping inputs to symbols, then reasoning over them. Structured reinforcement learning hybrids use this kind of approach, using approximation to process sensory data and search to choose actions. Raw, high-dimensional sensory data is too much for search to cope with, so approximation ‘reformats’ it into a simpler, structured, low-dimensional symbolic representation. In this case a convolutional autoencoder learns to ‘compress’ the raw data down to a small size and then back again, ensuring important information isn’t discarded by converting the sensory data to the smaller format. The low dimensional data are clustered and labelled as ‘objects’ with properties based on geometry and where they are on the screen. These objects can then be tracked as the world changes over time, to get learn their dynamics and spatial interactions. More conventional reinforcement learning techniques are then applied to learn a policy in these highly abstracted, symbolic terms. The resulting agent adapts far more efficiently²²⁵. Finally and most importantly there are fully autonomous, general purpose systems. Cognitive architectures like SOAR²²⁶ and ACT-

R²²⁷. These weave search and approximation together for flexible, multi-task competence. The most prominent examples are ongoing projects that have shown steady improvement year on year:

• HYPERON: Probabilistic logic networks meet neural nets in a bid for holistic cognition. Perception, memory and reasoning in one package. It aims to build AGI on a modular, distributed, self-organising system that can integrate new technology as it develops²²⁸. For example, new components have been proposed based on active inference and the free energy principle²²⁹.
• AERA: The Autocatalytic Endogenous Reflective Architecture (AERA) self-programs, reflecting on its own symbolic structures while learning statistically. It’s a stab at autonomy and growth²³⁰.
• NARS: The Non-Axiomatic Reasoning System (NARS) rejects rigid axioms for a fluid, adaptive logic. NARS operates under the Assumption of Insufficient Knowledge and Resources (AIKR), reasoning with incomplete, uncertain data via a non-axiomatic framework. It integrates symbolic reasoning with probabilistic inference, using a custom inheritance-based logic (NAL) to derive conclusions from limited evidence. Designed for real-time adaptability, NARS learns incrementally, refining its knowledge base as new inputs arrive—think of it as a brain that thrives on ambiguity, not a theorem prover shackled to certainty²³¹.

Hybrid systems give us the best of all worlds. Fusion of search and learning is a general approach that can be scaled. Hybrids are also more useful in the short term. Structured priors or search can narrow the problem space, improving sample and energy efficiency compared to brute-force approximation. The high-level symbolic abstractions often used for search are interpretable by humans. Conversely, we can easily integrate human priors into hybrid systems. Hybridisation can be a shortcut to autonomous agents. Hybrids can combine a persistent identity and interpretable goals with the ability to process raw, high-dimensional real-world environments. For example, scaffolding like memory can enable long term adaptation in ontologically stateless language models²³². Hybrids edge us closer to AGI by mimicking diversity of human cognition. Yet I have lingering questions. What is missing? Is a given hybrid system truly scalable or just a clever patchwork that exemplifies Sutton’s bitter lesson? Can we scale these systems to AGI?

META-APPROACHES

A meta-approach is a frame through which systems can be understood. It is a guiding principle I can use to tweak search, approximation or hybrid systems to be more ‘intelligent’. Meta-approaches are not mutually exclusive. The scaling hypothesis is an example of a meta approach through which I have framed search and learning. I call this scale-maxing because it works by maximising scale. For example, maximising the amount of training data, the avaible compute and the size of the model. There are two other meta-approaches I can identify. One is orthodox at prominent AI labs like Deepmind and OpenAI. I call it simp-maxing because it involves maximising the simplicity of forms. It is founded on Ockham’s Razor. For example, if I have a perfect compression algorithm and I use it to compress two files, then the smaller compressed file is the simpler one even if the uncompressed files were the same size. Likewise, if I use regularisation to make regression converge on a simpler function, then I am simp-maxing. The last meta-approach is my own invention, which I propose in this thesis^{233 234}. I call it w-maxing because it optimises for the least specific, weak constraints on functionality at the lowest possible levels of abstraction. So to reiterate, scale-maxing is about maximising available resources, simp-maxing is about maximising simplicity of forms, and w-maxing is about maximising the weakness of constraints implied by function. In this section I will focus on simp-maxing, and will explain my stack-based approach later.

Simp-maxing is about applying Ockham’s Razor to make more accurate models²³⁵. It posits that among competing hypotheses which might explain some observed data, the simplest one is most likely to be correct. In AI, this translates to favouring models or solutions with lower complexity, as they are less prone to overfitting and more likely to capture the underlying structure of the problem. Examples of simp-maxing include regularisation²³⁶, the minimum description length principle²³⁷ and Universal Artificial Intelligence (UAI)²³⁸. UAI is the dominant mathematical formalisation of artificial general intelligence. It relies on Kolmogorov complexity²³⁹, which defines the complexity of a string as the length of the shortest program that can generate it. For a dataset $D$D, the Kolmogorov complexity $K(D)$K of D is the smallest program $p$p such that $U(p)=D$U of p equals D, where $U$U is a universal Turing machine. This concept extends to models. This connected simplicity to compressibility²⁴⁰. Simpler representations are shorter, and according to Ockham’s Razor simpler models are more accurate. Kolmogorov Complexity is incomputable but we can approximate it. Computable alternatives exist, like minimum description length (MDL)²⁴¹ or Lempel-Ziv compression²⁴². Solomonoff²⁴³ subsequently proposed a universal method for inductive inference based on algorithmic probability, where the likelihood of a hypothesis is proportional to $2^{-K(h)}$two to the power of negative K of h, with $K(h)$K of h being the Kolmogorov complexity of the hypothesis $h$h. This formalizes Ockham’s Razor in a probabilistic framework, favoring simpler hypotheses. Hutter subsequently proposed AIXI, a general reinforcement learning agent that uses Solomonoff induction to make optimal decisions based on the simplest hypotheses²⁴⁴. This gives us a theoretical frame through which to view search and approximation. We can use it to come up with practical solutions. For example, machine learning techniques like regularization (e.g. $L1$L one and $L2$L two norms, or dropout²⁴⁵) explicitly penalize complexity to prevent overfitting. This improves out of domain generalisation. Similarly, Pruning in decision trees reduces model size while maintaining accuracy.

This serves to illustrate what a meta-approach is. It provides a guiding principle for adaptability and generalization. A meta-approach can be applied in the context of search or approximation.

CONCLUSION

I’ve defined intelligence in terms of adaptation, AGI as an artificial scientist and laid out some of the tools available for that quest. These include search, approximation, hybrids, meta-approaches, and the relentless march of scaling.

Foundational tools:

• SEARCH (e.g. navigation apps, DeepBlue),
• APPROXIMATION (e.g. GPT-3, deep Q-learning).

Hybrids:

• SIMPLE (e.g. AlphaGo, structured reinforcement learning),
• COMPLEX (e.g. Hyperon, AERA, OpenNARS²⁴⁶).

Meta-approaches:

• SCALE-MAXING: maximise available resources (e.g. OpenAI’s GPT series LLMs),
• SIMP-MAXING: maximise simplicity of forms (e.g. regularisation²⁴⁷, UAI²⁴⁸, minimum description length principle²⁴⁹),
• W-MAXING: maximise weakness of constraints implied by function²⁵⁰.

EACH OFFERS A PIECE OF THE PUZZLE. With sufficient resources any system that learns can eventually attain an arbitrary level of skill. Every system can be optimal. However not all systems are equally adaptable. Hence I’ll conclude this chapter by reiterating that intelligence is a matter of adaptability, and thus efficiency. All else being equal, the more resources the system needs to reach a certain level of performance, the less intelligent it is. What I offer in this thesis is a meta-approach that lets us measure and maximise adaptability.

IV. WOW, EVERYTHING IS COMPUTER

There is a problem with simp-maxing. It works, but there is no apparent reason it should. After all, the No Free Lunch Theorem shows no algorithm outperforms others across all problems²⁵¹²⁵². Indeed, it turns out AIXI’s performance is entirely subjective²⁵³. The root of this subjectivity is in the definition of Kolmogorov complexity. For a given string $(x)$x, its Kolmogorov complexity $K_U(x)$K sub U of x is defined as the length of the shortest program that, when run on a universal Turing machine $(U)$U, produces $(x)$x. Formally:

K sub U of x is the minimum length of a program p such that U of p equals x

where $(|p|)$the length of p is the length of program $(p)$p. However, this definition is inherently tied to the choice of $(U)$U. Different universal Turing machines can yield different complexity values for the same string. Specifically, for any two universal Turing machines $(U)$U and $(V)$V, there exists a constant $(c)$c such that:

for all x, the absolute difference between K sub U of x and K sub V of x is less than or equal to c

This is the invariance theorem²⁵⁴. While this suggests that the difference in complexity is bounded, the constant $(c)$c can be arbitrarily large in practice, making comparisons across different machines problematic. AI is inherently interactive, which means we are dealing with more than one machine. AIXI is only optimal if the UTM it uses matches some other arbitrarily chosen UTM used to measure intelligence²⁵⁵. AIXI was supposed to be the most intelligent agent according to Legg-Hutter intelligence²⁵⁶. Legg-Hutter intelligence is kind of the opposite of AIXI. AIXI is assumed to be intelligent becuase it behaves according to Ockham’s Razor. In contrast, Legg-Hutter intelligence measures intelligence according to Ockham’s Razor: agents behave in a way that can be described with shorter program are more intelligent. Simplicity is once again measured using Kolmogorov complexity. The invariance theorem claims Kolmogorov complexities only shift by a constant across UTMs²⁵⁷. This doesn’t hold in an interactive setting, because in an interactive setting we have two UTMs. One with respect to which AIXI is computed, and one with respect to which Legg-Hutter intelligence is measured. If Legg-Hutter intelligence is measured with respect to one UTM, and AIXI is computed using another, then AIXI might think it has chosen

a short program that Legg-Hutter intelligence interprets as a long
program. For instance, a string that appears simple relative to one
Turing machine might seem complex relative to another, depending on the machine’s instruction set or encoding scheme. Consider
the analogy of programming languages, which can be thought of as
different universal Turing machines. Suppose we have two programming languages, $L_1$L one and $L_2$L two. Language $L_1$L one has a built-in function that
directly generates the Fibonacci sequence, while $L_2$L two does not. Now,
consider a string $(x)$x that represents the first 100 Fibonacci numbers.
In $L_1$L one, the shortest program to generate $(x)$x might be a single function call, say print_fib(100), making $K_{L_1}(x)$K L one of x very small. In contrast,
in $L_2$L two, the shortest program would need to implement the Fibonacci
sequence from scratch, resulting in a much larger $K_{L_2}(x)$K L two of x. Thus, the
same string $(x)$x has vastly different complexities depending on the
chosen language, illustrating the subjectivity introduced by the choice
of reference machine.

REFRAMING THE PROBLEM

AGI is supposed to be capable of adapting to any task or environment. However, if its internal measure of simplicity is tied to a specific, arbitrary choice of reference machine, its adaptability may be constrained by that choice. This could lead to blind spots or inefficiencies in certain domains, undermining the goal of general intelligence. AIXI illustrates an extremely valuable idea, but its subjectivity is a problem. Some have explored complexity measures that are invariant under certain transformations, aiming to reduce dependence on the reference machine. For example, Levin complexity²⁵⁸ incorporates time complexity into the measure, potentially offering a more universal metric. However this is still a measure of form, not function. Simp-maxing is not optimal, and if I want to understand intelligence I need to know what I’m aiming at. I want to know what the upper bound on adaptability is.

To solve this problem I need to go down a level of abstraction. I need to reframe the problem. Kolmogorov complexity takes information in one format $A$A and represents it in another $B$B. $B$B is a language. It is how we represent information in a Universal Turing Machine (UTM). It is in $B$B that length is measured, and length depends on how we format information in $B$B. Short isn’t universal²⁵⁹. It is tied to the UTM you pick. The UTM is an interpreter. $B$B is an abstraction layer on top of $A$A. New UTM, new definition of simple, new AIXI. In hindsight this seems obvious. Lieke and Hutter also concluded that Pareto optimality is trivial. That seems less obvious to me, so I proposed an analogy to describe the issue. Lets assume the environment is a function $f_1$f one: it takes AIXI’s actions and coughs up observations and rewards. The UTM is $f_2$f two: it decodes AIXI’s guesses, which are programs describing what happens next. AIXI’s algorithmic, software ‘mind’ is $f_3$f three: it churns out those guesses. The reward $r$r comes from $f_1(f_2(f_3))$f one of f two of f three. You can’t judge AIXI by $f_3$f three alone. It’s the stack that determines success, by which I mean the environment, UTM and algorithm together. Switch the UTM, and the output changes. Pareto optimality is only trivial if you can change part of the stack. If we consider the entire stack, then Pareto optimality is far from trivial. The problem is that Kolmogorov complexity is a matter of form, not function²⁶⁰. Whatever notion of complexity we use, it is a matter of form²⁶¹. Software is just a state of hardware²⁶². Any claim regarding a software mind are symptomatic of a condition I call computational dualism. It is pointless to make claims about an optimal software mind if the environment and interpreter can be changed.

MORTALITY

Descartes thought it was ‘animal spirits’ and the pineal gland passing messages between mind and body. AI research has replaced the pineal gland with a Turing machine. It is a ghost haunting AGI labs like it’s 1637. Software is the mind, hardware is the meat, and never shall they meet. Computational dualism is the idea that artificial intelligence is about creating intelligent software²⁶³. It is not. That should be obvious. Software is a state of hardware. AI is about making an intelligent system, and its state its part of it. Computational dualism is a problem if we want to build an intelligent system, because it ignores half the equation. We need to know what intelligence is if we want to optimise for it. We need to know what optimal looks like so we can work towards it.

It is not as if people haven’t already pointed out there’s a problem, it is just that they have chosen to live with it²⁶⁴. As far as I can tell, only one other has gone to the trouble of attempting to formalise an alternative²⁶⁵. Orseau attempted to formalise a version of AIXI which was interpreted by the environment, using bounded optimality. It is a commendable and compelling attempt, but it does not go far enough. It does not answer the questions I want answered. Symptoms of computational dualism remain. Software is a convenient abstraction and it works well for building standardised applications for standardised hardware in standardised contexts. It becomes more of a problem when we consider an agent interacting with the world. Many appear to have forgotten software is nothing more than a state of hardware. It has spawned wild AGI myths, like superintelligent code rewriting physics or escaping its box²⁶⁶. Even Nobel laureate Geoffrey Hinton has resorted to doomsaying²⁶⁷.

He speaks of mortal vs immortal computation as if software lives on in the absence of hardware²⁶⁸. His arguments seem to hinge on software’s ability to leap from one hardware platform to another, retaining its functionality like some eternal digital essence. He suggests that because software can be duplicated across different machines without losing what it does, computation somehow transcends its hardware. At first glance, it seems plausible. Copy your code, run it elsewhere, and the process lives on. But that’s an illusion. Software is a state of hardware. When the hardware changes or fails, the computation changes or fails. Copying doesn’t preserve the original. It births a new instance as mortal as the last.

Hinton’s immortal computations don’t exist. There is only mortal software, because there are only finitely many devices. Software is a state of hardware, a pattern etched in silicon or flesh. Change the hardware, and the ‘mind’ follows. Human intelligence is embodied, not just symbol shuffling in the abstract²⁶⁹. Software is no different. The analogy of the brain as a computer only works if you treat it as a unified system with no distinction between hardware and software²⁷⁰. Intelligence isn’t a disembodied mind interacting with but a dance between hardware and environment. Cognition is a physical act, not a ghost in a shell²⁷¹. Every physical system computes simply by existing²⁷². It is a whole-of-system physical process. Hence, I take a whole-of-system approach. Hardware, software and world are entwined.

FLIPPING THE TABLE

Computational dualism is a dead end. Software is a state of hardware. Hardware is a part of a larger system. Here I argue everything is nested abstraction layers from software to hardware to the laws of physics. This brings together several of my papers²⁷³. To quote a certain ambulatory meme, everything is computer.

Software doesn’t exist. At least, not in the way people seem to think. A Python script relies on an interpreter written in C. C is compiled into assembly. Assembly to machine code. Machine code is hard-wired in silicon. We don’t ‘run’ software. We flip switches in a box. As AIXI illustrates, code is nothing if it is not etched in meat or metal. Same code, different rig, different mind. Why? Because a body isn’t a neutral translator. Hardware is goal-directed, just like software is. Some hardware is better for some tasks. It is an abstraction layer.

Abstraction does not end at hardware. Hardware is not the bedrock. Intuitively, think of a play. Software is a script, the hardware is the actor and the environment is the stage. Change the actor or the stage, and the show is not the same. An actor is not the play. Making hardware the foundation would repeat the mistake of computational dualism. Hardware is a body embedded in the world and bound by physics²⁷⁴. A CPU is a hunk of matter obeying laws we’ve barely glimpsed. Physical laws, only they are not really ‘laws’. That is just how we understand what is happening²⁷⁵. We scribble on blackboards. We approximate reality. Whatever it is that those laws approximate, that is hardware’s puppeteer. Nature’s machinery. A transistor flips because nature says so, not because some coder waved a wand. Hardware is a middleman, enacted by something less abstract. Hardware is just another abstraction layer, like software.

Put another way, reality is a Matryoshka doll. Software is a state of hardware, which is a state of the physical reality we inhabit. A human is a state of organs, which are states of cells, which are states of molecules²⁷⁶. Each level is a state of the one below. Everything a program running on a deeper machine.

I call it The Stack. It is patterns within patterns²⁷⁷. I’ll return to my function analogy. The mind is $f_3$f three, the body $f_2$f two and the local environment is $f_1$f one. Now can add $f_0$f zero for our physical laws or whatever the local environment is running on. The underlying hum of reality. The stack is then $f_0(f_1(f_2(f_3)))$f zero of f one of f two of f three. A computer is the same. We can break things down into more granular detail. $f_n$f n could be Python, $f_{n-1}$f n minus one C, $f_{n-2}$f n minus two assembly, $f_{n-3}$f n minus three machine code, $f_{n-4}$f n minus four a particular machine, $f_{n-5}$f n minus five the local environment and infrastructure with which the computer interacts all the way down to physics $f_0$f zero. Hardware is not a sacred boundary where abstraction stops and reality kicks in. Hardware is as abstract as code. A computer is designed by a human, but it obeys the laws of physics. It follows a script we did not write²⁷⁸.

THE LIMITS OF KNOWING

How far down can the stack go? Gravity, quarks and spacetime sound foundational, but they are human abstractions. Our physics is a guess scrawled in chalk by apes²⁷⁹. We know it is incomplete. Our formulae are programs running on a human brain. They are not fundamental²⁸⁰. Even numbers are not fundamental. Numbers are just a means by which we describe the order we perceive. Could there be more? Maybe $f_{-1}...f_{-n}$f sub minus one through f sub minus n layers beneath physics, like a simulation running our reality? This question resembles Derrida’s differance. Every layer defers to the next, and we cannot find the bottom²⁸¹. The Stack might stretch forever, or it might hit a wall. I don’t know.

Does this mean we are condemned to subjectivity? Solipsism? Intelligence is a whole system²⁸². It is the whole stack in motion. To understand intelligence we must rethink reality.

Our physical laws are models. They are programs we’ve written to predict nature’s machinery. We are the computers on which those programs run. Our tools are built for our slice of reality, not the whole pie²⁸³. There could be $f_{-\infty }$f sub minus infinity layers stacking down forever. I need a tool that doesn’t care. I need to identify what is true across every stack. Across every world. Anything less would be computational dualism all over again.

I’m going to propose a definition of environment that holds for every environment. It is the foundation of what I call Stack Theory²⁸⁴. It is a frame that holds no matter where the bottom lies. It’s not about finding $f_0$f zero. It is about sidestepping the need to know. Intuitively, we’re ants on a leaf, guessing at the tree. The only way to know about the tree is to work out what must be true of all trees our leaf might be attached to. Hutter’s Universal Artificial Intelligence was the right idea, but it made the wrong thing universal because it was hamstrung by computational dualism. We don’t need a universal description of intelligence, we need a universal description of the entire stack.

ALL POSSIBLE WORLDS

Here I lay the foundation of cognition within Stack Theory. Cognition within Stack Theory is enactive²⁸⁵ and pancomputational^{286 287}. Pancomputationalism says all physical systems are computational. Enactivism frames cognition as emerging from dynamic interactions between the system and its environment.

Stack Theory’s foundation is the environment. More specifically, it is a definition I name ‘environment’, but really it is what is common to all environments. All possibilities for an ‘underlying physics’. It is based on premises I refined over the course of several publications²⁸⁸. In those papers I called them axioms, but I’m no longer sure that description fits. They don’t depend on anything. They are not assumptions. In the first ‘axiom’ I merely define what I mean by environment. The second is a tautology.

• AXIOM 1: Where there are things, I call them the environment.
• AXIOM 2: If things change, then the environment has states.

What is a state? At the very least, it is a difference. If nothing changed then there could be no states. Without states the environment can only be some sort of unity or oneness. There is nothing in it we could point to. It just is. Perhaps even that is arguable. If something has no state and no content, is it anything? There must be difference for there to be something. Since there must be difference, there must be states. If one thing changes, then there must be two states. Before, and after. We don’t know what the thing is or what states are, and we don’t need to. That is unnecessary detail. All we need to know is that there is a difference between states. I don’t presuppose the environment is made up of objects or properties. Because of this there is a different, equivalent axiom we might use.

• ALTERNATIVE AXIOM 2: Time is difference.

Every state is point of difference is a different time in a particular timeline. This means states are mutually exclusive within a timeline.

Definition 1 (environment)

• We assume a set $\Phi$Phi whose elements we call states.
• A declarative program is $f\subseteq \Phi$f, a subset of Phi, and we write $P$ for the set of all declarative programs (the powerset of $\Phi$Phi).
• By a truth or fact about a state $\varphi$phi, we mean $f\in P$ such that $\varphi \in f$f in P such that phi is in f.
• By an aspect of a state $\varphi$phi we mean a set $l$ of facts about $\varphi$ s.t. $\varphi \in \bigcap l$l, such that phi is in the intersection of l. By an aspect of the environment we mean an aspect $l$ of any state, s.t. $\bigcap l\ne \varnothing$such that the intersection of l is not empty. We say an aspect of the environment is expressed, realised²⁸⁹ or embodied in state $\varphi$phi if it is an aspect of $\varphi$phi.

EVERYTHING THAT IS OR MIGHT BE MUST FALL WITHIN THE SCOPE of what this formalism can describe. Yes, my formalism is still an abstraction. However, some claims are so weak they are true of everything²⁹⁰

WE HAVE A SET OF STATES $\Phi$Phi. Each state $\varphi \in \Phi$phi in Phi represents a particular configuration of the environment at a given moment, capturing its current condition or state of affairs whatever that may be. The particulars don’t matter, just that each state represents a difference from other states. Non-equality. The power set $2^{\Phi }=P$two to the Phi, equals P of $\Phi$Phi, is all possible subsets of states, which I call declarative programs. Here, a declarative program is not a traditional algorithm but a subset of states. That program returns ‘true’ about states it contains.

A TRUTH OR FACT ABOUT A STATE $\varphi$phi is any program ($f$) that includes $\varphi$ , meaning ($f$) is true for that state. For the sake of intuitive example, if $\Phi =\{\text{on, off}\}$Phi equals the set containing on and off and $f=\{\text{on}\}$f equals the set containing on , then ($f$) is true for the state on . Now, this is a toy example. on and off are high level human abstractions. They are at the top of the stack, states are at the bottom of the stack, and the stack may be infinite. Nevertheless if we are somehow omniscient and given a state of the environment, then the sum total of everything that is true is the programs that contain that state. You can think of a true program as marking out points of sameness between states. If a program is true about two states, then that program is something they share in common. If a program is true of one state but not another, then it is a point of difference which separates them. Since each state represents a single point of difference, the set of all things which can be different or the same is the powerset of states $P$.

THE ENVIRONMENT ENCODES EVERYTHING through its state space. Whether objective or subjective every object, property, and goal is an aspect of the environment. An aspect of a state is a collection of facts

that all hold for that state, such as {light is on, door is closed} for a state where both are true. An aspect of the environment is one that holds for at least one state, and it is realised by a state if that state satisfies all the facts in the aspect. This formal structure allows us to model everything as programs, aligning with pancomputationalism’s view that all physical processes are computational²⁹¹.

TOY EXAMPLES

To demonstrate the framework’s generality, consider several examples from diverse domains. These illustrating how the environment can represent different systems. Now, in reality we don’t know what states contain. We see perceive them through a possibly infinite stack of abstraction layers. However, for the sake of example lets assume we are omniscient. This lets me use the framework to describe toy problems and ‘real world’ examples. In reality $\Phi$phi contains everything, but for the sake of example I pretend we have these very specific universes. Later, I will argue such things are abstraction layers but for now ignore that detail.

Light Switch System: Let $\Phi =\{\text{on},\text{off}\}$phi be the set containing on and off, the set of states for a light switch. Programs include $f_1=\{\text{on}\}$f one equals the set containing on (light is on) and $f_2=\{\text{off}\}$f two equals the set containing off (light is off). A fact about the state on is $f_1$f one, since $\text{on}\in f_1$on is an element of f one. An aspect could be $\{f_1\}$the set containing f one, realised by the state on. This simple example shows how even basic devices fit the framework, with states and programs defining goals.

Grid World in AI: In a grid world, $\Phi$phi is all possible positions of an agent and reward locations, e.g., $\Phi =\{(x,y,r_x,r_y)\mid x,y,r_x,r_y\in \{1,2,\dots ,n\}\}$phi is the set of tuples x, y, r x, r y, where each value is between one and n, where $(x,y)$x comma y is the agent’s position and $(r_x,r_y)$r x comma r y is the reward’s position. If we have a program $f=\{(x,y,r_x,r_y)\mid x=r_x\text{ and }y=r_y\}$f, defined as the set where x equals r x and y equals r y is true, then the agent is at the reward. Otherwise it is not. This aligns with reinforcement learning, where the agent interacts to achieve goals²⁹².

Biological Cell Metabolism: A cell’s environment includes metabolic states (e.g., healthy, stressed, dividing) and external conditions (e.g., nutrient levels). Let $\Phi$phi be the set of all such states, with programs like $f_1=\{\text{states where cell is healthy}\}$f one, the set of states where the cell is healthy or $f_2=\{\text{nutrient levels normal}\}$f two, the set where nutrient levels are normal. This illustrates that sometimes one program can be a subset of another, so $f_2\subset f_1$f two is a subset of f one. An aspect could be $\{f_1,f_2\}$the set containing f one and f two, realized by states where both hold.

These examples illustrate the framework’s flexibility, applying to digital, biological, and social systems, each with distinguishable states and goals.

V. TURTLES ALL THE WAY DOWN

So far I’ve framed the environment as a set of states $\Phi$phi. The contents of states are defined only by their differences from one another. These differences are formalised as programs, which are subsets of $\Phi$phi. Elements of the powerset $P=2^{\Phi }$P equals two to the power of phi. An aspect of the environment is a set of programs, and the aspect is realised or exists if it is true given a state²⁹³. It’s a minimalist setup that makes no assumptions. Yes, it is an abstraction but some abstractions are so weak they are true of everything. This particular abstraction holds for all possible environments. That is the point. Now I’m going to talk about embodiment.

Embodiment gets overlooked in computer science. That is why we have computational dualism. Anecdotally, when I have presented this research at conferences many of the questions I received straw-manned embodiment. The implication was that embodiment was a matter of sentimentality, or that I was arguing there is something non-computational about intelligence. After all, some proponents of enactive cognition believe that computation and true enactive cognition are incompatible²⁹⁴. However embodiment as I speak of it is just a fact of existence. Every body, whether it be human, machine, or a slab of granite, throws its weight around dictating what can happen next. A rock doesn’t care about your feelings, but drop it in a pond and the ripples tell a story. I am framing this as a kind of ontological “speech”. Not poetry, but a formal language baked into existence itself. Think of it as ontology with attitude. Entities say something by being what they are.

LAYER CAKE

The environment speaks in physical terms. Recall the definition of environment from the previous chapter:

• We assume a set $\Phi$phi whose elements we call states.
• A declarative program is $f\subseteq \Phi$f, a subset of phi, and we write $P$P for the set of all declarative programs (the powerset of $\Phi$phi).
• By a truth or fact about a state $\varphi$phi, we mean $f\in P$f in P such that $\varphi \in f$phi is in f.
• By an aspect of a state $\varphi$phi we mean a set $l$l of facts about $\varphi$phi s.t. $\varphi \in \bigcap l$phi is in the intersection of l.
  By an aspect of the environment we mean an aspect $l$l of any state, s.t. $\bigcap l\ne \varnothing$the intersection of l is not empty. We say an aspect of the environment is expressed, realised²⁹⁵ or embodied in state $\varphi$phi if it is an aspect of $\varphi$phi.

If every physical system computes²⁹⁶, then every physical system embodies a formal language. The environment is a physical system, so that means I should be able to re-frame it as a formal language. $P$P could be a vocabulary, and every aspect the environment a statement in this formal language. The set of all things the environment can say would then be the set of all aspects. Time is difference, so the only way we could have two different states at the same time would be if we had two different worlds. Seems rather like Everett’s interpretation of quantum physics²⁹⁷. Conversely, given a particular world there can only be one state at a time. That means aspects of the environment that are never realised by the same state never coexist in the same world. They are mutually exclusive. I could use that to build something like a logical nand gate. A nand gate $n\subset P$n, a subset of P would be an aspect of the environment, sure, but it is also more than that. Like the environment as a whole has a global state, a nand gate has its own local state. The different is a matter of detail. If $n$n is the aspect of the environment that is the nand gate in all its states, then $n$n is what does not change when the nand gate’s state changes. Each state of the nand gate is a more specific aspect of the environment than $n$n. If we want to formalise all these things together, then we need a subset $\mathfrak{v}$v of $P$P that contains the more specific aspects. I’ll give an example of this. First, I’ll define this formal language:

Definition 2 (abstraction layer)
By abstraction layer²⁹⁸ I mean:

• We single out a subset $\mathfrak{v}\subseteq P$v, a subset of P which we call the vocabulary of an abstraction layer. The vocabulary is finite unless explicitly stated otherwise. If $\mathfrak{v}=P$v equals P, then we say that there is no abstraction.
• $L_{\mathfrak{v}}=\{l\subseteq \mathfrak{v}:\bigcap l\ne \varnothing \}$L v, defined as the set of subsets l of v such that the intersection of l is not empty is a set of aspects in $\mathfrak{v}$v. We call $L_{\mathfrak{v}}$L v a formal language, and $l\in L_{\mathfrak{v}}$l in L v a statement.

• We say a statement is true given a state iff it is an aspect realised by that state.
• A completion of a statement $x$x is a statement $y$y which is a superset of $x$x. If $y$y is true, then $x$x is true.
• The extension of a statement $x\in L_v$x in L v is $E_x=\{y\in L_v:x\subseteq y\}$E sub x, defined as the set of all y in L v such that x is a subset of y. $E_x$E sub x is the set of all completions of $x$x.
• The extension of a set of statements $X\subseteq L_v$X, a subset of L v is $E_X={\bigcup }_{x\in X}{E}_x$E sub capital X, defined as the union of the extensions of its elements.
• We say $x$x and $y$y are equivalent iff $E_x=E_y$E sub x equals E sub y.

Our nand gate is embodied in silicon with inputs $a,b\in \{0,1\}$a and b in the set zero, one and output $c=\text{nand}(a,b)$²⁹⁹. For the sake of giving some clear intuition as to how this works I’m going to violate my own rule and write the states out as having specific contents rather than being contentless.

• states: $\Phi =001\cup 011\cup 101\cup 110\cup \neg \text{nand}$ where each value (e.g. $001$) denotes a set containing all states in $\Phi$Phi where $a,b$ and $c$ equal those values³⁰⁰, and $\neg \text{nand}$ contains all other states³⁰¹.
• vocabulary: $v=\{f_a,f_b,f_c\}$ s.t.
  • $f_a=101\cup 110$³⁰²
  • $f_b=011\cup 110$³⁰³
  • $f_c=001\cup 011\cup 101$³⁰⁴
• statements ($L_v$): Subsets $l\subseteq \{f_a,f_b,f_c\}$ with $\bigcap l\ne \varnothing$, e.g., $\{\},\{f_a\},\{f_c\},\{f_a,f_b\}$, but not $\{f_a,f_b,f_c\}$ ($\cap =\varnothing$).
• behaviour: $f_c=\Phi \setminus ((f_a\cap f_b)\cup \neg \text{nand})$, so $\{f_c\}$ is true iff $\{f_a,f_b\}$ is false and the gate is still operational.

Given a nand gate I can build a computer³⁰⁵. The nand is the basic building block of all computers today.

I return to Grid World for another illustrative example. Because we now have this formal definition of abstraction layer, we can consider how Grid World exists within our reality, rather than as a separate, simplified reality. In other words I assume there is a machine in the environment that computes Grid World. That machine is built out of nand gates that together form an abstraction layer for Grid World. Lets not worry about $\Phi$Phi now, because $\Phi$Phi is unknowable and infinite. We can only see our subjective abstraction layer. Grid World needs positions of an agent and reward locations³⁰⁶. Lets say $\text{positions}=\{(x,y,r_x,r_y)\mid x,y,r_x,r_y\in \{1,2,\dots ,n\}\}$positions, defined as the set of tuples x, y, r sub x, r sub y, where each coordinate is between 1 and n, where $(x,y)$ is the agent’s position and $(r_x,r_y)$ is the reward’s position. These are

just declarative programs, meaning positions $\subset P$a subset of P. To properly embody Grid World we also need programs like $g=\{(x,y,r_x,r_y)\mid x=r_x\text{ and }y=r_y\}$g, defined as the set of points x, y, r x, and r y, where x equals r x and y equals r y so that we can describe the goal state (when the agent is at the reward) and all possible actions in all possible orders $\text{actions}=\{{\text{up}}_1,{\text{up}}_2,{\text{left}}_1,{\text{right}}_1\dots \}$actions, defined as up one, up two, left one, right one, and so on. The end result is we need an machine (in this case made of nand gates) that physically embodies a vocabulary $\mathfrak{v}=\text{positions}\cup \text{actions}\cup \{g\}$v, defined as the union of positions, actions, and g, so that it can embody at least every state in Grid World. It could embody more, but that would consume resources that could be better spent on only what is relevant. After all, every computation has a physical cost³⁰⁷.

Every body carries a vocabulary, a subset $\mathfrak{v}\subseteq P$v, a subset of P of programs it can enact. Think muscle twitches, photon emissions, or gear shifts. In a computer, the vocabulary contains possible truths the system can physically encode and embody. A vocabulary is like a boundary of the system, at least in terms of its ability to process information as a coherent whole. For example, the programs in $\mathfrak{v}$v can describe every possible configuration of every possible bit in the system. Statements are subsets of $\mathfrak{v}$v that can hold together without clashing. For example, if $l\subset \mathfrak{v}$l is a subset of v is a statement then their intersection $\bigcap l\ne \varnothing$the intersection of l is not empty, meaning there exists a state where the statement is a true aspect of the environment. When the environment’s state $\varphi$phi hits that sweet spot, the statement is realised. It becomes a tangible fact carved into reality. Again, this distinction between subjective, semantic truth and existence is important because we’re trying to understand issues of consciousness. Delegating interpretation to the underlying states delegates the problem of interpretation to whatever the underlying physics of reality happen to be. It obviates the need for a translator and lets us ground symbols in a sense even DerridaJacques Derrida might accept³⁰⁸. A bent knee isn’t a stand-in for knee bent. It is knee bent. No middleman but the thing itself. The environment sets the rules and calls the shots. It cycles through states one at a time within the confines of a given world… or branching into many worlds³⁰⁹, either works but for the sake of explanation I will confine myself to one particular timeline. Each $\varphi \in \Phi$phi in capital Phi greenlights some programs while axing others. Existence evolves with every tick of time³¹⁰. Picture a robot clawing its way through a maze. Its vocabulary: sensor blips (wall close, path open) and motor grunts (pivot right, lurch forward). A statement like wall close, pivot left doesn’t fit in its embodied circuitry. It can’t turn left. It can’t represent left. It can turn right. The screech of its servos turning right as the sensor pings. That motion is the statement, alive in the grind of metal on floor³¹¹. No just the pondering, but the doing. This dodges old traps like Searle’sJohn Searle’s symbol-shuffling room³¹².

Bodies don’t represent reality. They are aspects of it.

SUBJECTIVE AND OBJECTIVE

When a body expresses a statement $l\in L_v$l in L sub v it filters the possibility space. Take a statement $x$x. Its extension $E_x$E sub x is the full roster of statements that imply it. If $x$x is the car is rolling, $E_x$E sub x might include the car is rolling downhill with the brakes shot. You might think of it as under-specification. A vague, weak statement (system’s active) maps to a swarm of more specific, stronger statements (gears turning, lights flashing). If two statements $x$x and $y$y share the same extension ($E_x=E_y$E sub x equals E sub y), they are implied by the same set of statements. Within an abstraction layer, this is like having the same truth conditions.

Now if I were omniscient, the environment would have one state at a time because time is difference. That state would determine what is true at that time. That would mean some programs would return true at that time, and I could know the rest to be false. Truth would be binary. The world deterministic. Everything that exists is a statement made in an environment’s embodied formal language, and which statements are true depends on the state.

But I am not omniscient. From my subjective perspective within my environment, I cannot know what the physical state is. I cannot see all the statements. I am a statement, and I exist for as long as the environment expresses me. The environment might be objectively deterministic, but from my subjective point of view it is non-deterministic. There are many possible futures. ‘Many worlds’ in which I may find myself, like Everett’s interpretation of quantum physics³¹³. Every statement $x$x has an extension $E_x$E sub x, which is the set of all statements in the language that imply the statement $x$x. These many possible worlds or futures are my extension.

By expressing a statement $x$x, the environment is constrained. It can only be in states that express $x$x. If the environment expresses both $x$x and $y$y then the possibilities are constrained even further, to the intersection of their extensions $E_x\cap E_y$E sub x intersect E sub y. In this sense, statements bump up against each other. They clash. Just as the same constraint can be realised by different systems³¹⁴, the same extension can be realised by different bodies or combinations of bodies. Intuitively, this reflects how the parts of a distributed, complex system interact. Upward and downward causation. For example assume the environment expresses cells. Those cells can interact to constrain each other’s behaviour and develop a collective identity³¹⁵.

Consider a human raising its arm. This sets in motion a particular future. When a body moves, it embodies a statement. A statement $(l)$l is expressed when the environment’s state $\varphi$phi lands in $\cap l$the intersection of l. The underlying physics may be the true interpreter, but within the confines of an abstraction layer we have access only to the programs. Objectively all programs are true or false, but from within the confines of an abstraction layer a program is subjectively true, false or unknown because the underlying state is unknown. Only the programs in the abstraction layer are accessible. This is fine. A rock rolling downhill isn’t pondering its path. It is merely interacting as part of a larger system. This is the loosest possible interpretation of computation. Just physics as the engine, no software required³¹⁶.

Each body has a vocabulary. A human is a chaotic symphony. A rock grunts single syllables. But each fits into the larger machine that is the environment. Computation here is the interaction of the body with its world. It affords the surround environment something³¹⁷. The world offers possibilities tailored to a body’s shape. A chair yells “sit” to a human, not to a boulder. The statements a body can pull off depend on what the environment hands it. This aligns with ideas like polycomputation, that a computation at one scale can perform an entirely different role as part of a computation at a larger scale³¹⁸. The same matter is part of many larger and smaller computations. This is a rejection of both the old computational mind³¹⁹, and strong enactivism that holds cognition to be non-computational³²⁰.

MATRYOSHKA DOLLS

A statement is a set of programs, but it is also equivalent to a program that has the same extension. Formally, I mean for every statement $x\subset P$x subset of P there exists a program $f\in P$f in P such that $E_x=E_{\{f\}}$the extension of x equals the extension of f. Hence I can map every statement’s extension to a set of equivalent programs. If I have a nand gate $n$n it has a very specific vocabulary as discussed earlier, but it can also be part of a larger system and thus have a much larger vocabulary. At most, it can be part of all the systems encompassed by its extension $E_n$E sub n. We can re-frame $n$n to an abstraction layer, much like how we treat Python as an abstraction layer over C. All we need to do is convert $E_n$E sub n to a set of equivalent programs, and we have the vocabulary of a new abstraction layer. A second order abstraction layer over the environment. I formalise this using an abstractor function:

Definition 3 (abstractor function) $\mathfrak{f}:2^P,2^P\to 2^P$f from 2 to the P, 2 to the P, to 2 to the P is an abstractor function that takes a vocabulary $\mathfrak{v}$v and a statement $l\subset \mathfrak{v}$l subset of v, and returns a new vocabulary ${\mathfrak{v}}^{\prime }=\{f\in P:\exists o\in E_l(\bigcap o=f)\}$v prime, defined as the set of programs f in P such that there exists an observation o in the extension of l where the intersection of o equals f.

Naturally I could also do this with a more constrained extension, taking into account other parts of the environment and how they constrain $n$n. I can take a vocabulary $\mathfrak{v}$v and form statements $x,y,z$x, y, and z. They can interact to give me the combined extension $E_x\cap E_y\cap E_z$the intersection of the extensions of x, y, and z implied by $x\cup y\cup z$x union y union z. From them I get a new vocabulary ${\mathfrak{v}}^{\prime }$v prime s.t. $\mathfrak{f}(\mathfrak{v},x\cup y\cup z)={\mathfrak{v}}^{\prime }$f of v and the union of x, y, z equals v prime. A higher level of abstraction. In this way, every statement the environment makes creates an abstraction layer. The outputs of the level below form the vocabulary of the level above. We go up a level of abstraction by looking at the 2nd order effects of the body we started with. An abstraction layer is like a smaller environment defined in the context of a larger environment. A ‘small world’ defined inside a ‘big world’³²¹.

CONCLUSION

The universeUNIVERSE is a firehose of information. No system within the universe can fully understand the universe unless we start making assumptions about iterated function system fractals. That would be interesting, but it isn’t what I’m doing here. The Bekenstein bound says bounded systems contain only finite information. A body is a bounded system, so a vocabularies are in general finite even if the universe isn’t. An abstraction layer picks the truths a body cares about and ignores the rest. I see this as a form of relevance realisation enforced by physics³²². A rock’s vocabulary says things like “I’m here” or “I’m falling”. A human is a sprawling mess from basics like run, grab and scream all the way through to divorce. These vocabularies are ontological rather than semantic. Concrete rather than abstract computations³²³. They are enacted by a particular body and interpreted by physics, rather than a person trying to reason out motives in the sense of Gricean meaning. Goertzel framed consciousness as a problem of moving from unary, to dyadic, to triadic relations³²⁴. A state is unary. A program is dyadic, in that it relates states to truth. By formalising an abstraction layer, we mimic the truth conditions of semantic structures using an ontological, unary foundation.

AnAN embodied language is governed by the rules etched into reality’s fabric. Each body has a formal grammar. At the core of this grammar lies the mutual exclusivity of states³²⁵. The logic of what can and cannot coexist. If I am omniscient then I can see the truth of every program unconstrained by any abstraction layer. Only one $\varphi$phi can hold sway at any moment in time, because time is difference. If two states could coexist then there would be programs which are both true and false. Hence, from an omniscient objective point of view things are true or false. Subjectively however, only the programs in one’s abstraction layer are true or false. All others are unknowable, and so the world appears non-deterministic. Under-determined. This leaves room for certain notions of free will and compatibilism³²⁶. Conversely there can only be one state at a time if everything is to be only true or false. Subjectively we don’t need to worry about states because we can only access the programs within the abstraction layer, and programs can be neither true or false. The point is that we have mutual exclusivity from an objective frame of reference, and this will give us the logical equivalent of nand. An aspect $l$l is true only if its programs can share a state, meaning $\cap l\ne \varnothing$the intersection of l is not empty. If programs within an aspect can’t coexist, then the aspect cannot exist.

VI. MASTER, WHAT IS MY PURPOSE?

This chapter is about purpose. It is based on the latter parts of my papers on abstraction layers³²⁷, tasks³²⁸ and consciousness³²⁹. What is normative? What ought to be? David Hume, fond of Guillotines³³⁰, said one cannot smuggle an ‘ought’ out of an ‘is.’ This leaves me in a pickle. If I am to build a conscious machine, presumably it must have a moral compass. Where do we anchor its sense of “should”? I could argue it is anchored it in satisfying homeostatic and reproductive needs, but then where do they come from? I’m a naturalist, not a vitalist. I need something more fundamental than mere life. Besides I want to explain life, not assume it. Hence I’m going to argue there is no “is”, only “ought”. Some things exist. Others do not. Is this a normative judgement? I say it is. What else can it possibly be? Ought stems from change, and change is time. Not just ticking away like some bored clock, but calling the shots on what sticks around and what gets yeeted into the void. Creation and destruction. It is not just about when but what lasts. Time sifts the wheat from the chaff, and what hangs on gets the cosmic thumbs-up.

Many have sought to patch the gap between is and ought. Some say ought is a matter of feeling (puts the cart before the horse), or social contract (arguably come from feelings), or divine memo (god did it). I find these lacking. Change seems more foundational. Fundamental, if anything is. Without change or difference, everything would be the same thing. If everything is the same, can you really say there is anything? Is there an environment if there are no things? I say no. There would be nothing. Just an irreducible oneness. It is hard to conceive of it as an internally consistent idea. To comprehend it, must I cease to exist as an observer? Becoming one with everything is beyond the scope of my thesis. Difference or change must be fundamental to existence, because without change it seems inconceivable that anything exists. Time is just the passage of this change.

Definition 4 (Time) Time is the ordered sequence of transitions between distinct states of the environment, where each state $\varphi \in \Phi$phi in Phi is a full snapshot of reality at a given tick.

Time is the process of becoming³³¹. Every tick of the cosmic clock is creation and destruction. Some aspects of the environment persist through many ticks of the clock.

Definition 5 (Persistence) An aspect $l$l persists across time if there’s a sequence of states ${\varphi }_1,{\varphi }_2,\dots ,{\varphi }_n$phi one, phi two, up to phi n where each ${\varphi }_i$phi i has a statement in $l$l’s extension $E_l$E sub l that’s expressed.

Persistence is survival. Darwin’s natural selection³³² on a universal scale. Stable atoms stick around because they vibe with physics³³³; critters adapt or get fossilized³³⁴. The universe is like a bouncer. Fit the rhythm and you stay. Clash with it and you’re out. New things are occasionally allowed in. This is the first whisper of “ought.” What persists is what is meant to, by the rules of the game.

THE ENVIRONMENT HAS AN OPINION

A state expresses some aspects, but not others. From the definition of environment, a statement $s$s is expressed, realised or embodied by state $\varphi$phi if all its programs are true in $\varphi$phi, i.e., $\varphi \in \bigcap s$phi is an element of the intersection of s. Something what ought to be. The environment, churning through time, picks winners and losers³³⁵. What sticks is the universe’s way of saying “I like this”. A sturdy molecule or a sneaky predator. The “ought not” pile is everything which doesn’t exist. Persistence over time sets the baseline for normativity³³⁶.

These statements form abstraction layers. Abstraction layers stack up like Matryoshka dolls, each layer refining the cosmic “ought” into sharper rules. From “thou shalt exist” at the base, we climb to “thou shalt compute efficiently” or “thou shalt not crash the system”³³⁷. Time, persistence, and expression give us this natural “ought”. Just the universe doing its thing³³⁸. For my conscious machine, this is the foundation.

PURPOSE

The fact of existence is a value judgement. Some things exist, and others do not. A rock doesn’t need to know physics to fall, but in doing so constrains what can happen next. It is an embodied ought that constrains what is, has been or ever will be. In this sense, every body is chattering away in a language forged by its form. When the state changes, some statements persist, and others are destroyed. This creates an incentive. The universe preserves that which preserves itself. Change is fundamental, and by its very nature change optimises for systems that cope with change, by deleting those that cannot. I want to formalise intelligence, which means I want to formalise a system that preserves itself. A living system.

A living, self preserving system is a statement $l$l made by the environment, and an abstraction layer. A self preserving system differs from other systems in that it exerts influence on the surrounding environment in order to preserve its own existence. If $l$l is a living organism, then some possible worlds end up with an organism dead or failing to reproduce. Not fit. There are more constraints on an organism’s possible worlds than just its body. It is embedded in the environment. The state of an organism’s nervous system is a statement in its embodied formal language. There is context to consider. Not all of the worlds in $E_l$E sub l are compatible with that context. Where another system would sit passively awaiting its fate, a self preserving system expresses additional statements to preserve its existence, actively imposing constraints on its own extension.

Therein lies the rub. Not everything serves homeostatic and reproductive goals. Not every possible world is a winner. Intelligent systems discriminate. Abstraction layers are biased toward some goals over others, but how exactly is that supposed to work? To really describe intelligence I need to formalise the idea of goals, but not in the abstract sense we humans are accustomed to. I can’t have goals separate from the systems that pursue them or I’m just going to end up with computational dualism again³³⁹. I need integrate goals with embodiment. A goal together with context and instructions is commonly known as a task. A task is what I use to formalise enactive cognition, in what I call Pancomputational Enactivism³⁴⁰${}^{,}$³⁴¹. A task is a formal description of a system in terms of its behaviour. That system is an abstraction layer, and the task is what it expresses. Outputs $O$O in the context of inputs $I$I. Typical computer science fare. I can choose any statement the body can make and call it an input.

The possible outputs are the extension $E_I$E sub I of the inputs $I$I. This makes sense because we have only so many possible worlds given the inputs. However not all the possible worlds are desirable³⁴², so the $O$O is a subset of $E_I$E sub I. This pairs inputs with the correct outputs. A body can be seen as a functional, computational system that maps inputs to outputs. Intuitively, these are the outputs that keep you breathing instead of bleeding out in a ditch.

Definition 6 (v-task)
For a chosen $v$v, a task $\alpha$alpha is a pair $⟨I_{\alpha },O_{\alpha }⟩$I sub alpha, O sub alpha where³⁴³:

• $I_{\alpha }\subset L_v$I sub alpha, a subset of L sub v is a set whose elements we call inputs of $\alpha$alpha.
• $O_{\alpha }\subset E_{I_{\alpha }}$O sub alpha, a subset of E sub I sub alpha is a set whose elements we call correct outputs of $\alpha$alpha.

$I_{\alpha }$I sub alpha has the extension $E_{I_{\alpha }}$E sub I sub alpha we call outputs, and $O_{\alpha }$O sub alpha are outputs deemed correct. ${\Gamma }_v$Gamma sub v is the set of all tasks given $v$v.

(generational hierarchy) A $v$v-task $\alpha$alpha is a child of $v$v-task $\omega$omega if $I_{\alpha }\subset I_{\omega }$I sub alpha is a subset of I sub omega and $O_{\alpha }\subseteq O_{\omega }$O sub alpha is a subset of or equal to O sub omega. This is written as $\alpha ⊏\omega$alpha is a child of omega. If $\alpha ⊏\omega$ then $\omega$ is then a parent of $\alpha$. $⊏$ implies a “lattice” or generational hierarchy of tasks. Formally, the level of a task $\alpha$ in this hierarchy is the largest $k$k such there is a sequence $⟨{\alpha }_0,{\alpha }_1,\dots {\alpha }_k⟩$alpha zero, alpha one, up to alpha k of $k$ tasks such that ${\alpha }_0=\alpha$ and ${\alpha }_i⊏{\alpha }_{i+1}$ for all $i\in (0,k)$. A child is always “lower level” than its parents³⁴⁴.

Tasks are like Matryoshka dolls. Little ones fit inside bigger ones. For example not choking on your coffee fits inside surviving the day. It’s a hierarchy. So how does your body pick the right output? Every statement your body makes constrains what can happen next. A policy is just a statement that constrains your outputs. A correct policy constrains you to correct outputs, given the additional constraint of the inputs. Correct policies keep you from face-planting. They steer you toward the outputs that don’t end in a Darwin Award. It works thusly:

Definition 7 (inference)

• A $v$v-task policy is a statement $\pi \in L_v$pi in L sub v. It constrains how we complete inputs.
• $\pi$ is a correct policy iff the correct outputs $O_{\alpha }$ of $\alpha$ are exactly the completions ${\pi }^{\prime }$ of $\pi$ such that ${\pi }^{\prime }$ is also a completion of an input.
• The set of all correct policies for a task $\alpha$ is denoted ${\Pi }_{\alpha }$.³⁴⁵

Assume $v$-task $\omega$ and a policy $\pi \in L_v$. Inference³⁴⁶ proceeds as follows:

1. we are presented with an input $i\in I_{\omega }$, and
2. we must select an output $e\in E_i\cap E_{\pi }$.

3. If $e\in O_{\omega }$e is in O omega, then $e$e is correct and the task “complete”. $\pi \in {\Pi }_{\omega }$pi in Pi omega implies $e\in O_{\omega }$e is in O omega, but $e\in O_{\omega }$e is in O omega doesn’t imply $\pi \in {\Pi }_{\omega }$pi in Pi omega (an incorrect policy can imply a correct output).

Mind and body are intimately connected³⁴⁷. But flesh or steel, the same constraints can be realised by wildly different systems³⁴⁸. Cellular automata show how simple rules birth complex life³⁴⁹. Reinforcement learning is basically evolution with better PR³⁵⁰. Am I saying bodies are just computers? No. Your mind is etched in meat, not silicon. Still, the vibes are the same. Inputs, outputs and constraints. The means of computation are less important than the resulting constraints. In that sense the ambulatory meme was correct when he said “wow, everything is computer”. Bodies are computational systems, tasks define the goals, and policies enforce the wins. It is a framework that scales from slime mould to Silicon Valley.

LEARNING THE STACK

Time does a frogmarch. Each step deletes something. Systems that stick around are those that avoid the jackboots of extinction. I’ll call this ‘fit’, but it is broader than the Darwinian notion³⁵¹. It applies even to non-living systems. Now, being ‘fit’ in this sense is hard-wired. Imposed from outside. Extrinsic. It does not require intelligence or agency. The jackbooted universe shapes matter into a form which is ‘fit’ by just deleting everything else. In that sense everything is an adaptation, which is a bit unsatisfying. This relentless change is an optimiser. Aspects “adapt” to whatever it is the environment has decided to optimise for. The extrinsic what that the environment optimises for is an uninstantiated task.

Definition 8 ($\lambda$lambda-tasks)
The set of all tasks with no abstraction (meaning $v=P$v equals P) is ${\Gamma }_P$Gamma sub P (it contains every task in every vocabulary). For every P-task $\rho \in {\Gamma }_P$rho in Gamma sub P there exists a function ${\lambda }_{\rho }:2^P\to {\Gamma }_P$lambda sub rho, mapping the power set of P to Gamma sub P that takes a vocabulary $v^{\prime }\in 2^P$v prime in the power set of P and returns a highest level child $\omega ⊏\rho$omega, a child of rho which is also a $v^{\prime }$v prime-task. We call ${\lambda }_{\rho }$lambda sub rho an uninstantiated-task, and ${\lambda }_1⊏{\lambda }_2$lambda one is a child of lambda two iff ${\lambda }_1(P)⊏{\lambda }_2(P)$lambda one of P is a child of lambda two of P.

This defines extrinsic, externally imposed purpose. It lets me consider purpose without pinning it to one vocabulary, so that we might compare embodiments. A $v$v-task does a good job describing hard-wired behaviour. For example, a simple reflex agent that responds to the world around it predictably, with preordained responses. Every behaviour is an adaptation baked into such organisms from birth. They cannot acquire new adaptations over the course of their lifetimes. If hard-wired adaptations are long term adaptations, then short term adaptations are those an organism learns. I mean adaptations acquired during a system’s existence, rather than baked into it from the start. Learning is an adaptation that facilitates adaptation. The ability to learn must be hard-wired into a system from its inception, but once a system can learn it can acquire new adaptations. A system which cannot learn has to store all of its policies from birth. That is inefficient, and limits how many tasks the system can complete. The alternative is an adaptation that allows a system to acquire new adaptations. To record and retrieve information to help them persist³⁵². A rock might “store” information about its past by having chunks knocked off it, but it does not then use that information to maintain its form. A rock does not pursue homeostasis. A living system maintains homeostasis. This means it optimises its internal and external world to maintain its form. Its integrity. Maintaining homeostasis is a very basic form of task. This seems to be where ‘learning’

begins. Learning requires a goal. We need something that defines ‘correct’ before a system can optimise for what is correct. Correct in general can be unrelated to homeostasis, but as I am trying to work from first principles I need to explain how we get to ‘correct’ in at least one case. Evolutionarily speaking that this is where we get a basic ought for the purpose of learning. A living system like a human can then build a computer that optimises for any arbitrary notion of correct³⁵³. A body embedded and extending into its environment is an abstraction layer. It can be constrained to ‘correct’ behaviour by expressing a policy $\pi$pi that constrains it to desirable possible worlds. Worlds in which conform to homeostatic goals. Such a system ceases to exist in other worlds³⁵⁴. A system learns by expressing a policy that constrains it to some arbitrary notion of correct (homeostatic or otherwise). A system which stores and retrieves information like a computer can express a policy by changing its internal state, and so can complete a far wider range of tasks than a system which cannot learn³⁵⁵.

Definition 9 (learning) Learning is a collection of definitions that describe the process by which a policy is constructed by any system³⁵⁶.

• A proxy $<$less than is a binary relation on statements, and the set of all proxies is $Q$Q.
• ${<}_w$less than w is the weakness proxy³⁵⁷. For statements $l_1,l_2$l one and l two we have $l_1{<}_w{l}_2$l one is weaker than l two iff $|E_{l_1}|<|E_{l_2}|$the cardinality of the extension of l one is less than that of l two.
• ${<}_d$less than d is the description length or simplicity proxy³⁵⁸. We have $l_1{<}_d{l}_2$l one is simpler than l two iff $|l_1|>|l_2|$the length of l one is greater than the length of l two.

(generalisation) A statement $l$l generalises to a $v$-task $\alpha$v task alpha iff $l\in {\Pi }_{\alpha }$l is in pi alpha. We speak of learning $\omega$omega from $\alpha$alpha iff, given a proxy $<$, $\pi \in {\Pi }_{\alpha }$a policy pi in pi alpha maximises $<$ relative to all other policies in ${\Pi }_{\alpha }$, and $\pi \in {\Pi }_{\omega }$.

(probability of generalisation) We assume a uniform distribution over ${\Gamma }_v$. If $l_1$ and $l_2$ are policies, we say it is less probable that $l_1$ generalizes than that $l_2$ generalizes, written $l_1{<}_g{l}_2$, iff, when a task $\alpha$ is chosen at random from ${\Gamma }_v$ (using a uniform distribution) then the probability that $l_1$ generalizes to $\alpha$ is less than the probability that $l_2$ generalizes to $\alpha$.

(efficiency) Suppose³⁵⁹ $\text{app}$ is the set of all pairs of policies. Assume a proxy $<$ returns 1 iff true, else 0. Proxy ${<}_a$ is more efficient than ${<}_b$ iff

(optimal proxy) There is no proxy more efficient than weakness. The weakness proxy formalises the idea that “explanations should be no more specific than necessary” (see Bennett’s razor in this ref³⁶⁰).

(intuitive summary) Learning is an activity undertaken by some manner of intelligent agent, and a task has been “learned” by an agent that knows a correct policy. Humans typically learn from “examples”. An example of a task is a correct output and input. A collection of examples is a child task, so “learning” is an attempt to generalise from a child to one of its parents. The lower level the child from which an agent generalises to parent, the “faster” it learns (it chooses policies that complete a wider variety of tasks, and thus are more sample and energy efficient choices), the more efficient the proxy. The most efficient proxy is weakness (see proofs 1 and 2, or these refs³⁶¹), which is why we’re using it here.

To learn, one must expressing a policy $\pi$pi that constrains future behaviour to desirable worlds. That generalises to future instances of a problem. If tasks are uniformly distributed, then the most effective way to learn is to maximise the number of tasks $\pi$pi completes. A proxy is a means of choosing between correct policies, in the hopes of selecting a policy that generalises to the relevant parent tasks. I give two proxies above, but there are others. In the next chapter I will explain why weakness is the optimal proxy, but for now don’t worry about that. It seems eminently reasonable to assume tasks are uniformly distributed. Anything else is an unnecessary assumption. A normative judgement beyond what is required. As we have already covered at length, the very fact of existence is a matter of ought. It is a sort of existential normativity. If a body exists and can store and retrieve information, then its representational capabilities are the product of that existential normativity. I’m not saying it isn’t theoretically conceivable some Lovecraftian horror reaches into an environment and changes the rules at a whim, but such a thing is already baked into this existential normativity. For the purpose of deciding which policies are optimal in general, it makes no difference why the state changes. It simply does.

Finally, I’ll integrate this idea of a task with The Stack I’ve been talking about. To do this, I’ll introduce the multilayer architecture (MLA). An abstractor function $f$f is applied here to policies, rather than just any statement.

Definition 10 (multilayer architecture) The multilayer architecture (MLA) found in both biological systems and computers. It integrates a stack with tasks to represent natural selection or ‘correctness’ at different layers of abstraction.

• The stack is represented here by a sequence of uninstantiated tasks $⟨{\lambda }^0,{\lambda }^1...{\lambda }^n⟩$lambda zero, lambda one, up to lambda n s.t ${\lambda }^{i+1}⊏{\lambda }^i$such that lambda i plus one is a sub-task of lambda i.
• $f$f is an abstractor function.
• The state of the MLA is a sequence of policies $⟨{\pi }^0,{\pi }^1...{\pi }^n⟩$pi zero through pi n and a sequence of vocabularies $⟨v^0,v^1...v^n⟩$v zero through v n such that $v^{i+1}=f(v^i,{\pi }^i)$v i plus one equals f of v i and pi i and ${\pi }^i\in {\Pi }_{{\lambda }^i(v^i)}$pi i is an element of the set of policies for task lambda i with vocabulary v i.

In the absence of abstraction where the system is seen as nothing more than the sum of its parts, the MLA is just a task ${\lambda }^0(v^0)$lambda zero of v zero, allowing us to look at the system across scales of distribution. We say the MLA is over-constrained when there exists $i<n$i less than n s.t. and ${\Pi }_{{\lambda }^i(v^i)}=\varnothing$the set of policies for the task at level i is empty, and multilayer-causal-learning (MCL) occurs when the MLA is not over-constrained and the proxy for learning is weakness. Note that the vocabulary is different at each level in the stack, which means each has its own generational hierarchy of tasks. By a higher level of abstraction, we mean a task higher in the stack (later in the causal chain).

Now, I won’t actually use this definition for a couple of chapters, but it is important to introduce it here for intuition. After all, I have been speaking endlessly about abstraction layers, and it would be strange to introduce goal directed behaviour within an abstraction layer without explaining how that goal directed behaviour propagates up and down the stack. As you can see in the above definition, the process of applying the abstractor function to obtain second, third or higher orders of behavioural effect is the same. However because it is applied to a policy, this is useful for formalising biological and other distributed goal directed systems. For example, say ${\alpha }_1$alpha one and ${\alpha }_2$alpha two are tasks representing the behaviour of two cells. Imagine those cells are both child tasks of $\alpha$alpha, which is an organ. A collective identity³⁶². They exist in the same abstraction layer. $\alpha$alpha is like looking at an organ as a collection of cells. However, if we move up a level of abstraction by taking the policy of $\alpha$alpha and applying the abstractor function, we are now looking at the organ, because ‘organ’ is a property we ascribe to the behaviour of cells. It is a second order of abstracted behaviour. I will discuss more about how this works in later chapters. For now, it is only important to note that $v$v-tasks are arranged in a stack, like abstraction layers. As we look higher in the stack, the goal directed behaviour gets narrower and more specific, because each successive layer is an effect of the goal directed behaviour in the layers below.

[blank page]

VII. WEAK

Human intelligence. It is sometimes hailed as the crowning achievement of evolution. But in the Darwinian arena, it’s not about being the smartest. It is about surviving long enough to pass on your genes. From a Darwinian perspective, intelligence is long-term adaptation that facilitates short-term adaptation during an organism’s lifetime. Long-term adaptation is like the genes we inherit. Short-term is what we learn as we go. If evolution is an optimiser, then biological intelligence is a mesa-optimiser (an optimiser within an optimiser)³⁶³. Without intelligence, a system would need all its knowledge pre-programmed. Like a robot with a fixed set of instructions. With intelligence a system can learn, adapt, and survive in a wider range of circumstances. It can complete a wider range of tasks³⁶⁴. A machine with intelligence can learn from its environment, adapt to new situations, and potentially develop something akin to consciousness. It’s not just about processing power. It is about the ability to change. Intelligence is the key to unlocking the door to consciousness. Without it, we’re just building a glorified abacus. This chapter is based primarily on my paper on weak versus simple hypotheses³⁶⁵, with some minor updates from later works³⁶⁶. I’ll show how adaptability is maximised by using weakness as the proxy, and I propose the following epistemological razor:

“Explanations should be no more specific than necessary.”³⁶⁷

INTELLIGENCE IN STACKISM

Intelligence is not about what something is, but what it does. I’ve framed everything as a stack of abstraction layers, each enacted by the one beneath. Like software on hardware, the layers go down. This is as true of human language as it is of human software. The cosmic ought determines functionality. Affordances³⁶⁸. It is why we have one abstraction layer instead of another. Intelligence isn’t about what an organism is but what it does. v-tasks. Subjected to inputs, a system produces outputs. Intelligence affords adaptation.

Tasks are interconnected, forming a generational hierarchy that reflects an organism’s temporal existence. Within this lattice, child tasks are specific cases of the broader parent tasks encompassing them. An organism’s past decisions is a child task of the task that includes every decision an organism might make over the course of its existence. The lattice structure links past and future behaviour. An example of a child task might be “take the succession of turns leading from home to office” and its parent could be “navigate the environment”. An organism’s survival depends on generalising from completed child tasks to meet the demands of the broader parent tasks that lie ahead.³⁶⁹ Thus, the generational hierarchy provides a dynamic framework for understanding intelligence as a process of bridging temporal scales.

POLICIES AND ADAPTATION

In the context of intelligence and adaptation, policies serve as the mechanisms that guide an organism’s behaviour towards survival. Policies can be innate or hard-wired, but that is less adaptable than being able to learn new through interaction with the environment. A learned policy is a learned statement or constraint that an organism embodies to ensure fit behaviour. It acts as a rulebook for survival, dictating how to respond to various inputs to achieve desirable outcomes. Assuming an organism remains alive and in some condition to procreate, its past behaviour is an ostensive definition of fit behaviour. Past behaviour is likely to imply at least some fit policies, and some unfit ones. The challenge of learning is to discern which policies will reliably constrain the organism to fit phenotypes. That complete the tasks of the past, and the widest range of possible future tasks³⁷⁰.

THE WEAK SHALL INHERIT THE WORK

A weaker policy is one that permits more possible behaviours while still being fit, akin to a versatile tool. Their lack of specificity allows them to address a wide range of future scenarios, enhancing adaptation. Weak policies are the key to generalisation. I’ll prove it. Recall from the definition of learning:

• A proxy $<$less than is a binary relation on statements, and the set of all proxies is $Q$Q.
• ${<}_w$the weakness proxy is the weakness proxy³⁷¹. For statements $l_1,l_2$l one and l two we have $l_1{<}_w{l}_2$l one is weaker than l two iff $|E_{l_1}|<|E_{l_2}|$the cardinality of the extension of l one is less than the cardinality of the extension of l two.
• ${<}_d$the simplicity proxy is the description length or simplicity proxy³⁷². We have $l_1{<}_d{l}_2$l one is simpler than l two iff $|l_1|>|l_2|$the length of l one is greater than the length of l two.

Now consider the follow two proofs:

Theorem 1 (sufficiency)

Assume $\alpha ⊏\omega$alpha is a child of omega. The weakness proxy sufficient to maximise the probability that a parent $\omega$omega is learned from a child $\alpha$alpha³⁷³.

Proof 1 You’re given the definition of $\upsilon$-taskupsilon task $\alpha$alpha from which you infer a hypothesis $\pi \in {\Pi }_{\alpha }$pi in the set of policies for alpha. To learn $\omega$omega, you need $\pi \in {\Pi }_{\omega }$pi in the set of policies for omega:

1. For every $\pi \in {\Pi }_{\alpha }$pi in the set of policies for alpha there exists a $\upsilon$-taskupsilon task ${\gamma }_{\pi }\in {\Gamma }_{\upsilon }$gamma pi in the set of tasks s.t. $O_{{\gamma }_{\pi }}=E_{\pi }$the outputs of gamma pi equal the extension of pi, meaning $\pi$ permits only correct outputs for that task regardless of input. We’ll call the highest level task ${\gamma }_{\pi }$ s.t. $O_{{\gamma }_{\pi }}=E_{\pi }$ the policy task of $\pi$.
2. $\omega$omega is either the policy task of a policy in ${\Pi }_{\alpha }$the set of policies for alpha, or a child thereof³⁷⁴.
3. If a policy $\pi$ is correct for a parent of $\omega$, then it is also correct for $\omega$. Hence we should choose $\pi$ that has a policy task with the largest number of children. As tasks are uniformly distributed, that will maximise the probability that $\omega$ is ${\gamma }_{\pi }$ or a child thereof.
4. For the purpose of this proof, we say one task is equivalent³⁷⁵ to another if it has the same correct outputs.
5. No two policies in ${\Pi }_{\alpha }$ have the same policy task³⁷⁶. This is because all the policies in ${\Pi }_{\alpha }$ are derived from the same set inputs, $I_{\alpha }$.
6. The set of statements which might be outputs addressing inputs in $I_{\omega }$ and not $I_{\alpha }$, is $\overline{E_{I_{\alpha }}}=\{l\in L_{\upsilon }:l\notin E_{I_{\alpha }}\}$³⁷⁷the complement of E I alpha, defined as the set of statements l in L upsilon such that l is not in E I alpha.
7. For any given $\pi \in {\Pi }_{\alpha }$, the extension $E_{\pi }$ of $\pi$ is the set of outputs $\pi$ implies. The subset of $E_{\pi }$ which fall outside the scope of what is required for the known task $\alpha$ is $\overline{E_{I_{\alpha }}}\cap E_{\pi }$³⁷⁸the intersection of the complement of E I alpha and E pi.
8. $L_{\upsilon }=E_{I_{\alpha }}\cup \overline{E_{I_{\alpha }}}$ and for all $\pi \in {\Pi }_{\alpha },E_{\pi }\subset L_{\upsilon }$. Apart from the inputs and correct outputs of $\alpha ,\overline{E_{I_{\alpha }}}$ contains only outputs which would be incorrect according to both $\alpha$ and $\omega$. Put another way, $E_{I_{\alpha }}\cap E_{\pi }=O_{\alpha }$ for every possible choice of $\pi$ in ${\Pi }_{\alpha }$. Hence the only way $|E_{\pi }|$ can increase is if $|\overline{E_{I_{\alpha }}}\cap E_{\pi }|$ increases. It follows that $|\overline{E_{I_{\alpha }}}\cap E_{\pi }|$ increases with $|E_{\pi }|$.

9. $2^{|E_{I_{\alpha }}\cap E_{\pi }|}$Two to the power of the size of the intersection of E sub I alpha and E sub pi is the number of non-equivalent parents of $\alpha$alpha to which $\pi$pi generalises. It increases monotonically with the weakness of $\pi$pi.

10. Given $\mathfrak{v}$-tasksv-tasks are uniformly distributed and ${\Pi }_{\alpha }\cap {\Pi }_{\omega }\ne \varnothing$the intersection of Pi alpha and Pi omega is not empty, the probability that $\pi \in {\Pi }_{\alpha }$pi in Pi alpha generalises to $\omega$omega is

$p(\pi \in {\Pi }_{\omega }\mid \pi \in {\Pi }_{\alpha },\alpha ⊏\omega )$The probability is maximised when $|E_{\pi }|$the size of E pi is maximised. Recall from definition 4 that ${<}_w$less than w is the weakness proxy. For statements $l_1,l_2$l one and l two we have $l_1{<}_w{l}_2$l one is weaker than l two iff $|E_{l_1}|<|E_{l_2}|$the size of E l one is less than the size of E l two. $\pi$pi that maximises ${<}_w$the weakness proxy will also maximise $p(\pi \in {\Pi }_{\omega }\mid \pi \in {\Pi }_{\alpha },\alpha ⊏\omega )$the probability. Hence the weakness proxy maximises the probability that³⁷⁹ a parent $\omega$omega is learned from a child $\alpha$alpha. $\square$

Theorem 2 (necessity)
To maximise the probability of learning $\omega$omega from $\alpha$alpha, it is necessary to use weakness as a proxy.

Proof 2 Let $\alpha$alpha and $\omega$omega be defined exactly as they were in proof 1.

1. If $\pi \in {\Pi }_{\alpha }$pi is in Pi alpha and $E_{I_{\omega }}\cap E_{\pi }=O_{\omega }$the intersection of E I omega and E pi equals O omega, then it must be he case that $O_{\omega }\subseteq E_{\pi }$O omega is a subset of E pi.
2. If $|E_{\pi }|<|O_{\omega }|$the size of E pi is less than the size of O omega then generalisation cannot occur, because that would mean that $O_{\omega }⊈E_{\pi }$O omega is not a subset of E pi.
3. Therefore generalisation is only possible if $|E_{\pi }|\ge |O_{\omega }|$the size of E pi is greater than or equal to the size of O omega, meaning a sufficiently weak hypothesis is necessary to generalise from child to parent.
4. For any two hypotheses ${\pi }_1$pi one and ${\pi }_2$pi two, if $|E_{{\pi }_1}|<|E_{{\pi }_2}|$the size of E pi one is less than the size of E pi two then the probability $p(|E_{{\pi }_1}|\ge |O_{\omega }|)<p(|E_{{\pi }_2}|\ge |O_{\omega }|)$the probability for E pi one is less than the probability for E pi two because tasks are uniformly distributed.
5. Hence the probability that $|E_m|\ge |O_{\omega }|$the size of E m is greater than or equal to the size of O omega is maximised when $|E_m|$the size of E m is maximised. To maximise the probability of learning $\omega$omega from $\alpha$alpha, it is necessary to select the weakest hypothesis.

To select the weakest hypothesis, it is necessary to use the weakness proxy. $\square$

Weak policies are thus essential, boosting both the sample and energy efficiency of adaptation.

META-APPROACHES

Ockham’s Razor favors simplicity. Earlier I described AGI in terms of tools (search and approximation) and meta-approaches (scale-maxing, simp-maxing and w-maxing). Ockham’s Razor is an example of simp-maxing (simplicity maximisation). My razor prioritises generality, advocating for broad yet effective explanations³⁸⁰. It promotes weak policies, enhancing adaptability by avoiding over-specification, thus improving the speed and efficiency of adaptation to varied scenarios. I call this w-maxing. Rules and explanations should remain as general as possible while fulfilling their purpose³⁸¹.

So to reiterate we now have three meta-approaches: simp-maxing, scale-maxing and w-maxing. W-maxing is the meta-approach I propose, achieved in part by using weakness as the proxy. Of course, how much you can w-max depends on the abstraction layer. Simp-maxing is likewise formalised using a proxy. Finally, scale maxing involves maximising the resources available, which formally would involve just increasing the size of the vocabulary.

INTUITION

For intuition on w-maxing, consider The Contravariance Principle³⁸². The contra-variance principle says that we are more likely to converge on the true underlying model the more we scale up the data and subsequently the apparent “difficulty” of a task (here in the informal sense). If we keep resources finite, then there are fewer and fewer possible explanations of the given data as we scale up difficulty. Scale up difficulty enough, and there will only be one possible explanation. This is useful because it tells us we can scale up data to get models to converge to the true process that generated the data, assuming of course that it can be represented given the system³⁸³. This contravariance principle appears to be like The Inventor’s Paradox, which says it can be easier to solve a more general problem that includes $A$A, than a $A$A alone. A solution to a more general problem is, of course, weaker. It completes a wider range of tasks. I can formalise the contravariance principle using tasks. In a proof earlier in this chapter I talk about a policy task. For every $\pi \in {\Pi }_{\alpha }$pi in Pi sub alpha there exists a v-task ${\gamma }_{\pi }\in {\Gamma }_v$gamma sub pi in Gamma sub v s.t. $O_{{\gamma }_{\pi }}=E_{\pi }$O gamma sub pi equals E sub pi, meaning $\pi$pi permits only correct outputs for that task regardless of input. The highest level task ${\gamma }_{\pi }$gamma sub pi s.t. $O_{{\gamma }_{\pi }}=E_{\pi }$O gamma sub pi equals E sub pi it the policy task of $\pi$pi. Now, if I increase the “difficulty” of the task by scaling up the data until there is only one solution, what I have done is construct a policy task for a weakest policy in the set. The contravariance principle amounts to the idea that one should scale up data until the weakest policy is the only one left. Now, given my result, we can skip the bit where we scale up the data and just choose a weakest policy. Same result, less work involved.

Before I move on, some have questioned whether my framework struggles with multiclass classification³⁸⁴. It’s fine, no need to worry. Yes, a task frames problems as one-class classification. However, two-class classification problems are a subset of one-class. As Simmons points out in his reply to one of my papers, you can just include an extra declarative program to discriminate between classes³⁸⁵. Say we have a two class classification problem $\alpha$alpha. It is in reality a parent task of two one-class classification tasks ${\alpha }_1$alpha one and ${\alpha }_2$alpha two. $O_{{\alpha }_1}\cup O_{{\alpha }_2}=O_{\alpha }$O sub alpha one union O sub alpha two equals O sub alpha. $O_{{\alpha }_1}$O sub alpha one is one correct class classification for outputs, and $O_{{\alpha }_2}$O sub alpha two is the other. Any correct policy for $\alpha$alpha much correctly classify both ${\alpha }_1$alpha one and ${\alpha }_2$alpha two. If we want to deal with continuous values, we just have an infinite class classification problem. Of course, no physical system actually implements a continuous value because that would require infinitely accurate read/write memory, and if we had those we could violate the Bekenstein bound³⁸⁶ and store infinite infor-

mation. But I digress. The important thing is my simple one-class classification set up can address any sort of task. Everything, in the end, amounts to an instance of one-class classification. By adding more details (subclasses etc) we just make learning and inference easier. Consider a task with $k$k classes, where each class offers distinct output constraints. The proof below formally shows that increasing the number of classes reduces the complexity of learning.

SIMP-MAXING WILL NOT SAVE YOU

Here I compare w-maxing and simp-maxing. I’ve a simple proof and some experiments that show the former destroys the latter, based on my paper on weak vs simple hypotheses³⁸⁷. Simp-maxing is a matter of form. Whether one thing has a shorter minimum description length than another depends on the language. It is like a ruler that changes length. W-maxing is about function. How loose is your policy’s leash. How many tasks it completes. Intuitively it is a bit like the difference between a Swiss Army knife and a scalpel. One is simple, sure, but the other is ready for anything. From the previous chapter we already know that w-maxing is necessary and sufficient to maximise adaptability. What we don’t know yet is whether we still need to simp-max, and to what extent there’s a difference. When I initially concieved of this framework, I actually thought that w-maxing would amount to simp-maxing. I was astounded when my experiments showed w-maxing far outperforming simp-maxing. So here I’ll prove you don’t need to simp for simplicity to win. I’m not saying it won’t help if you can’t w-max, but w-maxing is all that’s necessary for adaptation.

UPPER BOUND

Simp-maxers like AIXI³⁸⁸ hinge on complexity. They argue intelligence boils down to finding the simplest model that fits the data. As has already been pointed out, simplicity hinges on the choice of Universal Turing Machine (UTM)³⁸⁹. Pick the wrong one and the theoretical superintelligent god is reduced to a bag of hammers. Even so, it helps to be formal. How does this work in my formalism?

Theorem 3 (simplicity sub-optimality)
Description length is neither a necessary nor sufficient proxy for the purposes of maximising the probability that induction generalises.

Proof 3 In proofs 1 and 2 we proved that weakness is a necessary and sufficient choice of proxy to maximise the probability of generalisation. It follows that either maximising $\frac{1}{|m|}$one over the magnitude of m (minimising description length) maximises $|E_m|$the weakness E m (weakness), or minimisation of description length is unnecessary to maximise the probability of generalisation. Assume the former, and we’ll construct a counterexample with $\mathfrak{v}=\{a,b,c,d,e,f,g,h,j,k,z\}$v, defined as the set of elements a through z s.t. $L_{\mathfrak{v}}=\{\{a,b,c,d,j,k,z\},\{e,b,c,d,k\},\{a,f,c,d,j\},\{e,b,g,d,j,k,z\},\{a,f,c,h,j,k\},\{e,f,g,h,j,k\}\}$ and a task $\alpha$alpha where

• $I_{\alpha }=\{\{a,b\},\{e,b\}\}$I alpha equals the sets a b, and e b
• $O_{\alpha }=\{\{a,b,c,d,j,k,z\},\{e,b,g,d,j,k,z\}\}$O alpha consists of two specific sequences
• ${\Pi }_{\alpha }=\{\{z\},\{j,k\}\}$Pi alpha contains the sets z, and j k

Weakness as a proxy selects $\{j,k\}$, while description length as a proxy selects $\{z\}$. This demonstrates the minimising description length does not necessarily maximise weakness, and maximising weakness does not minimise description length. As weakness is necessary and sufficient to maximise the probability of generalisation, it follows that minimising description length is neither. $\square$

As you can see, it is possible to w-max without simp-maxing. In fact, they can be at odds. Given w-maxing is necessary and sufficient for generalisation, this bodes poorly for simp-maxing. It also raises questions about why simp-maxing works at all which I’ll get into in later chapters. That later chapter is kinder to the idea. For now, I can use this fact to establish an alternative to AIXI as an upper bound on intelligence. Arguably AIXI’s greatest contribution is to provide an ideal for which we can optimise. Even if it is based on a flawed premise, it is much better than nothing. This idea is an upper bound on intelligent behaviour. The flaw is that it is an upper bound on disembodied, software intelligence. The alternative is to establish an

upper bound on embodied intelligence. Now it is obvious from the earlier proofs that the upper bound on intelligent behaviour within the confines of an abstraction layer is achieved by simply w-maxing. Fine. But in general? AIXI gives an upper bound across all possible environments. For embodied intelligence, that is like an upper bound across all possible abstraction layers. To formalise that I’ll need some extra steps. Recall the definition of uninstantiated task from earlier:

• The set of all tasks with no abstraction (meaning $v=P$v equals P) is ${\Gamma }_P$Gamma sub P (it contains every task in every vocabulary). For every $P$-taskP task $\rho \in {\Gamma }_P$rho in Gamma sub P there exists a function ${\lambda }_{\rho }:2^P\to {\Gamma }_P$lambda rho from the power set of P to Gamma sub P that takes a vocabulary $v^{\prime }\in 2^P$v prime in the power set of P and returns a highest level child $\omega ⊏\rho$omega, a child of rho which is also a $v^{\prime }$-taskv prime task. We call ${\lambda }_{\rho }$ an uninstantiated-task, and ${\lambda }_1⊏{\lambda }_2$lambda one is a child of lambda two iff ${\lambda }_1(P)⊏{\lambda }_2(P)$if and only if lambda one of P is a child of lambda two of P.

I’ll now use this definition to establish the utility of w-maxing given an uninstantiated task and a vocabulary.

Definition 11 (utility of intelligence)
Every task $\gamma \in \Gamma$gamma in Gamma has a “utility of intelligence”³⁹⁰ computed as $ϵ:\Gamma \to \mathbb{N}$epsilon from Gamma to the natural numbers such that $ϵ(\gamma )={\max }_{m\in {\Pi }_{\gamma }}(|E_m|-|O_{\gamma }|)$. Maximisation of utility means maximising $\bullet \stackrel{ϵ}{<}\bullet$ that returns true iff $ϵ(\alpha )<ϵ(\omega )$.

This tells me the difference between w-maxing and not. Obviously utility is minimised by having a task with one policy. Likewise, there’s no utility if there is no correct policy, so predicting noise is out. However, in the absence of any abstraction there is always a correct policy. Utility is maximised when $v=P$v equals P, though in practice finite resources would limit us to smaller vocabularies. ${\Gamma }_P$Gamma sub P contains all tasks in all vocabularies. Hence, for every task $\rho$rho in ${\Gamma }_P$Gamma sub P we can define a function that takes a vocabulary $v$v and returns a $v$-taskv task which is a child of $\rho$rho. For this proof, I’ll set the vocabulary $v$v to $P$P and work “inwards” to more constrained abstraction layers.

Theorem 4 (upper bound) The most ‘intelligent’ choice of policy and vocabulary given uninstantiated task ${\lambda }_{\rho }$ is $\pi$ and $v$ s.t. $v$ maximises utility for ${\lambda }_{\rho }(v)$, $\pi \in {\Pi }_{{\lambda }_{\rho }(v)}$ and $\pi$ maximises weakness.

Proof 4 We have equated intelligence with sample efficient generalisation. The weakest correct policies have the highest probability of generalising. Given an uninstantiated task ${\lambda }_{\rho }$lambda rho, utility measures the weakness of the weakest correct policies. We can use this to compare vocabularies. By choosing a vocabulary $v$v which maximises utility for ${\lambda }_{\rho }(v)$lambda rho of v, we instantiate ${\lambda }_{\rho }$lambda rho in a vocabulary that maximises the weakness of correct policies for ${\lambda }_{\rho }$lambda rho even in the absence of abstraction (meaning when $v=P$v equals P). Then, using weakness proxy, we can select a policy that has the highest possible probability of generalising, and thus maximise sample efficiency. □

Now, this isn’t to say that w-maxing is always going to get you there given any old abstraction layer, but it works at least from an objective point of view where $\mathfrak{v}=P$³⁹¹v equals P. Generally speaking, the best way to optimise toward intelligent systems is to build abstraction layers that enable the system to embodied the weakest correct policies relevant to the task.

EXPERIMENTS

In the appendix is a Python script to perform two experiments using PyTorch with CUDA, SymPy and $A^{\ast }$³⁹²A star. Here I’ll summarise the results and how the experiments worked. First, I wrote a toy program based on $A^{\ast }$A star that learns policies for 8-bit string prediction tasks (binary addition and multiplication)³⁹³.

(abstraction layer) I used a simplified environment of 256 states, one for every possible 8-bit string. Basically an abstraction layer without our environment. The statements in $L$L were expressions regarding those 8 bits that could be written in propositional logic ($\neg$, $\wedge$ and $\vee$)not, and, and or.

(task) A task was specified by choosing $O\subset L$O as a subset of L such that all $d\in O$d in O conformed to the rules of either binary addition (for the first experiment) or multiplication (for the second experiment) with 4-bits of input, followed by 4-bits of output.

Each of the two experiments (addition and multiplication) involved repeated trials (sampling results). The parameters of each trial were “operation” (a function), and an even integer “number_of_trials” between 4 and 14 which determined the cardinality of the set $O_k$O sub k (defined below). Each trial was divided into training and testing phases.

(training phase)

1. A task $T_n$T sub n was generated:
   (a) First, every possible 4-bit input for the chosen binary operation was used to generate an 8-bit string. These 16 strings then formed $O_n$O sub n.
   (b) A bit between 0 and 7 was then chosen, and $I_n$I sub n created by cloning $O_n$O sub n and deleting the chosen bit from every string (meaning $I_n$I sub n was composed of 16 different 7-bit strings, each of which could be found in an 8-bit string in $O_n$O sub n).
2. A child-task $T_k=⟨I_k,O_k⟩$T sub k, defined as the pair I sub k and O sub k was sampled from the parent task $T_n$T sub n. Recall, $|O_k|$the size of O sub k was determined as a parameter of the trial.
3. From $T_k$T sub k two policies (formerly known as models) were generated; a weakest $c_w$c w, and a MDL $c_{mdl}$c M D L.

(testing phase) For each policy $c\in \{c_w,c_{mdl}\}$c in the set containing c w and c M D L:

1. The extension $E_c$E sub c of $c$c was then generated.
2. A prediction $O_{recon}$O recon was then constructed s.t. $O_{recon}=\{e\in E_c:\exists s\in I_n(s\subset z)\}$O recon is the set of elements e in E sub c such that there exists s in I sub n where s is a subset of z.

Figure 1: Rates for binary addition (solid) and mult. (dashed).

3. $O_{\text{recon}}$O recon was then compared to the ground truth $O_n$O n, and results recorded.

Between 75 and 256 trials were run for each value of the parameter $|O_k|$the size of O k. Fewer trials were run for larger values of $|O_k|$the size of O k due to restricted availability of hardware. The results of these trails were then averaged for each value of $|O_k|$the size of O k.

(MEASUREMENTS) Generalisation was deemed to have occurred where $O_{\text{recon}}=O_n$O recon equals O n. The number of trials in which generalisation occurred was measured, and divided by $n$n to obtain the rate of generalisation for $c_w$c w and $c_{mdl}$c m d l. Error was computed as a Wald 95% confidence interval. Even where $O_{\text{recon}}\ne O_n$O recon does not equal O n, the extent to which policies generalised could be ascertained. $\frac{|O_{\text{recon}}\cap O_n|}{|O_n|}$The size of the intersection of O recon and O n, divided by the size of O n was measured and averaged for each value of $|O_k|$the size of O k, and the standard error computed.

Table 1: Binary addition.

Table 2: Binary multiplication.

THE KING IS DEAD, LONG LIVE THE KING!

This has profound implications. Simp-maxing is treated with great reverence in many circles. When I have presented these results at conferences, reactions have varied wildly. This challenges the foundations of approaches to AI based on simp-maxing, and by extension information theory and thermodynamics. I have listed the assumptions, the proofs and the experimental results. It is surprising, but it seems to check out. Human cognition is about breadth, analogy and the minimising surprise³⁹⁴. I am not disagreeing with those fundamental ideas, but with complexity’s role in them. To effectively w-max, all of the above is likely to be necessary. W-maxing is a meta-approach after all. However, it seems hasty to completely dismiss simp-maxing. The next chapter will explore its significance.

VIII. STACKISM

Why does simplicity of form correlate with generalisation in function? I’ve already shown it isn’t necessary. Yet the subjective perception of simplicity remains stubbornly correlated with the objective generalisation of functionality. Why and how are subjective and objective related here? What causes this? Is there somehow an objective notion of complexity? Is this hard-coded into us by evolution? How? This chapter is based on my papers on complexity³⁹⁵, and on abstraction layers³⁹⁶. I’ll begin with why simpler forms are correlated with generalisation in function. Second, I’ll dissect how systems come to embody this correlation. I’ll map systems along dimensions of abstraction, distribution of work and delegation of control. This allows me to compare systems. Third, I’ll compare biological self-organisation to AI and conclude that biology is more adaptable because it delegates control to lower levels of abstraction. This causes weak constraints to be embodied by simple forms. This means w-maxing requires delegating control, and w-maxing while delegating control will cause simp-maxing. However, simp-maxing and delegating control will not cause w-maxing. In my papers I also discuss what this means for systems in general, exemplifying my points with human organisational and economic structures. I’ll touch on these in the conclusion of the thesis.

WHY SIMP-MAXING SORT OF WORKS?

Empirically, there is a correlation between simplicity of form and generalisation in function. I want to know why. I’ve established that the probability of generalisation and measure of weakness go hand in hand. Recall how I’ve defined simplicity and weakness.

• A proxy $<$ is a binary relation on statements, and the set of all proxies is $Q$Q.
• ${<}_w$ is the weakness proxy³⁹⁷. For statements $l_1,l_2$l one and l two we have $l_1{<}_w{l}_2$ iff $|E_{l_1}|<|E_{l_2}|$.
• ${<}_d$ is the description length or simplicity proxy³⁹⁸. We have $l_1{<}_d{l}_2$ iff $|l_1|>|l_2|$.

As the proof on the upper bound shows, there is a meaningful objective notion of weakness in the absence of an abstraction layer³⁹⁹. Is there a similarly meaningful notion of complexity in the absence of abstraction? By this, I mean does knowing something’s simplicity convey any information? Ideas like Legg-Hutter intelligence seem to suggest that there is an objective notion of simplicity. That the environment considers some things complex, and some simple. When Leike and Hutter found AIXI to be subjective, they still claimed it is optimal if the UTM AIXI uses matches the UTM with respect to which Legg-Hutter intelligence is measured. The latter UTM is like the objective measure of complexity, and AIXI’s UTM is the subjective. Their argument was effectively that if objective and subjective complexity match, then AIXI’s performance is optimal. I’ll now refute this claim indirectly, by showing there’s no such thing as objective complexity.

Theorem 5 (subjectivity) If there is no abstraction, complexity can always be minimized without improving sample efficiency, regardless of the task.

Proof 5 In accord with the definition of an abstraction layer, the absence of abstraction means the vocabulary is the set of all declarative programs, meaning $\mathfrak{v}=P$v equals P. It follows that for every $l\in L_{\mathfrak{v}}$l in L sub v there exists $f\in \mathfrak{v}$f in v such that $\bigcap l=f$. Statements $l$ and $\{f\}$ are equivalent iff $E_l=E_{\{f\}}$, which is exactly the case here because $\bigcap l=f$. Theorems 1 and 2 show that maximising weakness is necessary and sufficient to maximise the probability of generalisation, which means weakness maximises sample efficiency (is the optimal proxy). This means sample efficiency is determined by the cardinality of extension. For every correct policy $l$ of every task in ${\Gamma }_{\mathfrak{v}}$Gamma sub v there exists $f\in \mathfrak{v}$f in v s.t. $E_l=E_{\{f\}}$. Policy complexity can be minimised regardless of weakness, because the simplest representation of every extension is a set containing exactly one program. $\square$

When we don’t have abstraction, everything is equally complex. Simplicity becomes meaningless. Complexity is an illusion perpetrated by abstraction layers. So no, there is no objective notion of simplicity. Yet the fact remains the the subjective perception of complexity is correlated with generalisation in function. How can this be? The key lies in the fact that, in reality, we never have the absence of abstraction. In fact, we never even have infinite vocabularies. If we did, we could store an infinite amount of information on a computer. The Bekenstein bound says that a bounded system can store only a finite amount of information. That means there are only so many permutations. Only so much information needed to specify the system’s state. That is why vocabularies are finite. Can this be the cause of the correlation between simplicity of form and generalisation? We know that weak constraints on function are necessary and sufficient for generalisation, so the question can be reframed as “can a finite vocabulary correlate simple forms with weak constraints?”. Yes, it can.

Theorem 6 (confounding) If the vocabulary is finite, then policy weakness can confound⁴⁰⁰sample efficiency with policy simplicity.

Proof 6 We already have that policy weakness causes sample efficiency, in that it is necessary and sufficient to maximise it in order to maximise sample efficiency. Continuing from proof 1, in a finite vocabulary, there may not exist $f\in \mathfrak{v}$f in v s.t. $E_l=E_{\{f\}}$E l equals E of the set containing f, which means the complexity of all extensions will not be the same. If we choose any vocabulary in which weaker aspects take simpler forms, then simplicity will be correlated with weakness and so will also be correlated with sample efficiency. This means we would choose $\mathfrak{v}$v s.t. for all $a,b\in L_{\mathfrak{v}}$a and b in L sub v, the simpler statement has the larger extension, meaning $a{<}_{w}b\leftrightarrow a{<}_{d}b$a is weaker than b if and only if a is simpler than b. For example, suppose $P=\{a,b,c\dots \},a=\{1,2,4\},b=\{1,3,4\},\mathfrak{v}=\{a,b\},L_{\mathfrak{v}}=\{\{a\},\{b\},\{a,b\}\}$P, a, b, v, and L sub v are defined as sets, then it follows $\{a,b\}{<}_w\{a\},\{a,b\}{<}_w\{b\},\{a,b\}{<}_d\{a\},\{a,b\}{<}_d\{b\}$the set a b is weaker and simpler than both the set a and the set b. $\square$

In causal language, simplicity does not cause generalisation. Weak constraints on function cause generalisation, but if the vocabulary is finite weakness can also cause simplicity. Weakness confounds simplicity and generalisation. The Bekenstein bound slams a ceiling on how much information fits in any finite space. A sort of cosmic storage cap. Nothing overlaps. Two things can’t share the same spot at once. If we put too much in one spot we get a singularity, but not the good sort. That caps our vocabularies. From atoms to molecules to cells, at each layer of abstraction we are stuck with a finite set of terms. Intuitively, imagine you have to pick 100 words with which to construct a language. You would not pick 100 very specific words, like lantern or erudite. Instead, you’d pick words that are used in more circumstances. That have as many meanings as possible, like “swallow” which is both a bird and an action. You would pick “hot” and “wet” rather than “sultry”. “Look” instead of “gaze”. More precise words have more limited uses.

In mathematical terms think of Huffman coding , whereby the shortest codes are assigned to the most common messages⁴⁰¹. Compression becomes meaningful when space is limited. With a finite vocabulary, simplicity and weakness have to become correlated if the system is to work well. Complex policies become one-trick ponies. Waste space on overly-specific policies, and the system will fail to adapt.

SYSTEMS

I’ve established that weakness can confound simplicity and generalisation, but I’ve not explained what sort of system can actually do this. Computers are the way they are because we build them that way. We embody highly abstract human behaviours in silicon. Human mathematics and logical operations normally depend on a human to interpret them. What we have done with computers is hard-wire physical consequences into instruction set architectures, and those consequences conform to our expectations and interpretations of logic. However it came about, we have a natural tendency to embody weak constraints in simple forms. We do it in natural language, and we do it with programming languages. At least this is the case within the narrow confines of the problem sets to which we apply computers. The result is that shorter programs do tend to be more generalisable. It isn’t perfect, but it is empirically observable. How is it we came to have this tendency? Yes, natural selection prefers week constraints take simple forms, but what is this system natural selection has built as a consequence of that?

There is a good reason to ask this question, beyond my usual curiosity. The evidence suggests machine learning systems are far less sample and energy efficient than biological systems that learn the same tasks⁴⁰². There must be more going on than just search or approximation. Biology must be using a meta-approach that makes it more efficient. Sample and energy efficiency mean w-maxing. Biology is clearly very good at w-maxing. How? What is it that biological self-organising systems do differently?

There are fundamental constraints to the logic of living systems, particularly those that learn⁴⁰³. Most important for my purposes is the ability to store information in an internal state in order to retrieve it facilitate adaptation in future. The ability to learn, rather than have information hard coded. Put another way, a living system is a policy because it is an aspect of the environment. However that policy implies an abstraction layer, and that abstraction layer can then express a policy. A living, learning system is a policy that expresses yet another policy at a higher level of abstraction. That higher level of abstraction is the behaviour of the lower level. When I think of a plan, that plan is a policy I express in my neural substrate. It is the behaviour of my nervous system. This is upward causation from the body to the mental representation of the plan. When I act on that plan, I delegate it to my body. There is downward causation from the plan to my body, to my organs and eventually my cells⁴⁰⁴. A basic

feature of any adaptive system is valence, by which I mean they are attracted to or repelled away from physical states⁴⁰⁵. A cell in isolation might have very simple dichotomy of attraction and repulsion. However cells can network. They can support a bioelectric information structure⁴⁰⁶. What is a simple dichotomy of one-dimensional valence at the level of a single cell is part of a rich tapestry of valence across the network. The same matter is simultaneously involved in many different computations occurring at different scales and levels of abstraction. This is called “polycomputing”⁴⁰⁷. It is possible to “program” these biological polycomputing systems to do the same tasks as computers, so that we can compare the two, but it is not simple. One way to program them is by imposing constraints until the system is forced to conform to our expectations, like slime mould navigating a cave⁴⁰⁸. Another is to use bioelectricity. The right signal can trigger a dramatic shift in morphology, like the growth of an organ⁴⁰⁹ or cancer⁴¹⁰. These means of programming are extrinsic, top down impositions on the system.

W-maxing involves choosing not just a weak policy, but an abstraction layer. A body is an abstraction layer. The right morphology can express a weak policy. The wrong morphology cannot. When cells network, they can express complex policies beyond the capabilities of any one cell. They form a ‘solid brain’ with a persistent structure that supports bioelectric information processing. However, nature’s intelligence is not restricted to such solid brains. Information processing can be carried out by ‘liquid’ brains that are spread out across time and space⁴¹¹. Such a system may not actually store a coherent policy anywhere at any one time. Rather, the policy is implicit in the behaviour. Such liquid brains seem to be able to solve complex problems just as a solid brain might⁴¹². Biological systems seem to excel at creating abstraction layers. Sometimes liquid, sometimes solid. If we want to understand why biology adapts so well, then we can look at how it forms abstraction layers, and what sort of liquid and solid brains adapt well.

A group of humans is a liquid brain. It has layers of abstraction, just like everything else. The behaviour of a group of soldiers can be a squad, and the behaviour of a couple of squads can be a platoon. It isn’t always, but it can be if it obeys certain constraints. These abstractions are classifications of behaviour. We ascribe these labels to groups of people that behave a certain way.

Some organisational and economic structures can adapt more efficiently than others. I’ll embrace the military analogy as it has one very helpful example⁴¹³. When peacekeepers were deployed to Bosnia in the 90s, some units were successful in preventing massacres. Others were not. A force called NORDBAT 2 was successful, but they were also unconventional. They subscribed to the doctrine of Mission Command, which emphasises autonomy and independent action. Control is delegated to the units on the ground so that they can choose how to serve the larger objectives of the mission⁴¹⁴. Low ranking officers are taught to take the initiative. If the circumstances demand an unconventional approach, Mission Command enables it. It allows for more bottom up control in exactly the sort of organisation best known for autocratic top-down control. This enables adaptation. NORDBAT 2 were trigger happy, but effective in preventing massacres. The counterpoint to NORDBAT 2 was the Dutch force operating in the same region, under the same command. Their approach was different, as were the results. Micromanaged by their home government, they did not adapt. They did not engage, and they ended up presiding over a massacre.

This illustrates an important point. Biological brains, whether liquid or solid, are distributed systems made up of many parts. Like a decentralised network of computers. Those individual parts adapt, and that enables the system as a whole to adapt. By delegating control to the forces on the ground, Mission Command simply leverages the computational infrastructure available. The feedback loop is shorter, because information doesn’t need to propagate all the way up to a central command. Looking at systems this way, we can see parallels with computers. A computer can be made up of smaller computers, and it can delegate work to those smaller computers. In doing so, it can delegate more or less leeway in how those tasks are complete. Delegating work and letting smaller computers decide how that work is done is delegating control. Delegating work and rigidly constraining how the smaller computers go about it is just delegating work, not control. Looking at systems this way, we can chart every adaptive system along three dimensions:

1. abstraction: Every system is composed of abstraction layers.
2. distribution: The machinery of the abstraction layer can be distributed, or centralised. Distribution allows work to be divided up into subtasks that are undertaken in parallel. For example, a single core CPU is highly centralised, and a supercomputer made up of thousands of cores is distributed. A group of humans is highly distributed, and one human is not.

3. DELEGATION OF CONTROL: Within any system, control is delegated to a level of abstraction. Put another way goals, and the choice of how to go about them, can be delegated to different levels of abstraction. Say I have a group of humans. If I delegate control to the individual level, then those individuals choose what to do. If I exercise a little bit of control and insist they pursue a particular goal, then they can choose how they pursue it it. If I exercise more control I can tell them how to pursue it. I delegate less control to the individuals, concentrating more of it at the top. Less delegation of control means more top down micromanagement. Too little control and the system doesn’t do what I want it to do. Delegate enough control, and the system becomes controlled bottom-up rather than top-down.

Obviously every system is a balance of these things. Delegation of control and distribution are often correlated, but they are not the same thing. Figure 2 and 3 illustrate. Consider an economy. In a free market, the level of abstraction to which control is delegated is the individual. In a command economy, control is concentrated at the state level. Economies can be more or less distributed. Supply lines can be centralised, monopolies can form, and control can be rigidly top-down. In an ideal Soviet style command economy, control is not delegated at all. It is concentrated at the top. Work is distributed among the components of the system, but not control.

Figure 3: Notice that delegation in this sense is distinct from the distribution or decentralisation of work and resources. For example, if I program a cluster in C then my control is at the level of C, not lower. It doesn’t matter whether I program one computer or fifty: control is happening at the same level of abstraction. I can look at the same system at many different levels of abstraction, but control takes place at specific levels of abstraction.

In contrast, in an ideal free market of the sort that Friedman⁴¹⁵ or Hayek might have liked⁴¹⁶, control is entirely delegated. Individuals decide what to buy, and what to produce. Networks of individuals are allowed to form in order to exert localised top-down control over companies and other small organisations. Eventually, the upward causation of this bottom-up control governs the entire economy. Of course, these are just ideals to illustrate what I mean by delegated control. Free markets can degrade into centralised monopolies, and monopolies can be broken. Now I will explain how distribution and delegation of control are formalised.

DISTRIBUTION

Distribution amounts to having more than one policy expressed by an abstraction layer. For example in a collective of cells each cell is a policy, and if those cells are working towards the same goal then the intersection of their extensions is extension of their collective policy. That collective policy is the higher level of abstraction. The “identity” of the collective, to borrow the neurobiological term⁴¹⁷. For example, say $\omega ,\alpha ,\beta$ and $\gamma$omega, alpha, beta, and gamma are $\mathfrak{v}$-tasksv tasks. Let $\omega$omega be an organ made up of cells $\alpha ,\beta$ and $\gamma$alpha, beta, and gamma. That means $\alpha ,\beta$ and $\gamma$alpha, beta, and gamma are all children of $\omega$omega. ${\Pi }_{\omega }={\Pi }_{\alpha }\cap {\Pi }_{\beta }\cap {\Pi }_{\gamma }$the extension of omega is the intersection of the extensions of alpha, beta, and gamma, meaning there can only be a coherent collective policy if the parts of the system can share a policy, and if the vocabulary can express it. That collective policy is the aforementioned collective identity. If the task is too difficult, then there may be no possible collective identity and the collective may break apart. This is consistent with descriptions of cancer as a loss of collective identity, when cells become isolated from the collective information structure and revert to primitive transcriptional behaviour⁴¹⁸.

However I’ll discuss that in later chapters. For now, what matters is that we can represent the distributed nature of a system as a set of child tasks. Distribution is might be more conventionally conceived of in terms of connections in a graph (see 4). I don’t need to explicitly represent the connections. They are implicit in the correctness of tasks. What matters is whether it is possible for the parts of a system to work according to a shared policy, and whether they actually do.

Figure 4: Illustration of a decentralised system (left) vs a centralised system (right). Notice that if a central node from the system on the right is deleted, then large portions of the graph become disconnected. If a node is deleted from the left graph, then there would still be a path between all remaining nodes. This illustrates what I mean about correctness. The more fragile, sparsely connected graph may have fewer correct policies and thus may be more prone to failure.

DELEGATION

Delegation of control is how constrained the system is at different levels. Where are policies selected? Where does adaptation take place. Consider a typical machine learning stack.

Assuming a typical sort of machine learning model, adaptation just involves tuning some parameters until the model spits out responses resembling the training data. Adaptation occurs only at the level of the model. The library does not change to better suit the task at hand. CUDA does not change. The GPU does not change. In contrast, consider the system which originally constructed the model. It would look something like this.

This stack adapts at much lower levels. The human can construct new math to make a better model for the task at hand. When we swap out the human and math for a static computational stack, we lose that adaptability. We also lose the conscious effort involved, so it’s overall a good thing, but for that convenience we trade adaptability. Does this hold up in practice? Yes. Recall the utility measure from earlier, that says how useful intelligence is for a task by measuring the difference between the strongest and weakest policies for a task in a vocabulary. Also, the multilayered architecture.

• $ϵ:\Gamma \to \mathbb{N}$epsilon from Gamma to the set of natural numbers such that $ϵ(\gamma )={\max }_{m\in {\Pi }_{\gamma }}(|E_m|-|O_{\gamma }|)$epsilon of gamma equals the maximum over m in Pi sub gamma of the difference between the magnitude of E sub m and the magnitude of O sub gamma. Maximisation of utility means maximising $\bullet \stackrel{ϵ}{<}\bullet$ that returns true iff $ϵ(\alpha )<ϵ(\omega )$the epsilon-comparison operator that returns true if and only if epsilon of alpha is less than epsilon of omega.
• The stack is represented here by a sequence of uninstantiated tasks $⟨{\lambda }^0,{\lambda }^1\dots {\lambda }^n⟩$lambda zero through lambda n s.t ${\lambda }^{i+1}⊏{\lambda }^i$lambda i plus one is a refinement of lambda i.
• $f$f is an abstractor function.
• The state of the MLA is a sequence of policies $⟨{\pi }^0,{\pi }^1\dots {\pi }^n⟩$pi zero through pi n and a sequence of vocabularies $⟨v^0,v^1\dots v^n⟩$v zero through v n such that $v^{i+1}=f(v^i,{\pi }^i)$ and ${\pi }^i\in {\Pi }_{{\lambda }^i(v^i)}$v i plus one equals f of v i and pi i, and pi i is an element of the policy space of lambda i of v i.