HOW TO BUILD CONSCIOUS MACHINES
BY MICHAEL TIMOTHY BENNETT
© 2025 M. T. BENNETT
HOW TO BUILD CONSCIOUS MACHINES
How to build a conscious machine? For that matter, what is consciousness? Why is my world made of qualia like the colour red or the smell of coffee? Are these fundamental building blocks of reality, or can I break them down into something more basic? If so, that suggests qualia are like an abstraction layer in a computer. A simplification. Some say simplicity is the key to intelligence. Systems which prefer simpler models need fewer resources to adapt. They “generalise” better. Yet simplicity is a property of form. Generalisation is of function. Any correlation between them depends on interpretation. In theory there could be no correlation and yet in practice, there is. Why? Software depends on the hardware that interprets it. It is made of abstraction layers, each interpreted by the layer below. I argue hardware is just another layer. As software is interpreted by hardware, hardware is by physics. There is no way to know where the stack ends. Hence I formalise an infinite stack of layers to describe all possible worlds.
Each layer embodies policies that constrain possible worlds. A task is the worlds in which it is completed. Adaptive systems are abstraction layers are polycomputers, and a policy simultaneously completes more than one task. When the environment changes state, a subset of tasks are completed. This is the cosmic ought from which goal-directed behaviour emerges (e.g. natural selection). “Simp-maxing” systems prefer simpler policies, and “w-maxing” systems choose weaker constraints on possible worlds. I show w-maxing maximises generalisation, proving an upper bound on intelligence. I show all policies can take equally simple forms. Simp-maxing shouldn’t work. To explain why it does, I invoke the Bekenstein bound. It means layers can use only finite subsets of all possible forms. Processes that favour generalisation (e.g. natural selection) will then make weak constraints take simple forms.
I perform experiments. W-maxing generalises at
AUSTRALIAN NATIONAL UNIVERSITY
DOCTORAL THESIS IN COMPUTER SCIENCE
This dissertation is an account of research that began March 2019. It is comprised of 13 chapters, based on 13 of my papers, written under 13 advisors and completed on the 13th of May, 2025.
The work presented in this thesis is that of the candidate alone, except where indicated by due literature reference and acknowledgements in the text. It has not been submitted in whole or in part for any other degree at this or any other university.
Michael Timothy Bennett, May 2025
ACKNOWLEDGEMENTS
This work was mostly funded by my personal savings account. RIP. This work was partly funded by a Fundaçao para a Ciência e a Tecnologia (FCT) grant under the reference PTDC/FER-FIL/4802/2020, JST (JPMJMS2033), and an Australian Government Research Training Program (RTP) Scholarship.
I’d like to thank the 13 people who have advised me during my various attempts at research at the ANU, both during my masters and during my PhD1: Sean Welsh, Anna Ciaunica, Yoshihiro Maruyama, Colin Klein, Sylvie Thiebaux, Marcus Hutter, Marcus Hegland, Michael Barnsley, Elizabeth Williams, Ehsan Nabavi, Uwe R. Zimmer, Badri Vellambi and Samuel Allen Alexander. I’d particularly like to thank Yoshi, Sean and Anna. I would not be here without you. To Yoshi, who became my primary supervisor two years into my PhD: You saw something in me and my half-complete project, which I can only imagine must have sounded mad. If you hadn’t co-authored that first journal article with me, my academic career might have been over before it began. To Sean and Anna who have worked so tirelessly to get me over the finish line, without your support I might never have finished! My thesis has benefited immensely from your inputs, and your support. You have made a huge difference! On top of that I must note that Sean has done all this in his spare time as an independent researcher. Sean you have been incredibly generous with your time and feedback, and this thesis would be much less polished without your oversight. I will never forget it!
I also want to thank the AGI Society and its members. The 2023 and 2024 AGI conferences were the highlights of my PhD. The encouragement, awards and sense of belonging I felt there quite profoundly changed my life! I’d also like to acknowledge the many others who have helped me, but there are too many. To Ricard Solé, Lenore and Manuel Blum, Karl Friston, Peter Watts, Noel Hinton, Vincent Abbott, Lucas Scott, Simon Strauss, Elija Perrier, Paul McMahon, Tim Wicks, Seth Lazar and the many others who were so generous with either encouragement or feedback: Thank you!
Finally to Ashitha Ganapathy: You have been with me through the highs and lows of all of this. My maniacal obsessions with new topics that extended the length of this thesis by years. My moments of despair, forgetfulness and stubbornness. We even wrote our first paper together. You have listened to every part of this thesis more times than I can count. I don’t know what I would have done without you. More than anyone, credit for getting me this far goes to you. From the bottom of my heart, thank you. This thesis is your achievement as well.
SECTIONS ALREADY PUBLISHED
To validate my progress I have continuously published throughout my PhD. Key published include optimal learning2 (chapters 6 and 7), and my arguments regarding meaning3 (chapter 10), causality4 (chapters 9 and 12) which links consciousness to intelligence, the fermi paradox5 (chapter 11), complexity6 (chapter 7), the artificial scientist7 (chapter 13) and abstraction layers8 (chapters 4, 5, 6 and 8). I published The Mirror Symbol hypothesis, which informs many of the results in meaning, in an IEEE journal9 (chapter 10). My argument regarding the hard problem10 (chapters 12 and 13) and my more recent survey of AGI11 (chapter 3) are currently under review, but the former was accepted to and presented at both ASSC27 and MoC5. My paper on systems as a stack is of central importance to this thesis, and is forthcoming12 (chapters 4, 5, 8, 10 and 11). I co-authored and published a precursor to that paper at an IEEE cybernetics conference13. My paper on Computable Artificial General Intelligence14 was important but it has been under review with IEEE Transactions on Emerging Topics in Computational Intelligence for 3 years. Fortunately, I was able to publish the key result of that paper at AGI-23 and 24 instead. I’ve written 21 papers in total. I expect 19 of those will have passed peer review by the time this thesis is out. My other papers are also cited but are not particularly important for this thesis. So this thesis is comprised of 13 chapters, based mostly on 13 of my papers, written under 13 advisors and completed on the 13th of May, 2025. Though many of these results were published in stand alone papers, they were all written in service of the vision I present here.
DISCLAIMER
This thesis is a draft that is under review. I have already published most of the results in peer reviewed books and journals, but the thesis itself may still change based on reviewer feedback. Take it with a grain of salt.
Please send questions, feedback, hate and fan mail to michael.bennett@anu.edu.au.
Contents
I. FOREWORD AND CHAPTER SUMMARIES
Humans overlook subtractive solutions. We refuse to reduce. Engineers cobble together bits of code into webs so monstrous the errors cannot be found. On a more human scale governments add laws with reckless abandon, but how often do you see them repeal the old? This bias for expansion over contraction is well documented across the spectrum of human endeavour15. For scientific and philosophical pursuits, I suspect our tendency to overlook subtractive solutions has made many problems more difficult than they need to be. When we encounter data we cannot explain within the confines of existing theory, an additive solution would be to construct more and more convoluted theories to reconcile the old theory with the new data. However, we do not always need to reconcile the new with the old. We just need to explain what is, and sometimes that means throwing out preconceptions. For example Milton Friedman proposed simple monetary models instead of complex cyclical models, informing monetary policy that allows us to avoid repeating the great depression. I am interested in broad reaching questions which are similarly burdened by precedent. How can we build a conscious machine? Why is anything conscious? Alive? What is life? Is complexity an illusion? Are biological systems more intelligent than artificial intelligence? Why? In search of answers I have published a number of papers16 in peer reviewed books and journals. I wrote these papers not as disconnected works but as interconnected parts of a larger vision, culminating in this thesis.
Overall this thesis is about how to build a conscious machine. I don’t actually have a conscious machine, because that seemed like overkill for a thesis. What I do have is an explanation of what consciousness is and how it came about. There remain one or two unanswered questions. I also have some proofs and experimental results showing how to ‘adapt’ as efficiently as possible, which is useful for building artificial superintelligence.
There are a few other results too. I’ve given explanations of the origins of life, language, the Fermi paradox, causality, an alternative to Ockham’s Razor, the optimal way to structure control within a company or other organisation, and instructions on how to give a computer cancer. They’ve mostly been published and, strange as it is, they all tie in to a coherent vision. They weren’t conceived in isolation, but as parts of a whole.
What follows now is a summary of the whole thesis. The purpose of this summary is to give you a narrative overview of what I am doing and why, before I get into the weeds. As such, it uses terms like causal-identity and task without formally defining them. These terms are formally defined later in the thesis main body, but here and now they are to be read intuitively. Brevity is the only virtue to which this chapter aspires.
II-III. LITERATURE REVIEWS
Chapters II and III are literature reviews.
Chapter II surveys philosophy and neuroscience. What is a conscious entity? To build one, I must know. Philosophy, psychology and neuroscience all provide insight. However the matter is far from settled. I must take concrete positions on disputed issues within these fields before I can say how to build a conscious machine. Hence I survey some relevant concepts and disputes, combining the introductory sections of my publications on enactive and ethical AI17, communication18 and consciousness19. Topics covered include the mind body problem, functionalism, theories of consciousness, self organisation, the free energy principle, enactivism, epistemology, semiotics, structuralism, post-structuralism and theories of meaning.
Chapter III deals with AGI, which is the foundation of this thesis. It is a survey from on of my earliest publications20, updated to reflect more recent developments21. I begin by discussing several definitions of intelligence and AGI. I end up framing intelligence as adaptation22, and AGI as that which adapts generally. For the purposes of benchmarking, I define AGI is an artificial scientist. I take inspiration from Sutton’s ‘Bitter Lesson’23, which is that throwing compute at a wall consistently beats human ingenuity. With sufficient resources any general approach to optimisation can eventually attain an arbitrary level of skill. Two have consistently scaled: search and approximation. I discuss strengths, weaknesses and examples of each. Hybrids of search and approximation are best. I discuss some hybrids including Hyperon24, AERA25 and NARS26. I introduce the concept of meta-approaches that can be applied to search, approximation or hybrids. One example of a meta-approach is the maximisation of scale and available resources (scale-maxing), in accord with Sutton’s bitter lesson. Another is simplicity maximisation (simp-maxing) based on Ockham’s Razor. I evaluate the strengths and weaknesses of these approaches. They allow us to speculate about how a superintelligence might behave. However, simplicity is a matter of interpretation. It is subjective, and so these claims are also subjective. In this thesis I propose an alternative meta-approach that is optimal. Overall the meta-approaches I discuss in this thesis are to maximise the simplicity of form (simp-maxing), to maximise the scale (scale-maxing) and to maximise the weakness of constraints on function (w-maxing). This latter one is my proposal.
IV. WOW, EVERYTHING IS COMPUTER
This chapter explains why complexity is subjective and what can be done to formalise objective performance. The key result is a concept I call computational dualism, described in my publication of the same name27. I begin by pointing out that the very idea of a software intelligence is broken. The behaviour of software is determined by the hardware on which it runs. It interprets the environment for the software, and the software for the environment. I use the term ‘computational dualism’ to describe theories that treat ‘minds’ as disembodied entities that interact with the environment through an interpreter. I conclude that to make claims regarding the objective behaviour of an intelligence, we must avoid computational dualism.
I propose a solution, which I published earlier in several of my papers28. To avoid computational dualism, it might be tempting to think we just need to focus on the hardware. However this would repeat the same mistake. Computer systems are organised into “abstraction layers”. Higher abstraction layers run in lower abstraction layers. For example Python is interpreted by a C program. I argue the abstraction layers do not end at hardware, and that hardware is interpreted by physical laws just as software is interpreted by hardware.
Taken to its logical conclusion, everything is a stack of abstraction layers29. Software is a state of hardware. A human is a state of organs which are states of cells. If the mind is
V. TURTLES ALL THE WAY DOWN
Chapter V is about embodiment. Each body is an abstraction layer. When I do something with my body like raise my arm, I change the possibilities for what happens next. I impose a constraint on the world. In this sense, a body speaks a formal language34. This embodied language is ontological, meaning a statement is rather than refers to something. Every physical thing is an abstraction layer that speaks a formal language, not just living bodies. A computer speaks a formal language of hardware states. The universe speaks a formal language of physics35. This idea is once again from my publications on abstraction layers36. I show how Stack Theory expresses an embodied formal language of declarative programs. Those programs are the vocabulary of the language. Using this, the body makes statements that have truth values. In an embodied formal language, something is physically ‘said’ by the environment. This is the language of physical laws. If I was omniscient the environment would have one state at a time, because time is difference37. That state would determine what is true at the present time. The grammar of the language comes from the fact that states of the environment are in this sense mutually exclusive, and some programs in a vocabulary can never be expressed together. Everything that exists is a statement made in an environment’s embodied formal language, and which statements are true depends on the state. However from my subjective perspective within my environment, I cannot know what the physical state is. I am a statement, and I exist for as long as the environment expresses me. When a statement is made, it constrains the space of what else can happen. Each statement has an extension. Intuitively, my extension is like the ‘many worlds’ in which I exist.
Each statement implies another higher abstraction layer. The extension of a statement forms a vocabulary of the layer above. In this way, every statement the environment makes creates an abstraction layer. The outputs of the level below form the vocabulary of the level above. We go up a level of abstraction by looking at second order effects of a body we started with. An abstraction layer is like a smaller environment defined in the context of a larger environment. A ‘small world’ defined inside a ‘big world’38. It has its own formal language that is equivalent to a subset of the things the bigger environment can say. Each statement the environment makes is a body, and each body has an extension and thus its own, more restricted embodied formal language in which further statements can be expressed. Layer upon nested layer of abstraction.
VI. MASTER, WHAT IS MY PURPOSE?
Chapter VI is about purpose. These results were published earlier in my papers on abstraction layers39 and consciousness40. The result is a formal definition of an embodied tasks, inference and stacks. In earlier chapters, I defined bodies or hardware as embodied formal languages that express statements. I can choose any statement the body can make and call it an input. The possible outputs are the extension of the input. So a body can be seen as a computational system that maps inputs to outputs. I can single out a subset of those possible outputs and call them correct. I call a set of inputs and outputs a task. This is a way of formalising a arbitrary notions of correctness, or what ought to be. According to Hume’s Guillotine, I cannot derive what ought to be from what is so I need a universal, cosmic ought from which to derive all others. I argue this comes from time. Change is foundational. Statements are destroyed as states change. A body is a statement that lasts for as long as the environment expresses it. Subjectively, we can interpret the process of creation and destruction as statements ‘moving’ relative to one another. Statements that persist are those that move away from circumstances in which they are destroyed. As the environment transitions from one state to another, this eliminates that which doesn’t seek to preserve its existence. This is like natural selection, but applied to every aspect of the environment. It creates an incentive I call the cosmic ought. What I argue here is that everything that the environment expresses is a statement of what ought to be, and the rest that which ought not.
Each statement implies a narrower abstraction layer than the one in which it was expressed, like a window or small world within a big world. As we go to higher in a stack, the ought gets more specific. For example, an environment could be an abstraction layer. Lifeforms would then be statements made in that abstraction layer, growing ever more specific with each additional layer of abstraction. A lifeform might be considered ‘fit’ if it continues to exist, so the set of all fit outputs for a fit organism could be its extension. However organisms are often unfit. Such ambiguously ‘fit’ organisms would be statements whose extension contains unfit as well as fit behaviour. Were it to engage in unfit behaviour it would still exist, just not in any condition to maintain homeostatic and reproductive goals. It is that distinction between ‘fit’ and not that a task formalises, by pointing out the set of outputs considered ‘correct’ and the inputs in which being correct actually matters. Hence I formalise goal directed behaviour in a stack of tasks41.
VII. WEAK
Chapter VIII is about intelligence. The key results were published in my papers on ‘weak’ hypotheses42. The result is a theory of optimal learning. I propose a meta-approach I call w-maxing, and an upper bound on intelligent behaviour based upon it. I formally prove and demonstrate experimentally that w-maxing is optimal, and simp-maxing is not43.
If we take a Darwinian point of view then intelligence is long-term adaptation44 that facilitates short-term adaptation45. Without intelligence, an organism would need to have all knowledge hard coded from birth. With intelligence, it can adapt during its lifetime to survive in more circumstances than without intelligence46. To represent this in my formalism I describe an organism by what it does, rather than is47. What it does is a task. I explain how the task an organism does can be subdivided, by choosing subsets of inputs and outputs I call child tasks. Tasks thus exist in a generational hierarchy. An organism’s past is a child task of its future task. A task implies a set of policies that constrain an organism’s behaviour to the task definition. An organism embodies a ‘fit’ policy if it is constrained to fit behaviour. The process of learning is inferring a policy from the past that ensures future behaviour is fit. Intuitively, a policy is like a tool. A tool can complete more than one task. A hammer can be either a weapon or a paper weight. A weaker policy is a tool that completes more tasks. The weakest policies complete the largest number of tasks. I prove that, among all policies, the weakest policies are the most likely to generalise, maximising efficiency of adaptation. I call this the meta-approach of w-maxing4849.
I go on to compare w-maxing and simp-maxing50. I prove that we can w-max without simp-maxing. I support this claim with experiments comparing the two meta-approaches. I have them attempt to learn binary multiplication and addition. The w-maxing system outperforms the simp-maxing system by
VIII. STACKISM
This chapter brings together my papers on complexity51 and abstraction52. I explain why simplicity of form has anything to do with function. In theory there could be no correlation, but in practice there is53. My result is proofs explaining this correlation, and this explains why biological systems seem to adapt more efficiently than AI. I begin by proving that at the lowest level of abstraction, all policies are equally simple. There is no such thing as objective complexity. Then I argue bodies must use finite vocabularies, because of the Bekenstein bound54 55. I show that there exist abstraction layers in which simple statements are weaker. Because vocabularies are finite, an abstraction layer in which weak statements take simple forms will be able to express more weak policies than an abstraction layer where weak policies do not take simple forms. This means complexity is an illusion perpetrated by abstraction layers. To maximise adaptability given finite resources, it is necessary56 for abstraction layers to express weaker constraints using simpler forms. In other words, maximising the weakness of constraints on function (w-maxing) will cause simplicity of form to be maximised (simp-maxing), but simp-maxing may not cause w-maxing. Natural selection prefers bodies that can express policies that are more versatile. This forces a correlation between weakness and simplicity. Since we are products of natural selection, our languages reflect this.
Next I explore how systems do this. Biological systems seem to do a better job than AI of building versatile abstraction layers. To understand why, I look at how systems vary along dimensions of abstraction, delegation and distribution. I argue systems which delegate control to lower levels of abstraction are more adaptable. I illustrate this point using examples from biological, computational, human organisational, military and economic systems. Using Stack Theory I prove that adaptability at higher levels of abstraction requires adaptability at lower levels of abstraction. I call this The Law of the Stack. I argue biological systems are more ‘intelligent’ than because they delegate control to lower levels of abstraction. To put it provocatively, artificial intelligence is like an inflexible bureaucracy that only adapts top down. By adapting at lower levels of abstraction, biological systems can ensure weak constraints take simple forms at higher level57. I argue this is why naturally occurring systems enforce a correlation between simplicity of form and weak constraints on function. This is why there is a correlation between simplicity of form and the weakness of constraints on function.
IX. LETS GET PSYCHOPHYSICAL
Chapter IX is about how there are objects and properties. This is brings together my work on causality58 59 and consciousness60. The key result is the formalisation of causal-identites explaining how systems learn cause and effect, the Psychophysical Principle of Causality explaining why systems learn the objects and properties they do based on w-maxing, and the formalisation of selves that will inform the later theories of consciousness and meaning.
Normally to describe causality I would start with a set of variables representing objects and their properties, and then experiment to figure out if changing one variable changes another. However this only works if I already have the world divided up into variables, which my formalism doesn’t yet have. Fortunately, we have attraction and repulsion from physical states. Valence, which is a causal relation. Hence, I can flip the problem and learn the objects instead. We have proofs of optimal adaptability, and any system that adapts optimally must correctly identify cause and effect. I show that by w-maxing in response to attraction and repulsion from environmental states61, a system embodies policies that classify causes of valence. I call these policies causal-identities. They are prelinguistic classifiers. Weaker causal-identities classify more commonly encountered causes of valence. This explains why and how a contentless environment is divided up into objects and properties. I call this The Psychophysical Principle of Causality62. I identify two preconditions for a system to construct a causal-identity for an object: incentive and scale. First there must be an incentive, for example the object is relevant to survival. Second, the system must be able to embody the causal-identity63.
To survive, I must be able to tell the difference between what I have caused, and what I did not. This implies the construction of causal-identities for one’s self. I introduce ‘orders’ of causal-identity for self, and show that if the scale and incentive preconditions are met they will be constructed. 1st-order-self classifies my interventions. A 2nd-order-self is my prediction of your prediction of my 1st-order-self. This is needed for theory of mind, or to herd and capture prey. Finally, a 3rd-order-self permits one to predict one’s own 2nd-order-selves, which is needed to predict social environments and complex narratives.
X. LANGUAGE CANCER
Chapter X is about language and cancer. This integrates my first paper, in which I proposed The Mirror Symbol Hypothesis64, with my subsequent papers on symbol emergence65 and the formalisation of Gricean pragmatics66. The results are the formalisation of how meaning is communicated, of how norms are formed and how this relates to cancer, and a refutation of the Orthogonality Thesis.
I show how 2ND-order-selves are necessary for communication as described by
This explains the emergence of norms. Organisms that can communicate can co-operate. Now that we know how, it is easy to see how language would evolve. I formalise protosymbols and preferences to connect causal-identities to established semiotic theory. I explain how co-operation facilitates social predation, and how sufficient predictive accuracy in repeated interactions incentivises honesty. I argue members of a species have similar preferences, and thus efficiency dictates an organism use its own preferences to predict others6869. Finally, I relate normativity to cancer. In an ecosystem computation is distributed and concurrent. Different organisms act upon one another at the same time, forming collectives. When they constrain one another in service of a goal, they form a collective informational structure with an identity.
XI. WHY IS ANYTHING ALIVE?
In this chapter I ask what drives the emergence of life in an ostensibly indifferent universe? Why is it that life is complex, when complex forms are less likely to exist? In answering these questions I respond to criticisms of Pancomputational Enactivism which allege my theory does not formalise cognition in a manner which aligns with the Free Energy Principle, and accounts for a boundary71. I argue that a rock persists by simp-maxing alone, and that causes it to persist because simpler forms tend express weaker constraints. On the other hand, a system that self-repairs does the opposite. It w-maxes at the expense of simp-maxing. This is only possible in a stable environment. When the underlying stack is static, weak constraints do not need to take simple forms. A slime mold is more fragile than a rock in general, but in the context of earth’s environment it is more adaptable in the sense that it can do more, to spread and multiply. Systems like this can optimise for adaptability within the constraints of higher levels of abstraction. I then relate this to The Law of Increasing Functional Information72 73, which I translate into Stack Theory and subsequently prove. Finally, I explain the Fermi Paradox using the incentive and scale preconditions for causal-identities. Intelligent systems might be all around us, but we do not recognise them as intelligent because we cannot construct a rationale for their behaviour. They fall outside the scale and incentive preconditions afforded by the human stack. This integrates my papers on abstraction layers74, complexity75 and most importantly my early paper on The Fermi Paradox76. This serves to further illuminate how and why we divide our subjective worlds up into the objects and properties that we do. I do all of this in order to lay the foundations for chapter XII.
XII. WHY IS ANYTHING CONSCIOUS?
Chapter XII addresses the hard problem. I describe what is consciousness, and why is anything conscious. I published these results earlier in my papers on consciousness77. First, Higher Order Thought theories argue that we are conscious of higher order meta-representations of lower order conscious states like ‘the smell of coffee’ or ‘the colour red’, but they don’t explain where the latter come from. To understand higher order consciousness we must explain how lower order local states of consciousness arise.
I begin by examining valence. At the most basic level we have ‘one-dimensional’ valence. How a cell is attracted or repelled, for example. Such a system cannot learn a causal-identity for any object. However, when we have two cells we now have a richer vocabulary. We can express more. If we scale up the system, we can have many parts which are being attracted or repelled by the state at any given time: a ‘tapestry of valence’. A vast orchestra of cells playing a symphony of valence. Every state of the environment would evoke such a symphony, which can be reduced to causal-identities for those aspects of the environment which cause valence. This is the point of The Psychophysical Principle of Causality. An organism learns causal-identities from valence alone. They form an abstraction layer. Causal-identities can be categorical variables like hunger and thirst, which at the higher level of abstraction have the same ‘one-dimensional’ valence, but have a fundamentally different qualities because they are different tapestries of valence at the lower level of abstraction. An organism does not have a lookup table of ordered causal-identities, and does not choose to use a policy to interpret inputs. It embodies causal-identities as policies and is impelled by valence to act accordingly. A tapestry of valence does not have the luxury of separating a representation from its estimated utility. Reward is not a label applied after the fact. Interpretation and value judgement are one and the same. I call this integrated representation and value judgement. This is counterintuitive from a computer science point of view, where we are used to dealing in key-value pairs for neat databases. However from an evolutionary perspective such a separation of description from valuation is implausible.
Consciousness is something an organism does, rather than is. It is being impelled by a hierarchy of causal-identities. I argue phenomenal consciousness begins with a 1ST-order-self. A 1ST-order-self accompanies every intervention an organism makes and so, having a character, it answers Nagel’s famous question of ‘what it is like’ to be an organism. Causal-identities become qualia. A philosophical zombie has access but not phenomenal consciousness. The contents of access consciousness are those available for communication. I argue that means access consciousness requires a 2ND-order-self, because that is what is required to communicate meaning in the pragmatic sense as humans do. Communicating requires reasoning about interventions. Hence, it also requires a 1ST-order-self. A philosophical zombie that behaves exactly like a human is therefore impossible. Intelligent behaviour at a human level requires a 1ST and 2ND-order-selves. Efficiency demands the delegated computational architecture of biological self-organisation with persistent structure that supports a tapestry of valence. Intelligence adaptability, so there is no way to achieve human-level intelligence without consciousness. Increasing intelligence is reflected in increase scale that facilitates the construction of causal-identities. I conclude the chapter by I describing the stages a conscious organism, from rocks to humans, as intelligence increases.
XIII. HOW TO BUILD CONSCIOUS MACHINES
Chapter XIII is about how to engineer conscious machines. It integrates my papers on the artificial scientist78, and consciousness79. The key result is a description of the features necessary and sufficient to build a conscious machine, the proposal of an unresolved problem I call The Temporal Gap, and two options describing strategies we might take to build a conscious machine, or to avoid building a conscious machine respectively.
I begin by discussing existing theories of conscious machines and AGI. I argue that, because intelligence begets consciousness and consciousness requires intelligence, these are one and the same. I frame Stack Theory and subsequently Pancomputational Enactivism as bottom up frameworks. I argue they should be used to improve rather than supplant existing theories that focus on top-down implementation of conscious or intelligent systems. I subsequently enumerate the features of an artificial scientist that should in turn lead to a conscious machine.
I examine the shortcomings of conventional computing hardware in contrast to biological polycomputers, and argue there are several features we must build into our systems if we want them to be as adaptive as a human scientist, and thus conscious. I identify a problem I call The Temporal Gap, which is that it is unclear whether a conscious state is at a point in time80, or can be smeared across time81. Machines that satisfy the former definition are conscious according to the latter definition82. This has profound implications not just for what sort of machines can be conscious, but for our understanding of human subjective experience. There does not appear to be any way to conclusively resolve The Temporal Gap. However I argue that if we want to build a conscious machine we should assume consciousness is at a point in time, and design a machine accordingly. If we wish to avoid building a conscious machine, we should assume consciousness is smeared across time and avoid building potentially conscious machines accordingly.
Finally, I conclude the thesis by summarising the many and varied results.
II. SOME PHILOSOPHY
I MUST KNOW WHAT IT IS I WANT TO BUILD before I can build it. I want to build a mind, so that means I have to take concrete positions on disputed issues within philosophy of mind, psychology, cognitive science and neuroscience. The following is a survey of some relevant material from those fields. It is based on the introductory sections of my publications on enactive and ethical AI83, communication84 and consciousness85. Topics covered include the mind body problem, functionalism, the “hard problem” of consciousness, various theories of consciousness, self-organisation and the free energy principle, enactivism, epistemology, semiotics, structuralism, post-structuralism and theories of meaning. Though this is a very broad ranging survey, I try to tie these concepts together into a coherent, sequential story from beginning to end.
A BRIEF HISTORY OF THE MIND BODY PROBLEM
There is public knowledge, and private knowledge86. When I see, smell, touch, hear or taste an object, I am said to be directly observing that object. However, I cannot directly observe someone else’s experience. I can see evidence that they might be experiencing pain, for example. I could run a test and directly observe C-fibre stimulation in their brain, but that is not the same thing as directly observing their experience of pain. One’s own subjective experiences are “private” knowledge. To say something is “public” information is to say it is at least possible for more than one person to observe the event. A private event is never observable by more than one person. Even if I somehow built a computer to “read” someone’s mind, store the information, and then “write” that subjective experience into another’s mind, how could I know the experience is truly the same? When a scientific experiment is run, it is to test whether one publicly observable event reliably follows another publicly observable event. One reason it is so difficult to study the mind is because the things for which we are testing are not publicly observable87. This brings us to the “mind-body” problem.
“What is a mind” is a loaded question, because it seems to suggest a mind is a publicly observable object88. We know that minds are had by particular things. So instead we could ask “what does it mean when we say something has a mind?”. We know the things we can observe that have minds also have physical substance. Objects with physical substance are spatially extended, meaning that for each moment in time that they exist they must occupy space. No other physical object may occupy that same space at that same time.
However, when people speak of minds and mentality they often talk like these are not part of their physical form. For example, there is mental and physical illness. This hints at something like a mental substance. Something non-physical. However, mental and physical phenomena clearly have a causal relationship. A mind causes the body to act, and that the body causes the mind to experience what it does.
EARLY 1600s - SUBSTANCE DUALISM
The idea that there exist distinct mental and physical substances is called substance dualism. It was most famously argued by
Descartes, 16th century French philosopher and namesake of “Cartesian Dualism”. He sought to describe the union of immaterial mind and material body. His position is unsurprising, given prevailing beliefs in the 16th century. What is surprising is that his arguments were compelling enough for us to be mentioning them four centuries later.
According to Descartes mental substance does not occupy space. Mental events are not spatially extended. Presumably this is how a mind can be inside a body without making it explode. Descartes thought mental substance interacts with physical substance through the pineal gland, which acts as a sort of interpreter89. An interpreter is like an abstraction layer in a computer. It takes one sort of thing and turns it into another. This idea that the mental and physical causally interact is called interactionism. In the case of Cartesian Dualism, the mental and physical directly interact through the pineal gland. He speculated fluids called “animal spirits” act upon the gland, causing it to move, which causes the conscious states of the mind. The mind then acts directly upon the gland, causing it to move and affect the animal spirits.
This argument has problems. I don’t need to enumerate them. Cartesian Dualism hasn’t aged well, but somehow it is still here. The reason I mention it is because I will later argue that dualism seems to have been baked into computer science90. The idea that AI is a software “mind” running on a hardware “body” echoes Cartesian Dualism. Software is just a state of hardware, and yet many still seem to treat software as something that interacts with the world through hardware. I call this computational dualism91.
MID 1600s - PREESTABLISHED HARMONY
Following Descartes, others asked why mental substance should affect the physical through only the pineal gland, and not elsewhere? Why this inconsistency? Either mental substance affects physical substance, in which case the mental is a sort of physical substance, or it affects nothing physical. In order to preserve substance dualism (likely for religious reasons), some philosophers argued the latter, holding that causal interactions between mental and physical are an illusion perpetrated by god. Leibniz argued that mental and physical processes are set in motion by god in preestablished harmony so that they look like they interact, but never do. Like clocks synchronized by a clockmaker. The practicalities of quantum communication
are strangely reminiscent of this idea92. Malebranche was another philosopher who proposed yet another alternative to interactionism. He argued the physical can affect the mental only indirectly, through the intervention of god93. Each time you will your body to move, god intervenes in the physical world to move your body as you wish94. Any time your body is affected by something in the physical realm, god affects your mind. In occasionalism, god causes all interactions between the mental and physical by intervening constantly to create the illusion of interaction, whereas in Leibniz’s preestablished harmony god intervenes only once, to synchronize the mental and physical worlds so that they appear to causally interact. Either way, there is still an interpreter (god, rather than the pineal gland). I mention all this because the idea of just moving the interpreter or abstraction layer is a central theme of this thesis. It’ll come back a lot.
LATE 1600s - NEUTRAL MONISM
Both Leibniz and Malebranche denied there are any direct causal interactions between mental and physical, invoking god as a means of indirect influence. Spinoza was yet another who denied direct causal interaction, but circumvented the need for divine intervention by arguing that both mental and physical are mere aspects of a third, unobserved substance that is neither mental nor physical. In other words, reality is neither mental nor physical. This position is now called neutral monism. There is a secret third thing. Physical and mental are just aspects of this secret third thing. This idea will come up later when I formalise abstraction layers.
1800s - EPIPHENOMENALISM
Much of the difficulty in understanding the apparent two way causal interaction between mental and physical stems from the assumption that our perception of mental activity as causing physical activity is accurate. What if instead I just consider a one way causal relation? By this I mean that the mental has no causal effect upon the physical, but physical events cause mental events. We might believe we act upon the physical, but this belief is an illusion. For example, I might think I chose to get up and get a glass of water, but every aspect of my decision was determined by physical processes in my body. My mental processes are the effect, not the cause, of physical processes. This is epiphenomenalism. It was proposed by Thomas Huxley, who argued neural events in the brain are the physical events that
cause mental events. However mental events don’t actually do anything. Epiphenomenalism is a means of preserving dualism, but it leaves me wondering why anything would evolve to have consciousness? From an evolutionary perspective, epiphenomenalism seems a bit pointless. The alternative is materialism, or physicalism in contemporary terms. That’s the idea that mental events are just part of the physical world. It seems a lot more compelling, because it means we can come up with an evolutionary explanation for mental events.
NOW - PHYSICALISM
Physicalism comes in two flavours: reductive and non-reductive. The reductive physicalists think we will be able to reduce mental events to non-mental physical events. The non-reductive physicalists believe we will not be able to do that. They hold that certain physical processes have mental properties which are irreducible, meaning we can’t break them down into anything simpler and so we can’t reduce them to non-mental physical parts. This position is basically that “qualia” are fundamental building blocks of reality. This still requires mental causal efficacy, in that mental events cause other mental and physical events. Mental events must supervene on the physical, meaning two objects that are physically identical must be mentally identical. I am a reductive physicalist. Perhaps what might be called a Hobbesian Stackist95. The main point of this thesis is to explain how. I’m now going to steel-man the non-reductive physicalists for the sake of argument.
Psychoneural identity theory is one example of reductive physicalism. It holds that the mind is the brain. Feelings, sensations and thoughts may be reduced to neural activity in the brain, or more generally to a specific physical event. Each “identity” equates publicly observable physical event with a private mental event (they are one and the same thing). One objection to psychoneural identity theory is this: if a mental event like pain is a particular neural event like C-fibre stimulation, then why is it that the same mental event can be caused by entirely different physical events? If I experience pain when my C-fibres are stimulated, and an animal appears to experience pain but has no C-fibres, does that mean it is not experiencing pain? It seems unlikely. Instead, it would seem a mental event like pain can be “realised” by any number of physical events. This is called multiple realisability.
BEHAVIOURALISM AND FUNCTIONALISM
We need to say how systems behave in order to describe mental events. Behaviouralism is the idea that one can equate mental events with outwardly observable behaviour. By observable behaviour, I mean inputs and outputs. Behaviour would be a set input-output pairs. This is one way around multiple realisability. However it mostly depends on how we define input and output. These depend on the level of abstraction. If an input is something so vague as an intuitive human definition of “pain” then yes it would appear an octopus in pain is experiencing pain like a human. If the inputs are as specific as C nerve fibre stimulation then the octopus does not have C nerve fibres and so cannot experience pain. There are many possible processes which map
This brings us to the Chinese Room. Much debated, but a good example of multiple realisability. Imagine I sit in a room. I don’t speak Chinese. Through one door I am passed a note written in Chinese. I pull out my laptop, get Google to translate the note, then pass a response back out the door. Someone outside the room then starts to believe I speak Chinese. Likewise, just because something behaves as if it has a mind, does not mean it does. Behaviouralism discounts mental activity in favour of observable behaviour. It reduces meaning to inputs and outputs. The obvious problem with behaviouralism is that there is more to the story. I think, I know I think, and I can do so without giving an output. Machine functionalism96 tries to resolve this kind of problem by adding a causal intermediary between inputs and outputs. This causal intermediary is an interpreter like a Turing machine. It maps inputs
For a reductive account of the mind to be convincing, it must deal with private first person behaviours (e.g. understanding meaning)97, and show why these behaviours arise. The problem one faces then is arguing “is this behaviour really what I experience”? And we’re back to the public-private knowledge debate. To get around the public-private distinction, I argue we have to step outside the universe and look in. The only way to do that is to establish axioms that hold in every universe. That is the approach I will take in chapter 5. For now, I’ll delve further into background.
CONTEMPORARY EXPLANANDUM AND HARD PROBLEMS
Contemporary theories frame consciousness98 as having two aspects: functional and phenomenal99. Functional means the behaviour of consciousness, however it might be realised. Anything which might be explained by natural selection. Some equate this with “access” consciousness100, which is the contents one can consciously “access” for reasoning and report101. I will point out some inconsistencies in how access consciousness is typically understood. I’ll argue access consciousness is merely part of functional consciousness.
The other aspect of consciousness is phenomenal102. This is the subjective experience of having “global” and “local” states of consciousness. A global state, for example, is being awake or asleep. Local states or “qualia” are the specific experiences of how coffee smells in the morning, or how wet grass feels underfoot. These are hard to define in rigorous terms. By function, I mean anything that serves reproductive and homeostatic103 goals. That serves any goal really, but later I will make the argument that all goals stem from persistence and survival. So for now just take it as that. This information processing results in behaviour natural selection deems to be fit. Some of this information we are consciously aware of, as I am aware of the words I am writing on the page at this very moment. However most of the information processing in our bodies goes on “in the dark”. We are unaware of it, as I am unaware of whether my muscles have decided to atrophy because I have spent too long sitting in this chair writing. Why doesn’t all the information processing go on in the dark? Why do I have conscious access to some information, and not other information? Why is there phenomenal consciousness if we can just do everything in the dark?
Some speculate104 we might take phenomenally conscious being like a human, and make a “zombie” of it. The zombie is a clone that has all the function of the original, but not phenomenal consciousness. From the outside it looks and acts the same, but inside it is dead. If zombies are possible, then that means phenomenal consciousness has no function. If zombies can exist then there is no evolutionary explanation for phenomenal consciousness. Supposing this is true, some have asked why is there something it is like105 to be me, instead of nothing? Why do I subjectively experience some events, when it seems possible106 for information processing to occur without any subject experiencing it? So to reiterate in the simplest possible terms, the functional aspect of consciousness is everything
we can explain as a consequence of evolutionary processes, and the phenomenal part is the rest107. The question is whether there is such a thing as phenomenal consciousness distinct from function. My plan is to explain is to explain phenomenal consciousness as something functional, which will kill the distinction between the two.
THIS IS THE SO CALLED HARD PROBLEM OF CONSCIOUSNESS. Some interpret it as demanding a reductive explanation for local states. However, phenomenal consciousness is arguably an easy problem. Sensory processing may explain the character of qualia, and the “subject” of subjective experience may be explained in causal terms. A representation of the self lets an organism identify the effect of its actions108, and is necessary for accurate inference in many circumstances109. In other words natural selection demands there exist a self to be subject to sensations. What some have called phenomenal consciousness is really the function of consciousness in the first person110, What has been called “functional” is the very same thing from the third person perspective111. However, that doesn’t explain why the same behaviour couldn’t come about112 without consciousness. Some have separated phenomenal consciousness into first person functional and “hard” consciousness113. In that sense, hard consciousness is whatever remains unexplained by function. For the sake of this thesis and the associated papers, I interpret the hard problem as demanding an explanation of why a world in which a zombie is possible is inconceivable114. I’ll address the hard problem by describing how consciousness115 follows from evolutionary processes116, which follow from the very fact of existence. I describe a formalism that applies to every conceivable environment, and show that a zombie is impossible according to that formalism.
LEVELS OF CONSCIOUSNESS
Beyond the phenomenal and functional aspects, there are levels. Morin proposed four levels117, which I describe below. I will later make it 6 levels:
- Unconsciousness: the absence of consciousness, including sensorimotor information processing.
- Consciousness: a minimal level of consciousness in which one has subjective experience of local states. Phenomenal and access consciousness both begin here.
- Self Awareness: this is where there is a distinction between public and private knowledge. One now has an inner monologue, and a concept of self. Importantly, this is where self knowledge becomes possible. It is also, according to Morin, where symbolic representations come into the picture. I interpret this as akin to the “meta-representations” in higher order theories, and I will later show that access consciousness must be equated with self awareness, because it is not possible without it118.
- Meta Self Awareness: the logical conclusion of self awareness if we simply “scale up” the reflective aspect, so that one’s reflection contains a reflection. It is where one becomes aware that one is aware.
CONTEMPORARY EXPLANANS
Functional and phenomenal are broad categories that leave a lot of questions unanswered. There are several dominant theories which seek to explain how the phenomenal and functional aspects of consciousness come about.
HIGHER ORDER THOUGHT THEORIES
Higher order thought theories
“sit”. Grounded, multi-modal symbols. The lower order mental states are more primitive parts of which we cannot be aware, because our awareness is constructed from them. While the focus of HOTs is on access consciousness, they have also been used to shed light on the character of local states120. The theory I put forward in this thesis tacitly embraces HOTs, although its origins lie with AI rather than neuroscience121. Furthermore, I define conscious access in very different terms, and point out flaws in how HOTs define access.
GLOBAL WORKSPACE THEORIES
Like HOTs, global workspace theories
AN ASIDE ON REENTRY
Reentry refers to the bidirectional exchange of signals between brain areas125. It is thought to play a role in synchronized firing of neurons, allowing information to be integrated126, forming patterns within patterns. Higher levels of activity. Some associate consciousness with top down signalling resulting from this127.
INTEGRATED INFORMATION THEORY
Unlike GWTs and HOTs which begin with information processing and focus foremost on access, Integrated Information Theory (IIT) begins with the phenomenal, from first principles regarding the character of qualia128. From those axioms necessary preconditions for consciousness are derived, and then it is claimed that satisfying these preconditions is sufficient to instantiate consciousness129. This is all formalised in mathematical terms. IIT speaks of a “cause-effect structure” and the “causal power of a system to influence itself”. Global states of consciousness are associated with the quantity
AN ASIDE ON SELF ORGANISATION AND NATURALISM
Wʜᴇɴ I sᴀʏ ᴀ sʏsᴛᴇᴍ sᴇʟғ-ᴏʀɢᴀɴɪsᴇs, I mean its parts interact to produce a coherent pattern or whole. Intuitively, think of a drone swarm with no central controller. Distributed computation. The internet. Self-organization130 is more typically defined as the spontaneous emergence of order from interactions131. The notion applies to physics132, biology133 neuroscience134 and of course computer science. A typical assignment for a distributed systems programming class is to write a program that interacts with copies of itself in a simulated environment, and the various copies must co-operate to achieve a goal without electing a central controller. For example, these programs might be nodes in a simulated network, and be tasked with delivering messages to specific addresses in the network without any prior knowledge of where nodes are. That is a great example of engineered self-organisation.
Sᴇʟғ-ᴏʀɢᴀɴɪsᴀᴛɪᴏɴ ɪs ɪᴍᴘᴏʀᴛᴀɴᴛ in biology because biological systems distribute and delegate control down to the level of cells and proteins. Supposing life did not begin with a centralised controller, the only possible means of organisation is self-organisation. They form a multiscale competency architecture where cells form organs, which form organisms which form ecosystems135. In other words, a self-organising system made up of self-organising systems. To be self-organising, a system must act to occupy only a subset of possible states. A system which does not seek some states over others merely exists, rather than self-organises. More intuitively, a self-organising system will break down in some states, so it must act to remain out of those states. It must “resist a natural tendency to disorder”136. It must optimise, or at least satisfice to a survival level. To do this, a self-organising system must predict future states in order to remain within the set of acceptable states. When I talk about an organism, I mean a biological self-organising system motivated to act in a manner deemed fit by natural selection. A conscious human is a self-organising system. So is a snowflake. Self-organisation is only part of the picture, but one well suited to naturalist explanations137 that treat the phenomenal as something that must be functional.
FREE ENERGY
Pʀᴇᴅɪᴄᴛɪᴠᴇ ᴄᴏᴅɪɴɢ explains human perception as the result of predicting the causes of sensory signals. It frames cognition as optimisation, and thus self-organisation. Minimising expected prediction
error cost, or equivalently maximising expected utility or reward. Active inference builds upon this idea to frame cognition not only as optimising one’s internal state or “model” to correctly predict the surrounding environment, but the surrounding environment to match one’s model138. This allows for the possibility of experimentation, to falsify one’s hypotheses. In this context, “free energy” is a bound on prediction error. By minimising free energy, one can minimise prediction error, and so active inference seeks to minimise free energy.
This is called the free energy principle. It is formalised as variational Bayesian inference139, an approach borrowed from machine learning. It is claimed that a system which minimises free energy is optimal, making the most accurate predictions possible. Overall you can think of these ideas as a reformulation of control systems for the purpose of understanding life. To explain consciousness as a consequence of free energy minimisation is to explain it as an adaptation. A functional adaptation. Solms140
REAFFERENCE
One remaining noteworthy account of consciousness is that of Merker141
agrees, though for quite different reasons145. That we arrived at the same conclusion from two different points of origin lends it credence. However my explanation of the emergence of causality also accounts for why organisms divide the world up into the particular objects we do. In other words, it links causality to relevance and symbol grounding.
LIQUID AND SOLID BRAINS
It should be noted here that reafference requires a degree of centralisation. It serves to integrate and unify information for navigation and other purposes. Brains, like those in humans, are solid. The neurons remain in place, and support a bioelectric network. Information is passed synchronously through this network. Timing and direct access to information is important. All of this, in service of the ability to predict and adapt.
However, brains are not the only thing that predict and adapt. Ant colonies can solve shortest path problems. Each and has a brain, sure, but the ant colony doesn’t and it seems far more intelligent than any individual ant.
RELEVANCE AND ENACTIVISM
Relevance realisation is the formation of a cognitive language in which inference can take place. For example, active inference describes how an organism models the world using mathematical tools like variational Bayes. In predictive coding a self-organising system assigns higher weight to more relevant aspects of the world, treating relevant aspects of the world as more precise147. However, where do these aspects come from? How and why is the world divided up into particular objects? Before one can model the world and predict, one must have a language for doing so. I don’t mean a spoken language like human speech. I mean the circuitry of cognition. A vocabulary of primitive structures of which more abstract machinery can be constructed. That vocabulary determines which problems are hard, and which are easy. So before one can engage in active inference or predictive coding, one must first tackle the problem of relevance realisation, turning semantics into syntax. The organism learns a world or language relevant to its motivations148.
Relevance realisation requires the organism be embodied. Yet where does the body begin and end? There is now a great deal of evidence to suggest that mental processes extend beyond the brain, into the immune system149 and even the environment an organism inhabits150. Intuitively, my language of cognition is not constrained to my body. I can use a pencil to write reminders on a piece of paper, and extend my memory into the surrounding environment. I am embedded in a particular environment through which my cognition is extended, and if you take me out of that environment my cognitive capabilities change. Finally, different people may interact to enact cognition, co-creating this text in co-operation with the environment, which I affect and am affected by in turn. Such distributed processing takes place not just between people, but within them. What we call human intelligence is the collective or swarm intelligence of cells151. This blurs the line between organism and environment, but it means we can dispense with the idea of an interpreter152. This is called enactive cognition. Intuitively, a human simplifies the world into abstract objects like “chair” and “pen”. We don’t think about details, just whole objects. We reduce the big world of all details to a small world of things which impact our survival153. Such concepts have emerged from the interaction between humans and our environment. They are “co-created”154.
PANCOMPUTATIONALISM
Computationalism, computational cognitivism or computational theory is the idea that mental processes are computational processes. From this point of view, artificial intelligence is the engineering branch of philosophy of mind. It is an attempt to formalise the systems that support mental processes and thus recreate them. This is hard to reconcile with enactivism because it presupposes some form of interpreter between the organism and its environment. It makes a firm distinction between the organism and its environment, where enactivism blurs the line between the two. In contrast, pancomputationalism is the idea that everything is computation, not just mental processes. Pancomputationalism is trivially true given a weak notion of computation155. More importantly, it does not require we make any distinction between the organism and its environment, so it leaves room to formalise enactivism.
Over the course of this thesis I will formalise enactivism in terms that are compatible with functionalist, computational ideas regarding the mind. To do so I will formalise the stack in which boundaries or interpreters are formed, rather than presupposing them. Unfortunately, notions like enactivism and relevance realisation and often considered to be at odds with computation156. Of course, that depends on what we consider computation to be. Some might consider computation to be just that which occurs in a human made computational system like an Apple Silicon M4 processor that uses ARM system architecture. Others might consider it to be more more abstract and general notion of Turing computation, in the sense of any machine which mimics the operations of a Turing Machine. Piccinini divides computation up into abstract and concrete sorts157. Abstract is whatever we interpret it to be, much like a mathematical symbol. Concrete is that which is physically manifest in the environment. It is this latter variety I’ll formalise, in order to describe the possible worlds that might exist.
EPISTEMOLOGY
To argue such a position is justified we must also consider how it is one might come to know anything. At the beginning of this chapter I spoke of the difference between explanans (explanation) and explanandum (the thing to be explained). Given an explanandum, there may be many equally plausible explanations. This is
particularly relevant when we are considering explanations of private knowledge, that cannot be easily verified by experiment. There are so many theories of consciousness for exactly this reason. Hence, when we consider theories of consciousness we need a means of evaluating explanations, to decide which is most plausible.
OCKHAM’S RAZOR
Something more complex is more difficult to understand or predict. Ockham’s Razor amounts to the idea that simpler explanations are more likely to hold true158, or are more likely to generalise. Yet simplicity is a measure of form, not function. As a subjective measure of how difficult something is to understand, complexity makes perfect sense. As a measure of something objective, namely how likely something is to hold true in future interactions with an objective reality (an environment of which one’s self is part), it makes little sense. Nevertheless, empirically it is the case that simpler explanations usually hold up better under scrutiny. One could perhaps interpret this as suggesting the environment is the product of one’s perception, or that there is something else going on. The important thing is that subjective perception of simplicity does correlate with empirical veracity. As part of this thesis, I explain why this is the case. I show this correlation is due to causal confounding159. In the context of the mind body problem,
PRINCIPLE OF INFERENCE TO THE BEST EXPLANATION
Not all hypotheses are equally “good”. If one explains why it rains, and another explains why it rains and why the sun rises, then the latter is a “better” explanation. It explains something that otherwise would not be explained. The Principle of Inference to the Best Explanation is merely that one should prefer “better” hypotheses161. Of course, such a principle is not without its critics162. One obvious problem is that we might construct an infinite number of hypotheses which are equally “good” according to the criterion given above, including some which are implausibly convoluted and specific. Yet the principle is still worth mentioning as, like Ockham’s Razor, it can help identify useful explanations.
STRUCTURALIST BRAINS IN VATS
Structuralism is the idea that words and ideas are not intelligible as isolated items, but become so through their interrelations. Those interrelations are the “structure” that structuralism refers to. For example, semiotics is the study of symbols. Saussure’s semiotics defines a symbol as a sign, for example a sound or visual pattern like the word “cat”, and a thing which is signified, called a referent. For the word “cat” the referent would typically be a cat, but of course that can change depending on context. According to Saussure, signs gain their meaning from their interrelations with other signs. Put another way, meaning is the difference between signs. Structuralism became extremely influential over the course of the 20th century, until the rise of a counter-movement called post-structuralism. For our purposes a notable post-structuralist was Derrida163, who argued that any structural description that seeks to fully encapsulate the semantics, truth or unmediated pure experience of something will be deferred or incomplete. He coined the term “différance” to describe this combination of difference and deferral. Structuralism and post-structuralism are particularly relevant given the recent success of language models. A language model like GPT-4 is optimised to learn the structure of language through their signs alone. The results have been impressive, lending credence to structuralism. However, just because a language model writes like a human does not mean it has understood the aforementioned semantics, truth or unmediated pure experience a human might have. I mention post-structuralism because my discussions with post-structuralists have proven useful. To answer my questions I’ll take a primarily structuralist approach, but one that seeks to acknowledge and formalise Derrida’s post-structural critique. If we want to know if a machine is truly intelligent, and has conscious experience like a human, then we must first answer what those things are for a human. We cannot begin at the level of any one concept. We must formalise the space of everything conceivable.
Over the course of this thesis I’ll discuss computational dualism, a criticism I published concerning software ‘intelligence’164. A brain in a vat can know only what it is fed by its senses. It has no idea what is objectively true. Likewise, a computer program can know only what it is fed through hardware. The “meaning” of code is entirely determined by the hardware on which we run it. If a computer program is a model of the world, then its accuracy depends on the interpretation of it. In other words if we’re to know what a
computer program knows, then the conventional distinction between software and hardware is going to have to be abandoned. We’re going to have to avoid computational dualism, and to do that we need to formalise the space of all conceivable environments and see what holds in all of them. I’ll argue every conceivable environment has at least one state. The power set of states is the set of every possible difference. There is no difference within a state, only between states, and differences are the programs of which aspects of the environment are formed. Intuitively, it doesn’t matter what states are because we assume nothing about them. After all a human can only interact with aspects of his environment. If he were to try and pinpoint what an aspect is made of, the answer would be deferred to other aspects. This is analogous to the treatment of foundational concepts in structuralism and post-structuralism. Any answer that sought to fully encapsulate the semantics, truth or unmediated pure experience of a state would, in the language of post structuralism, be always already delayed, deferred or incomplete. This does not render such attempts vacuous, but does speak to their inherent contingency and conditionality165. From there I take a firmly naturalist approach to explaining what might or must exist in every conceivable environment, working from first principles with pragmatic assumptions of natural selection and self-organisation.
PRAGMATICS AND THE ORIGINS OF OUGHT
The philosopher Hume famously showed a statement describing what ought to be cannot be derived from a statement of what is. This dissociation of value from description is named “Hume’s Guillotine”166. To take a naturalist approach to explaining everything, I need to dissolve Hume’s Guillotine by showing where an original ought comes from, to get natural selection. Finally, I’ll take a moment to describe an alternative to Saussure’s structuralist semiotics. Note that Saussure’s symbols were dyadic, meaning they contained two parts. A sign, and a referent. In contrast the semiotics of Peirce defines a symbol as triadic. A Peircean symbol as a sign, a referent and an interpretant. The interpretant is the effect of the sign upon the person who interprets it. For example, if I see the word “cat” and feel hungry, this has implications for what I’ll do next. Such a pragmatic, consequence oriented account of symbols is useful for a naturalist account of meaning. I’ll dispense with “is” by arguing the very fact of continued existence constitutes an ought from which purpose and behaviour follow. As part of that I’ll formalise meaningful communication in terms of pragmatics, namely Gricean theories of mean-
ing167. Grice held that the meaning of an utterance168, is whatever the speaker intends the listener hold in their mind as a consequence of listening. Likewise, the listener has understood the meaning of an utterance if they come to hold in their mind approximately what the speaker intended. As an unintended consequence of following the thread from first principles to try and explain consciousness, the formalisation I’ll present just happens to align with both Peircean semiotics and Gricean pragmatics, unifying the two.
This chapter brought together many ideas. The rest of this thesis tells a more straightforward story, from beginning to end. The next chapter is yet another survey, but it tells a nice story about a bitter lesson.
III. WHAT THE F*CK IS AGI?
CONTEMPORARY AI SYSTEMS ARE NARROW169 170, brittle, and proficient only within stable environments. Artificial General Intelligence (AGI) represents the pinnacle of artificial intelligence research: a machine that learns and adapts with the ferocity of a human mind171.
Many peg AGI to human-level performance across a broad range of tasks172. I myself have done this. It is a cozy, intuitive benchmark. It is also anthropocentric and so vague it’s practically a Rorschach test. This definition is insufficient, but arguably necessary. Human intelligence has many aspects. Some have emphasised autonomy, agency, and a balance of exploration in search of knowledge against exploitation of that knowledge173. AGI is not a passive observer of the world but part of it. As Pearl puts it, a truly intelligent agent must surmount a ‘ladder of causality’174. It must discriminate between events it has caused and events it merely observes. It must evaluate counterfactuals and imagine entirely alternative paths to the same end. Certainly these are all necessary for AGI. At a higher level,
I give two testable definitions. The first is a quantifiable definition of intelligence 183. It is the ability to complete a wide range of tasks: a nod to Legg-Hutter intelligence, but most closely aligned with Wang’s definition. It deals in systems as a whole. If system
Hence I define
This isn’t just about
EVERYTHING IS A BITTER LESSON
Having defined what this thing is I now need to say how anyone hopes to get there.
To solve a problem I can hand-craft clever solutions to problems, or I can apply general methods like search or approximation and just optimise for what I want190. If resources are not a consideration, then general methods will eventually beat any approach that relies on human-crafted knowledge or structures. AI started to be of practical use because hardware improved to the point where AI could be applied at scale, not because anything significant changed with the algorithms. The Bitter Lesson gives you The Scaling Hypothesis. The Scaling Hypothesis asserts that by amplifying the size of AI models, the volume of training data, and the computational power deployed, we’ll eventually rival or surpass human capabilities. The Scaling Hypothesis has surged in prominence, fueled by the striking achievements of large-scale models across diverse domains. For example, OpenAI’s GPT-3, boasting 175 billion parameters, showcased remarkable proficiency in generating human-like text, executing tasks with minimal prompting, and even hinting at basic reasoning191.
Likewise, DeepMind’s AlphaFold 2 harnessed vast computational resources and biological datasets to revolutionize protein structure prediction, solving a decades-old challenge in biology192. These breakthroughs demonstrate that scaling does get results, at least to an extent. Empirical support for the scaling hypothesis is bolstered by scaling laws, which reveal predictable performance gains as model size, data, and compute increase.
There are critics. I count myself among them. While scaling might eventually work, the word “eventually” is doing a lot of work. There are diminishing returns. Beyond a certain threshold, additional parameters yield only incremental gains, suggesting a ceiling to this approach. In language models, performance gains taper off as size increases. Marginal improvements don’t justify exponential resource costs. This plateau challenges the notion that scale alone can bridge the gap to AGI. The environmental toll of training behemoth models is staggering, with carbon emissions rivalling those of small industries194. This is exacerbated by the fact that scaled models excel in their training domains but often falter beyond them. Large language models generate fluent text yet stumble on tasks demanding deep reasoning or contextual nuance195. Some suggest this neural networks are fundamentally incapable of reasoning or causal understanding196. I don’t know about that. A full human brain integrated with a human body is quite spectacular. A chunk of human brain sitting on a counter-top tends to be rather ghoulish and unimpressive. I do know these systems are sample inefficient, meaning they need a lot of data or many ‘examples’ to learn from. That is a criticism I find compelling. Adaptability is about dealing with edge cases, not rote learning. A system that needs a data centre to learn tic-tac-toe isn’t intelligent: It’s a whale beached on silicon. Finally, scaling assumes you know what you want and can measure it. That is quite the assumption. We can mimic human behaviour, but is that really what we want? To replicate ourselves?
The Scaling Hypothesis is potent. Yet it is not a silver bullet. Empirical success must be weighed against diminishing returns, theoretical gaps, and ethical trade-offs. To understand how we can do this, we must examine what exactly it is that we’re scaling. The typical ML and AI concepts like supervised learning, reinforcement learning, inference, reasoning, planning and so on aren’t useful because an artificial scientist must able to do all of it. Instead I will take my cue from Sutton’s bitter lesson and speak only of the means by which these things are achieved. These means are the search and approximation. This is not the only way to think about this, so I then discuss hybrids. Hybrids are those systems which do not fall neatly into the buckets of search and approximation. Finally, I discuss meta-approaches, which are frameworks through which search, approximation and hybrid systems can be understood. Meta-approaches give us a quantifiable answer to ‘what is intelligence’ that other systems can optimise for.
BASIC TOOLS
SEARCH
SEARCH IS THE HISTORICAL WORKHORSE OF AI197. I include any symbolic reasoning and planning in this bucket. In its most basic form search involves representing a problem space and solution criteria. Then every nook and cranny of the problem space is explored and tested until until a solution is found. Rooted in the foundational era of computation, search-based methods embody the belief that intelligence can be distilled into systematic exploration of well-defined possibilities. This section dissects search-based AI. I discuss operational principles, its strength in structured domains, its limitations in the face of complexity, and its fit within the broader quest for AGI. It is a precision instrument. At its core, search-based AI is about exhaustive exploration. Whether it’s planning a route, solving a puzzle, or proving a theorem, search involves a representation of the problem’s state space (often as a graph or tree). A search algorithm then systematically traverses it, evaluating paths against a defined goal. This is the essence of algorithms like breadth-first search (BFS), which explore all nodes at the current depth before moving deeper. Depth-first search (DFS) dives deep into one path before backtracking. A more sophisticated method called A*198
Search has its advantages. Here are a few:
- OPTIMALITY: When properly configured (e.g. with an admissible heuristic in A*
A star ), search algorithms guarantee the discovery of the optimal solution, provided one exists. This is invaluable in domains where precision is non-negotiable, such as automated theorem proving200 or mission-critical planning in aerospace. - INTERPRETABILITY: The process is transparent. Each step can be traced and understood. That makes search-based systems easier to debug, verify, and trust than their approximated counterparts.
- STRUCTURE EXPLOITATION: Search excels in problems with well-defined structures, where the state space, though potentially vast, is navigable through clever pruning and heuristic guidance. This makes it a go-to for tasks like game playing (e.g. chess engines pre-AlphaGo) and pathfinding in robotics.
These strengths have cemented search as a cornerstone of AI, particularly in environments where correctness and transparency are paramount.
HOWEVER, SEARCH ALSO HAS DRAWBACKS:
- COMBINATORIAL EXPLOSION: The primary curse of search is its scalability. For problems with large state spaces, the number of possible paths grows exponentially, a phenomenon known as the combinatorial explosion. Even with heuristics, search can become computationally intractable for all but the most carefully constrained problems. In chess the state space is approximately
ten to the forty sixth nodes. This is too large for brute-force exploration without aggressive pruning. Prior or contextual knowledge can be used constrain the search space and mitigate this problem. - SEQUENTIAL NATURE: Search algorithms are sequential, making them ill-suited for modern parallel hardware like GPUs, which thrive on matrix operations and batch processing. This puts search at a severe disadvantage compared to approximation-based methods, which can leverage massive parallelism to accelerate learning and inference. Concurrent and distributed search algorithms exist, but have not yet matured into user friendly and scalable libraries201.
- RIGIDITY IN PROBLEM FRAMING: Search demands a pristine problem definition. This means explicit states, transitions, and goals. Real-world problems are often riddled with uncertainty. Search falters in these environments, requiring human intervention to massage the problem into a tractable form. This reliance on human pre-processing is a far cry from the autonomous adaptability we seek in AGI. However this ceases to be a significant problem if search can be made more efficient and scalable.
In its current form, search-based AI is a perfectionist that thrives in controlled, sterile environments but wilts when faced with the chaos of reality.
Search has a few notches in its belt:
- SATPLAN: By converting planning problems into SAT instances, SatPlan has solved complex logistics and scheduling tasks with precision202. However, its reliance on well-defined constraints limits its applicability to more fluid, real-world scenarios.
- CHESS ENGINES (E.G. DEEP BLUE): Chess engines like Deep Blue203 relied on search algorithms augmented with evaluation functions to defeat world champions.
- PATHFINDING ALGORITHMS:
A star and its variants remain the gold standard for navigation in robotics and video games204, efficiently plotting optimal routes in static environments. But again, their effectiveness diminishes with increased uncertainty and dimensionality.
These examples underscore search’s prowess in structured domains while highlighting its limitations in more complex, adaptive settings.
APPROXIMATION
By approximation I mean curve fitting. I mean all those artificial intelligence techniques that address complex problems by approximating underlying functions, distributions, or decision surfaces, rather than relying on exhaustive computation or exact solutions. Unlike search, approximation-based approaches excel in environments with high dimensionality and noise. Computer vision and natural language processing systems depend heavily on approximation. This section briefly examines its defining characteristics, advantages and limitations. At its core, approximation-based AI optimises a model reflect patterns in data so it can be used to make predictions about other data generated by the same source. In its simplest form this would be like writing down and averaging someone’s score in a game so you can predict what they will get in future. There is something that generated data (the player and games), and you train a model (by taking the average) until it reflects some aspect of the generator. I can train a model to classify data, answering ‘which thing generated this data?’. I can also train a model generate new data.
Typically a parameterized model such as a neural network approximates a target function by minimizing a loss function over a training dataset. Mathematically, given an input space
Approximation has advantages over search:
- scalability: These methods efficiently process large-scale, high-dimensional data. For example, convolutional neural networks can
classify millions of images by learning compact feature representations, bypassing the need for exhaustive hand-crafted rules.
-
ROBUSTNESS TO UNCERTAINTY: By modeling data distributions probabilistically or incorporating regularization, approximation-based models can generalize from noisy or incomplete inputs. Techniques like dropout in neural networks208 or Bayesian methods enhance this resilience, making them suitable for applications like speech recognition in variable acoustic conditions.
-
FLEXIBILITY AND AUTOMATION: Search often requires a domain-specific heuristic. Approximation is cheaper, which means it can learn directly from data. This is ideal for problems where the relationship between inputs and outputs is highly non-linear or poorly understood. It can minimise the need for human-engineered features. This adaptability has fueled its adoption in fields from genomics to finance, with minimal reconfiguration.
Scalability in particular has led to widespread adoption, as you might expect given the bitter lesson. Some examples:
-
CONVOLUTIONAL NEURAL NETWORKS (CNNS): CNNs exploit spatial locality and parameter sharing to achieve state-of-the-art performance in visual tasks209.
-
TRANSFORMERS: Transformers rely on self-attention mechanisms to model long-range dependencies in sequences210. Models like BERT211 and GPT-3212 have set benchmarks in natural language understanding and generation, leveraging massive datasets (e.g. GPT-3 was trained on 45TB of text) to approximate linguistic structures.
-
DEEP REINFORCEMENT LEARNING: Deep Q-Networks (DQN)213 combine neural networks with Q-learning to approximate value functions, achieving human-level performance in Atari games. Similarly, Proximal Policy Optimization214 has advanced policy approximation in continuous control tasks.
These examples highlight the ability of approximation-based AI to address diverse challenges with tailored architectures.
Despite recent success, approximation is not a panacea:
- UNRELIABILITY: Approximation is only approximate. Stochastic. It is unreliable by design215. This makes it difficult to apply to problems where failure cannot be tolerated. This is why search is used for applications like maps and directions. Directions that ‘approximate’ a route through a river are not useful.
- INTERPRETABILITY: The complexity of models like deep neural networks, often with millions of parameters (e.g. GPT-3 has 175 billion), renders them opaque. This “black box” nature complicates understanding of decision rationales, a critical issue in domains requiring accountability, such as medical diagnostics or legal systems. Efforts like LIME216 and SHAP217 provide post-hoc explanations, but these are often approximations themselves and lack the rigor of causal insight.
- SAMPLE INEFFICIENCY: High performance hinges on access to large, labeled datasets. For instance, training ResNet-50 on ImageNet requires 1.28 million labeled images, while GPT-3’s training consumed computational resources equivalent to thousands of GPU days218. In data-scarce domains, such as rare disease diagnosis, this dependency limits applicability and risks overfitting, where
f theta fits noise rather than signal (bias-variance trade-off). In other words, approximation is maladaptive. Techniques like transfer learning can mitigate costs, but performance still drops sharply outside the training distribution219. - COMPUTATIONAL COST: The training of approximation-based models incurs substantial energy and infrastructure demands. For example,
Strubell et al. (2019) Strubell and colleagues 220 estimate that training a single transformer model emits carbon equivalent to 626,000 miles of car travel, raising sustainability concerns.
These drawbacks underscore the trade-offs inherent in approximation, necessitating careful consideration of context and resource constraints.
HYBRIDS
Hybrids are those systems which do not fit neatly into the search or approximation buckets. Biological self-organising systems learn and adapt, but they are not clearly a case of just search or approximation. Hybrid approaches are inherently more general because I can pick choose any general approach for any occasion. I can fuse search and approximation, or something else. By combining complementary strengths, hybrid systems offer a tantalizing path toward AGI, promising robustness where monolithic approaches falter221. Perhaps no single AI paradigm holds the key to AGI. Search excels at precision. Approximation thrives on raw data and uncertainty. Hybrid systems bridge these gaps, blending precision with flexibility, logic with learning. The goal? Synergy. Emulate humanity’s versatility, tackling everything from sensory processing to scientific discovery.
Hybrids take many forms. AlphaGo222 is the simplest example of how approximation and search can complement one another. This hybrid crushed Go’s world champion in a testament to blending search and approximation223. Search allowed it to plan sequences of moves that conformed to the rules of Go, while approximation allowed it to figure out which sequences of moves were most likely to win. Hybrids can also take the opposite approach. Neuro-symbolic hybrids tackle the symbol grounding problem by linking raw data to abstract concepts224. Think neural nets mapping inputs to symbols, then reasoning over them. Structured reinforcement learning hybrids use this kind of approach, using approximation to process sensory data and search to choose actions. Raw, high-dimensional sensory data is too much for search to cope with, so approximation ‘reformats’ it into a simpler, structured, low-dimensional symbolic representation. In this case a convolutional autoencoder learns to ‘compress’ the raw data down to a small size and then back again, ensuring important information isn’t discarded by converting the sensory data to the smaller format. The low dimensional data are clustered and labelled as ‘objects’ with properties based on geometry and where they are on the screen. These objects can then be tracked as the world changes over time, to get learn their dynamics and spatial interactions. More conventional reinforcement learning techniques are then applied to learn a policy in these highly abstracted, symbolic terms. The resulting agent adapts far more efficiently225. Finally and most importantly there are fully autonomous, general purpose systems. Cognitive architectures like SOAR226 and ACT-
R227. These weave search and approximation together for flexible, multi-task competence. The most prominent examples are ongoing projects that have shown steady improvement year on year:
- HYPERON: Probabilistic logic networks meet neural nets in a bid for holistic cognition. Perception, memory and reasoning in one package. It aims to build AGI on a modular, distributed, self-organising system that can integrate new technology as it develops228. For example, new components have been proposed based on active inference and the free energy principle229.
- AERA: The Autocatalytic Endogenous Reflective Architecture (AERA) self-programs, reflecting on its own symbolic structures while learning statistically. It’s a stab at autonomy and growth230.
- NARS: The Non-Axiomatic Reasoning System (NARS) rejects rigid axioms for a fluid, adaptive logic. NARS operates under the Assumption of Insufficient Knowledge and Resources (AIKR), reasoning with incomplete, uncertain data via a non-axiomatic framework. It integrates symbolic reasoning with probabilistic inference, using a custom inheritance-based logic (NAL) to derive conclusions from limited evidence. Designed for real-time adaptability, NARS learns incrementally, refining its knowledge base as new inputs arrive—think of it as a brain that thrives on ambiguity, not a theorem prover shackled to certainty231.
Hybrid systems give us the best of all worlds. Fusion of search and learning is a general approach that can be scaled. Hybrids are also more useful in the short term. Structured priors or search can narrow the problem space, improving sample and energy efficiency compared to brute-force approximation. The high-level symbolic abstractions often used for search are interpretable by humans. Conversely, we can easily integrate human priors into hybrid systems. Hybridisation can be a shortcut to autonomous agents. Hybrids can combine a persistent identity and interpretable goals with the ability to process raw, high-dimensional real-world environments. For example, scaffolding like memory can enable long term adaptation in ontologically stateless language models232. Hybrids edge us closer to AGI by mimicking diversity of human cognition. Yet I have lingering questions. What is missing? Is a given hybrid system truly scalable or just a clever patchwork that exemplifies Sutton’s bitter lesson? Can we scale these systems to AGI?
META-APPROACHES
A meta-approach is a frame through which systems can be understood. It is a guiding principle I can use to tweak search, approximation or hybrid systems to be more ‘intelligent’. Meta-approaches are not mutually exclusive. The scaling hypothesis is an example of a meta approach through which I have framed search and learning. I call this scale-maxing because it works by maximising scale. For example, maximising the amount of training data, the avaible compute and the size of the model. There are two other meta-approaches I can identify. One is orthodox at prominent AI labs like Deepmind and OpenAI. I call it simp-maxing because it involves maximising the simplicity of forms. It is founded on Ockham’s Razor. For example, if I have a perfect compression algorithm and I use it to compress two files, then the smaller compressed file is the simpler one even if the uncompressed files were the same size. Likewise, if I use regularisation to make regression converge on a simpler function, then I am simp-maxing. The last meta-approach is my own invention, which I propose in this thesis233 234. I call it w-maxing because it optimises for the least specific, weak constraints on functionality at the lowest possible levels of abstraction. So to reiterate, scale-maxing is about maximising available resources, simp-maxing is about maximising simplicity of forms, and w-maxing is about maximising the weakness of constraints implied by function. In this section I will focus on simp-maxing, and will explain my stack-based approach later.
Simp-maxing is about applying Ockham’s Razor to make more accurate models235. It posits that among competing hypotheses which might explain some observed data, the simplest one is most likely to be correct. In AI, this translates to favouring models or solutions with lower complexity, as they are less prone to overfitting and more likely to capture the underlying structure of the problem. Examples of simp-maxing include regularisation236, the minimum description length principle237 and Universal Artificial Intelligence (UAI)238. UAI is the dominant mathematical formalisation of artificial general intelligence. It relies on Kolmogorov complexity239, which defines the complexity of a string as the length of the shortest program that can generate it. For a dataset
This serves to illustrate what a meta-approach is. It provides a guiding principle for adaptability and generalization. A meta-approach can be applied in the context of search or approximation.
CONCLUSION
I’ve defined intelligence in terms of adaptation, AGI as an artificial scientist and laid out some of the tools available for that quest. These include search, approximation, hybrids, meta-approaches, and the relentless march of scaling.
Foundational tools:
- SEARCH (e.g. navigation apps, DeepBlue),
- APPROXIMATION (e.g. GPT-3, deep Q-learning).
Hybrids:
- SIMPLE (e.g. AlphaGo, structured reinforcement learning),
- COMPLEX (e.g. Hyperon, AERA, OpenNARS246).
Meta-approaches:
- SCALE-MAXING: maximise available resources (e.g. OpenAI’s GPT series LLMs),
- SIMP-MAXING: maximise simplicity of forms (e.g. regularisation247, UAI248, minimum description length principle249),
- W-MAXING: maximise weakness of constraints implied by function250.
EACH OFFERS A PIECE OF THE PUZZLE. With sufficient resources any system that learns can eventually attain an arbitrary level of skill. Every system can be optimal. However not all systems are equally adaptable. Hence I’ll conclude this chapter by reiterating that intelligence is a matter of adaptability, and thus efficiency. All else being equal, the more resources the system needs to reach a certain level of performance, the less intelligent it is. What I offer in this thesis is a meta-approach that lets us measure and maximise adaptability.
IV. WOW, EVERYTHING IS COMPUTER
There is a problem with simp-maxing. It works, but there is no apparent reason it should. After all, the No Free Lunch Theorem shows no algorithm outperforms others across all problems251252. Indeed, it turns out AIXI’s performance is entirely subjective253. The root of this subjectivity is in the definition of Kolmogorov complexity. For a given string
where
This is the invariance theorem254. While this suggests that the difference in complexity is bounded, the constant
a short program that Legg-Hutter intelligence interprets as a long
program. For instance, a string that appears simple relative to one
Turing machine might seem complex relative to another, depending on the machine’s instruction set or encoding scheme. Consider
the analogy of programming languages, which can be thought of as
different universal Turing machines. Suppose we have two programming languages,
directly generates the Fibonacci sequence, while
consider a string
In print_fib(100), making
in
sequence from scratch, resulting in a much larger
same string
chosen language, illustrating the subjectivity introduced by the choice
of reference machine.
REFRAMING THE PROBLEM
AGI is supposed to be capable of adapting to any task or environment. However, if its internal measure of simplicity is tied to a specific, arbitrary choice of reference machine, its adaptability may be constrained by that choice. This could lead to blind spots or inefficiencies in certain domains, undermining the goal of general intelligence. AIXI illustrates an extremely valuable idea, but its subjectivity is a problem. Some have explored complexity measures that are invariant under certain transformations, aiming to reduce dependence on the reference machine. For example, Levin complexity258 incorporates time complexity into the measure, potentially offering a more universal metric. However this is still a measure of form, not function. Simp-maxing is not optimal, and if I want to understand intelligence I need to know what I’m aiming at. I want to know what the upper bound on adaptability is.
To solve this problem I need to go down a level of abstraction. I need to reframe the problem. Kolmogorov complexity takes information in one format
MORTALITY
Descartes thought it was ‘animal spirits’ and the pineal gland passing messages between mind and body. AI research has replaced the pineal gland with a Turing machine. It is a ghost haunting AGI labs like it’s 1637. Software is the mind, hardware is the meat, and never shall they meet. Computational dualism is the idea that artificial intelligence is about creating intelligent software263. It is not. That should be obvious. Software is a state of hardware. AI is about making an intelligent system, and its state its part of it. Computational dualism is a problem if we want to build an intelligent system, because it ignores half the equation. We need to know what intelligence is if we want to optimise for it. We need to know what optimal looks like so we can work towards it.
It is not as if people haven’t already pointed out there’s a problem, it is just that they have chosen to live with it264. As far as I can tell, only one other has gone to the trouble of attempting to formalise an alternative265. Orseau attempted to formalise a version of AIXI which was interpreted by the environment, using bounded optimality. It is a commendable and compelling attempt, but it does not go far enough. It does not answer the questions I want answered. Symptoms of computational dualism remain. Software is a convenient abstraction and it works well for building standardised applications for standardised hardware in standardised contexts. It becomes more of a problem when we consider an agent interacting with the world. Many appear to have forgotten software is nothing more than a state of hardware. It has spawned wild AGI myths, like superintelligent code rewriting physics or escaping its box266. Even Nobel laureate Geoffrey Hinton has resorted to doomsaying267.
He speaks of mortal vs immortal computation as if software lives on in the absence of hardware268. His arguments seem to hinge on software’s ability to leap from one hardware platform to another, retaining its functionality like some eternal digital essence. He suggests that because software can be duplicated across different machines without losing what it does, computation somehow transcends its hardware. At first glance, it seems plausible. Copy your code, run it elsewhere, and the process lives on. But that’s an illusion. Software is a state of hardware. When the hardware changes or fails, the computation changes or fails. Copying doesn’t preserve the original. It births a new instance as mortal as the last.
Hinton’s immortal computations don’t exist. There is only mortal software, because there are only finitely many devices. Software is a state of hardware, a pattern etched in silicon or flesh. Change the hardware, and the ‘mind’ follows. Human intelligence is embodied, not just symbol shuffling in the abstract269. Software is no different. The analogy of the brain as a computer only works if you treat it as a unified system with no distinction between hardware and software270. Intelligence isn’t a disembodied mind interacting with but a dance between hardware and environment. Cognition is a physical act, not a ghost in a shell271. Every physical system computes simply by existing272. It is a whole-of-system physical process. Hence, I take a whole-of-system approach. Hardware, software and world are entwined.
FLIPPING THE TABLE
Computational dualism is a dead end. Software is a state of hardware. Hardware is a part of a larger system. Here I argue everything is nested abstraction layers from software to hardware to the laws of physics. This brings together several of my papers273. To quote a certain ambulatory meme, everything is computer.
Software doesn’t exist. At least, not in the way people seem to think. A Python script relies on an interpreter written in C. C is compiled into assembly. Assembly to machine code. Machine code is hard-wired in silicon. We don’t ‘run’ software. We flip switches in a box. As AIXI illustrates, code is nothing if it is not etched in meat or metal. Same code, different rig, different mind. Why? Because a body isn’t a neutral translator. Hardware is goal-directed, just like software is. Some hardware is better for some tasks. It is an abstraction layer.
Abstraction does not end at hardware. Hardware is not the bedrock. Intuitively, think of a play. Software is a script, the hardware is the actor and the environment is the stage. Change the actor or the stage, and the show is not the same. An actor is not the play. Making hardware the foundation would repeat the mistake of computational dualism. Hardware is a body embedded in the world and bound by physics274. A CPU is a hunk of matter obeying laws we’ve barely glimpsed. Physical laws, only they are not really ‘laws’. That is just how we understand what is happening275. We scribble on blackboards. We approximate reality. Whatever it is that those laws approximate, that is hardware’s puppeteer. Nature’s machinery. A transistor flips because nature says so, not because some coder waved a wand. Hardware is a middleman, enacted by something less abstract. Hardware is just another abstraction layer, like software.
Put another way, reality is a Matryoshka doll. Software is a state of hardware, which is a state of the physical reality we inhabit. A human is a state of organs, which are states of cells, which are states of molecules276. Each level is a state of the one below. Everything a program running on a deeper machine.
I call it The Stack. It is patterns within patterns277. I’ll return to my function analogy. The mind is
THE LIMITS OF KNOWING
How far down can the stack go? Gravity, quarks and spacetime sound foundational, but they are human abstractions. Our physics is a guess scrawled in chalk by apes279. We know it is incomplete. Our formulae are programs running on a human brain. They are not fundamental280. Even numbers are not fundamental. Numbers are just a means by which we describe the order we perceive. Could there be more? Maybe
Does this mean we are condemned to subjectivity? Solipsism? Intelligence is a whole system282. It is the whole stack in motion. To understand intelligence we must rethink reality.
Our physical laws are models. They are programs we’ve written to predict nature’s machinery. We are the computers on which those programs run. Our tools are built for our slice of reality, not the whole pie283. There could be
I’m going to propose a definition of environment that holds for every environment. It is the foundation of what I call Stack Theory284. It is a frame that holds no matter where the bottom lies. It’s not about finding
ALL POSSIBLE WORLDS
Here I lay the foundation of cognition within Stack Theory. Cognition within Stack Theory is enactive285 and pancomputational286 287. Pancomputationalism says all physical systems are computational. Enactivism frames cognition as emerging from dynamic interactions between the system and its environment.
Stack Theory’s foundation is the environment. More specifically, it is a definition I name ‘environment’, but really it is what is common to all environments. All possibilities for an ‘underlying physics’. It is based on premises I refined over the course of several publications288. In those papers I called them axioms, but I’m no longer sure that description fits. They don’t depend on anything. They are not assumptions. In the first ‘axiom’ I merely define what I mean by environment. The second is a tautology.
- AXIOM 1: Where there are things, I call them the environment.
- AXIOM 2: If things change, then the environment has states.
What is a state? At the very least, it is a difference. If nothing changed then there could be no states. Without states the environment can only be some sort of unity or oneness. There is nothing in it we could point to. It just is. Perhaps even that is arguable. If something has no state and no content, is it anything? There must be difference for there to be something. Since there must be difference, there must be states. If one thing changes, then there must be two states. Before, and after. We don’t know what the thing is or what states are, and we don’t need to. That is unnecessary detail. All we need to know is that there is a difference between states. I don’t presuppose the environment is made up of objects or properties. Because of this there is a different, equivalent axiom we might use.
- ALTERNATIVE AXIOM 2: Time is difference.
Every state is point of difference is a different time in a particular timeline. This means states are mutually exclusive within a timeline.
Definition 1 (environment)
- We assume a set
Phi whose elements we call states. - A declarative program is
f, a subset of Phi , and we write for the set of all declarative programs (the powerset ofPhi ). - By a truth or fact about a state
phi , we mean such thatf in P such that phi is in f . - By an aspect of a state
phi we mean a set of facts about s.t.l, such that phi is in the intersection of l . By an aspect of the environment we mean an aspect of any state, s.t.such that the intersection of l is not empty . We say an aspect of the environment is expressed, realised289 or embodied in statephi if it is an aspect ofphi .
EVERYTHING THAT IS OR MIGHT BE MUST FALL WITHIN THE SCOPE of what this formalism can describe. Yes, my formalism is still an abstraction. However, some claims are so weak they are true of everything290
WE HAVE A SET OF STATES
A TRUTH OR FACT ABOUT A STATE
THE ENVIRONMENT ENCODES EVERYTHING through its state space. Whether objective or subjective every object, property, and goal is an aspect of the environment. An aspect of a state is a collection of facts
that all hold for that state, such as {light is on, door is closed} for a state where both are true. An aspect of the environment is one that holds for at least one state, and it is realised by a state if that state satisfies all the facts in the aspect. This formal structure allows us to model everything as programs, aligning with pancomputationalism’s view that all physical processes are computational291.
TOY EXAMPLES
To demonstrate the framework’s generality, consider several examples from diverse domains. These illustrating how the environment can represent different systems. Now, in reality we don’t know what states contain. We see perceive them through a possibly infinite stack of abstraction layers. However, for the sake of example lets assume we are omniscient. This lets me use the framework to describe toy problems and ‘real world’ examples. In reality
Light Switch System: Let
Grid World in AI: In a grid world,
Biological Cell Metabolism: A cell’s environment includes metabolic states (e.g., healthy, stressed, dividing) and external conditions (e.g., nutrient levels). Let
These examples illustrate the framework’s flexibility, applying to digital, biological, and social systems, each with distinguishable states and goals.
V. TURTLES ALL THE WAY DOWN
So far I’ve framed the environment as a set of states
Embodiment gets overlooked in computer science. That is why we have computational dualism. Anecdotally, when I have presented this research at conferences many of the questions I received straw-manned embodiment. The implication was that embodiment was a matter of sentimentality, or that I was arguing there is something non-computational about intelligence. After all, some proponents of enactive cognition believe that computation and true enactive cognition are incompatible294. However embodiment as I speak of it is just a fact of existence. Every body, whether it be human, machine, or a slab of granite, throws its weight around dictating what can happen next. A rock doesn’t care about your feelings, but drop it in a pond and the ripples tell a story. I am framing this as a kind of ontological “speech”. Not poetry, but a formal language baked into existence itself. Think of it as ontology with attitude. Entities say something by being what they are.
LAYER CAKE
The environment speaks in physical terms. Recall the definition of environment from the previous chapter:
- We assume a set
phi whose elements we call states. - A declarative program is
f, a subset of phi , and we writeP for the set of all declarative programs (the powerset ofphi ). - By a truth or fact about a state
phi , we meanf in P such thatphi is in f . - By an aspect of a state
phi we mean a setl of facts aboutphi s.t.phi is in the intersection of l .
By an aspect of the environment we mean an aspectl of any state, s.t.the intersection of l is not empty . We say an aspect of the environment is expressed, realised295 or embodied in statephi if it is an aspect ofphi .
If every physical system computes296, then every physical system embodies a formal language. The environment is a physical system, so that means I should be able to re-frame it as a formal language.
Definition 2 (abstraction layer)
By abstraction layer298 I mean:
- We single out a subset
v, a subset of P which we call the vocabulary of an abstraction layer. The vocabulary is finite unless explicitly stated otherwise. Ifv equals P , then we say that there is no abstraction. L v, defined as the set of subsets l of v such that the intersection of l is not empty is a set of aspects inv . We callL v a formal language, andl in L v a statement.
- We say a statement is true given a state iff it is an aspect realised by that state.
- A completion of a statement
x is a statementy which is a superset ofx . Ify is true, thenx is true. - The extension of a statement
x in L v isE sub x, defined as the set of all y in L v such that x is a subset of y .E sub x is the set of all completions ofx . - The extension of a set of statements
X, a subset of L v isE sub capital X, defined as the union of the extensions of its elements . - We say
x andy are equivalent iffE sub x equals E sub y .
Our nand gate is embodied in silicon with inputs
- states: where each value (e.g. ) denotes a set containing all states in
Phi where and equal those values300, and contains all other states301. - vocabulary: s.t.
Given a nand gate I can build a computer305. The nand is the basic building block of all computers today.
I return to Grid World for another illustrative example. Because we now have this formal definition of abstraction layer, we can consider how Grid World exists within our reality, rather than as a separate, simplified reality. In other words I assume there is a machine in the environment that computes Grid World. That machine is built out of nand gates that together form an abstraction layer for Grid World. Lets not worry about
just declarative programs, meaning positions
Every body carries a vocabulary, a subset knee bent. It is knee bent. No middleman but the thing itself. The environment sets the rules and calls the shots. It cycles through states one at a time within the confines of a given world… or branching into many worlds309, either works but for the sake of explanation I will confine myself to one particular timeline. Each wall close, pivot left doesn’t fit in its embodied circuitry. It can’t turn left. It can’t represent left. It can turn right. The screech of its servos turning right as the sensor pings. That motion is the statement, alive in the grind of metal on floor311. No just the pondering, but the doing. This dodges old traps like
Bodies don’t represent reality. They are aspects of it.
SUBJECTIVE AND OBJECTIVE
When a body expresses a statement
Now if I were omniscient, the environment would have one state at a time because time is difference. That state would determine what is true at that time. That would mean some programs would return true at that time, and I could know the rest to be false. Truth would be binary. The world deterministic. Everything that exists is a statement made in an environment’s embodied formal language, and which statements are true depends on the state.
But I am not omniscient. From my subjective perspective within my environment, I cannot know what the physical state is. I cannot see all the statements. I am a statement, and I exist for as long as the environment expresses me. The environment might be objectively deterministic, but from my subjective point of view it is non-deterministic. There are many possible futures. ‘Many worlds’ in which I may find myself, like Everett’s interpretation of quantum physics313. Every statement
By expressing a statement
Consider a human raising its arm. This sets in motion a particular future. When a body moves, it embodies a statement. A statement
Each body has a vocabulary. A human is a chaotic symphony. A rock grunts single syllables. But each fits into the larger machine that is the environment. Computation here is the interaction of the body with its world. It affords the surround environment something317. The world offers possibilities tailored to a body’s shape. A chair yells “sit” to a human, not to a boulder. The statements a body can pull off depend on what the environment hands it. This aligns with ideas like polycomputation, that a computation at one scale can perform an entirely different role as part of a computation at a larger scale318. The same matter is part of many larger and smaller computations. This is a rejection of both the old computational mind319, and strong enactivism that holds cognition to be non-computational320.
MATRYOSHKA DOLLS
A statement is a set of programs, but it is also equivalent to a program that has the same extension. Formally, I mean for every statement
Definition 3 (abstractor function)
Naturally I could also do this with a more constrained extension, taking into account other parts of the environment and how they constrain
CONCLUSION
The
VI. MASTER, WHAT IS MY PURPOSE?
This chapter is about purpose. It is based on the latter parts of my papers on abstraction layers327, tasks328 and consciousness329. What is normative? What ought to be? David Hume, fond of Guillotines330, said one cannot smuggle an ‘ought’ out of an ‘is.’ This leaves me in a pickle. If I am to build a conscious machine, presumably it must have a moral compass. Where do we anchor its sense of “should”? I could argue it is anchored it in satisfying homeostatic and reproductive needs, but then where do they come from? I’m a naturalist, not a vitalist. I need something more fundamental than mere life. Besides I want to explain life, not assume it. Hence I’m going to argue there is no “is”, only “ought”. Some things exist. Others do not. Is this a normative judgement? I say it is. What else can it possibly be? Ought stems from change, and change is time. Not just ticking away like some bored clock, but calling the shots on what sticks around and what gets yeeted into the void. Creation and destruction. It is not just about when but what lasts. Time sifts the wheat from the chaff, and what hangs on gets the cosmic thumbs-up.
Many have sought to patch the gap between is and ought. Some say ought is a matter of feeling (puts the cart before the horse), or social contract (arguably come from feelings), or divine memo (god did it). I find these lacking. Change seems more foundational. Fundamental, if anything is. Without change or difference, everything would be the same thing. If everything is the same, can you really say there is anything? Is there an environment if there are no things? I say no. There would be nothing. Just an irreducible oneness. It is hard to conceive of it as an internally consistent idea. To comprehend it, must I cease to exist as an observer? Becoming one with everything is beyond the scope of my thesis. Difference or change must be fundamental to existence, because without change it seems inconceivable that anything exists. Time is just the passage of this change.
Definition 4 (Time) Time is the ordered sequence of transitions between distinct states of the environment, where each state
Time is the process of becoming331. Every tick of the cosmic clock is creation and destruction. Some aspects of the environment persist through many ticks of the clock.
Definition 5 (Persistence) An aspect
Persistence is survival. Darwin’s natural selection332 on a universal scale. Stable atoms stick around because they vibe with physics333; critters adapt or get fossilized334. The universe is like a bouncer. Fit the rhythm and you stay. Clash with it and you’re out. New things are occasionally allowed in. This is the first whisper of “ought.” What persists is what is meant to, by the rules of the game.
THE ENVIRONMENT HAS AN OPINION
A state expresses some aspects, but not others. From the definition of environment, a statement
These statements form abstraction layers. Abstraction layers stack up like Matryoshka dolls, each layer refining the cosmic “ought” into sharper rules. From “thou shalt exist” at the base, we climb to “thou shalt compute efficiently” or “thou shalt not crash the system”337. Time, persistence, and expression give us this natural “ought”. Just the universe doing its thing338. For my conscious machine, this is the foundation.
PURPOSE
The fact of existence is a value judgement. Some things exist, and others do not. A rock doesn’t need to know physics to fall, but in doing so constrains what can happen next. It is an embodied ought that constrains what is, has been or ever will be. In this sense, every body is chattering away in a language forged by its form. When the state changes, some statements persist, and others are destroyed. This creates an incentive. The universe preserves that which preserves itself. Change is fundamental, and by its very nature change optimises for systems that cope with change, by deleting those that cannot. I want to formalise intelligence, which means I want to formalise a system that preserves itself. A living system.
A living, self preserving system is a statement
Therein lies the rub. Not everything serves homeostatic and reproductive goals. Not every possible world is a winner. Intelligent systems discriminate. Abstraction layers are biased toward some goals over others, but how exactly is that supposed to work? To really describe intelligence I need to formalise the idea of goals, but not in the abstract sense we humans are accustomed to. I can’t have goals separate from the systems that pursue them or I’m just going to end up with computational dualism again339. I need integrate goals with embodiment. A goal together with context and instructions is commonly known as a task. A task is what I use to formalise enactive cognition, in what I call Pancomputational Enactivism340341. A task is a formal description of a system in terms of its behaviour. That system is an abstraction layer, and the task is what it expresses. Outputs
The possible outputs are the extension
Definition 6 (v-task)
For a chosen
I sub alpha, a subset of L sub v is a set whose elements we call inputs ofalpha .O sub alpha, a subset of E sub I sub alpha is a set whose elements we call correct outputs ofalpha .
(generational hierarchy) A
Tasks are like Matryoshka dolls. Little ones fit inside bigger ones. For example not choking on your coffee fits inside surviving the day. It’s a hierarchy. So how does your body pick the right output? Every statement your body makes constrains what can happen next. A policy is just a statement that constrains your outputs. A correct policy constrains you to correct outputs, given the additional constraint of the inputs. Correct policies keep you from face-planting. They steer you toward the outputs that don’t end in a Darwin Award. It works thusly:
Definition 7 (inference)
- A
v -task policy is a statementpi in L sub v . It constrains how we complete inputs. - is a correct policy iff the correct outputs of are exactly the completions of such that is also a completion of an input.
- The set of all correct policies for a task is denoted .345
Assume -task and a policy . Inference[^347] proceeds as follows:
- we are presented with an input , and
- we must select an output .
\left( \sum_{(l_1, l_2) \in \text{app}} |(l_1 <_g l_2) - (l_1 <_a l_2)| - |(l_1 <_g l_2) - (l_1 <_b l_2)| \right) < 0
[^354]: Within the bounds of what we can conceive of, as systems whose nature it is to maintain homeostasis. [^355]: If the system did not maintain homeostasis, then it is dead. [^356]: All else being equal. [^357]: (intuitive summary) Learning is an activity undertaken by an adaptive system, and a task has been **learned** by a system that embodies a correct policy. Humans typically learn from **examples**. An example of a task is a correct output and input. [^358]: By the weakness of a statement, we mean the cardinality of its extension. By the weakness of an extension we mean its cardinality. [^359]: When we speak of simplicity with regards to a policy $\pi \in \Pi_\alpha$ we mean the cardinality of the smallest correct policy $\pi' \in \Pi_\alpha$ s.t. $E_{\pi'} = E_\pi$. The complexity of an extension is the **simplest** statement of which it is an extension. [^360]: (further intuitive summary) A collection of examples is a child task, so learning is an attempt to generalise from a child, to one of its parents. The lower level the child from which an agent generalises to parent, the ‘faster’ it learns, the more sample efficient the proxy. (**optimal proxy**) There is no proxy more efficient than weakness. The weakness proxy formalises the idea that "explanations should be no more specific than necessary" (see Bennett's razor in this ref[^361]). (**intuitive summary**) Learning is an activity undertaken by some manner of intelligent agent, and a task has been "learned" by an agent that knows a correct policy. Humans typically learn from "examples". An example of a task is a correct output and input. A collection of examples is a child task, so "learning" is an attempt to generalise from a child to one of its parents. The lower level the child from which an agent generalises to parent, the "faster" it learns (it chooses policies that complete a wider variety of tasks, and thus are more sample and energy efficient choices), the more efficient the proxy. The most efficient proxy is weakness (see proofs 1 and 2, or these refs[^362]), which is why we're using it here. To learn, one must expressing a policy $\pi$<yap-speak>pi</yap-speak> that constrains future behaviour to desirable worlds. That *generalises* to future instances of a problem. If tasks are uniformly distributed, then the most effective way to learn is to maximise the number of tasks $\pi$<yap-speak>pi</yap-speak> completes. A proxy is a means of choosing between correct policies, in the hopes of selecting a policy that generalises to the relevant parent tasks. I give two proxies above, but there are others. In the next chapter I will explain why weakness is the optimal proxy, but for now don't worry about that. It seems eminently reasonable to assume tasks are uniformly distributed. Anything else is an unnecessary assumption. A normative judgement beyond what is required. As we have already covered at length, the very fact of existence is a matter of *ought*. It is a sort of *existential normativity*. If a body exists and can store and retrieve information, then its representational capabilities are the product of that existential normativity. I'm not saying it isn't theoretically conceivable some Lovecraftian horror reaches into an environment and changes the rules at a whim, but such a thing is already baked into this existential normativity. For the purpose of deciding which policies are optimal in general, it makes no difference why the state changes. It simply does. Finally, I'll integrate this idea of a task with The Stack I've been talking about. To do this, I'll introduce the multilayer architecture (MLA). An abstractor function $f$<yap-speak>f</yap-speak> is applied here to *policies*, rather than just any statement. **Definition 10 (multilayer architecture)** *The multilayer architecture (MLA) found in both biological systems and computers. It integrates a stack with tasks to represent natural selection or 'correctness' at different layers of abstraction.* [^361]: Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In *Artificial General Intelligence*. Springer Nature, 2023a [^362]: Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In *Artificial General Intelligence*. Springer Nature, 2023a; and Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a * The *stack* is represented here by a sequence of uninstantiated tasks $\langle\lambda^0, \lambda^1...\lambda^n\rangle$<yap-speak>lambda zero, lambda one, up to lambda n</yap-speak> s.t $\lambda^{i+1} \sqsubset \lambda^i$<yap-speak>such that lambda i plus one is a sub-task of lambda i</yap-speak>. * $f$<yap-speak>f</yap-speak> is an *abstractor* function. * The *state* of the MLA is a sequence of policies $\langle\pi^0, \pi^1...\pi^n\rangle$<yap-speak>pi zero through pi n</yap-speak> and a sequence of vocabularies $\langle v^0, v^1...v^n\rangle$<yap-speak>v zero through v n</yap-speak> such that $v^{i+1} = f(v^i, \pi^i)$<yap-speak>v i plus one equals f of v i and pi i</yap-speak> and $\pi^i \in \Pi_{\lambda^i(v^i)}$<yap-speak>pi i is an element of the set of policies for task lambda i with vocabulary v i</yap-speak>. In the absence of abstraction where the system is seen as nothing more than the sum of its parts, the MLA is just a task $\lambda^0(v^0)$<yap-speak>lambda zero of v zero</yap-speak>, allowing us to look at the system across scales of distribution. We say the MLA is *over-constrained* when there exists $i < n$<yap-speak>i less than n</yap-speak> s.t. and $\Pi_{\lambda^i(v^i)} = \emptyset$<yap-speak>the set of policies for the task at level i is empty</yap-speak>, and ***multilayer-causal-learning*** (MCL) occurs when the MLA is not over-constrained and the proxy for learning is weakness. Note that the vocabulary is different at each level in the stack, which means each has its own generational hierarchy of tasks. By a higher level of abstraction, we mean a task higher in the stack (later in the causal chain). Now, I won’t actually use this definition for a couple of chapters, but it is important to introduce it here for intuition. After all, I have been speaking endlessly about abstraction layers, and it would be strange to introduce goal directed behaviour *within* an abstraction layer without explaining how that goal directed behaviour *propagates* up and down the stack. As you can see in the above definition, the process of applying the abstractor function to obtain second, third or higher orders of behavioural effect is the same. However because it is applied to a policy, this is useful for formalising biological and other distributed goal directed systems. For example, say $\alpha_1$<yap-speak>alpha one</yap-speak> and $\alpha_2$<yap-speak>alpha two</yap-speak> are tasks representing the behaviour of two cells. Imagine those cells are both child tasks of $\alpha$<yap-speak>alpha</yap-speak>, which is an organ. A collective identity[^363]. They exist in the same abstraction layer. $\alpha$<yap-speak>alpha</yap-speak> is like looking at an organ as a collection of cells. However, if we move up a level of abstraction by taking the policy of $\alpha$<yap-speak>alpha</yap-speak> and applying the abstractor function, we are now looking at the organ, because ‘organ’ is a property we ascribe to the behaviour of cells. It is a second order of abstracted behaviour. I will discuss more about how this works in later chapters. For now, it is only important to note that $v$<yap-speak>v</yap-speak>-tasks are arranged in a stack, like abstraction layers. As we look higher in the stack, the goal directed behaviour gets narrower and more specific, because each successive layer is an effect of the goal directed behaviour in the layers below. [^363]: [Patrick McMillen and Michael Levin. Collective intelligence: A unifying concept for integrating biology across scales and substrates. *Communications Biology*, 2024]() <yap-show>[blank page]</yap-show> # VII. WEAK **Human intelligence.** It is sometimes hailed as the crowning achievement of evolution. But in the Darwinian arena, it's not about being the smartest. It is about surviving long enough to pass on your genes. From a Darwinian perspective, intelligence is long-term adaptation that facilitates short-term adaptation during an organism's lifetime. Long-term adaptation is like the genes we inherit. Short-term is what we learn as we go. If evolution is an optimiser, then biological intelligence is a mesa-optimiser (an optimiser within an optimiser)[^364]. Without intelligence, a system would need all its knowledge pre-programmed. Like a robot with a fixed set of instructions. With intelligence a system can learn, adapt, and survive in a wider range of circumstances. It can complete a wider range of tasks[^365]. A machine with intelligence can learn from its environment, adapt to new situations, and potentially develop something akin to consciousness. It's not just about processing power. It is about the ability to change. Intelligence is the key to unlocking the door to consciousness. Without it, we're just building a glorified abacus. This chapter is based primarily on my paper on weak versus simple hypotheses[^366], with some minor updates from later works[^367]. I'll show how adaptability is maximised by using weakness as the proxy, and I propose the following epistemological razor: > "Explanations should be no more specific than necessary."[^368] [^364]: Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Risks from learned optimization in advanced machine learning systems, 2021 [^365]: All else being equal, of course. [^366]: Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In *Artificial General Intelligence*. Springer Nature, 2023a; and Michael Timothy Bennett. A formal theory of optimal learning with experimental results. *Forthcoming, IJCAI 2025*, 2025e [^367]: Michael Timothy Bennett. Optimal policy is weakest policy. *Under review*, 2025d; and Michael Timothy Bennett. A formal theory of optimal learning with experimental results. *Forthcoming, IJCAI 2025*, 2025e [^368]: In the publication I originally proposed this, I named it Bennett’s Razor. Once published, there are no backsies. # INTELLIGENCE IN STACKISM **Intelligence is not about** what something *is*, but what it *does*. I’ve framed everything as a stack of abstraction layers, each enacted by the one beneath. Like software on hardware, the layers go down. This is as true of human language as it is of human software. The cosmic *ought* determines functionality. Affordances[^369]. It is why we have one abstraction layer instead of another. Intelligence isn’t about what an organism *is* but what it *does*. **v-tasks**. Subjected to inputs, a system produces outputs. Intelligence *affords* adaptation. **Tasks are interconnected**, forming a **generational hierarchy** that reflects an organism’s temporal existence. Within this lattice, **child tasks** are specific cases of the broader **parent tasks** encompassing them. An organism’s past decisions is a child task of the task that includes every decision an organism might make over the course of its existence. The lattice structure links past and future behaviour. An example of a child task might be “take the succession of turns leading from home to office” and its parent could be “navigate the environment”. An organism’s survival depends on generalising from completed child tasks to meet the demands of the broader parent tasks that lie ahead.[^370] Thus, the generational hierarchy provides a dynamic framework for understanding intelligence as a process of bridging temporal scales. [^369]: James J. Gibson. *The Ecological Approach to Visual Perception*. Houghton Mifflin, 1979 [^370]: Adaptation requires flexibility—past success does not guarantee future survival. # POLICIES AND ADAPTATION In the context of intelligence and adaptation, policies serve as the mechanisms that guide an organism’s behaviour towards survival. Policies can be innate or hard-wired, but that is less adaptable than being able to learn new through interaction with the environment. A learned **policy** is a learned statement or constraint that an organism embodies to ensure fit behaviour. It acts as a rulebook for survival, dictating how to respond to various inputs to achieve desirable outcomes. Assuming an organism remains alive and in some condition to procreate, its past behaviour is an ostensive definition of fit behaviour. Past behaviour is likely to imply at least some fit policies, and some unfit ones. The challenge of learning is to discern which policies will reliably constrain the organism to fit phenotypes. That complete the tasks of the past, and the widest range of possible future tasks[^371]. [^371]: An organism that relies on overly specific policies may struggle to adapt to new environments. # THE WEAK SHALL INHERIT THE WORK A weaker policy is one that permits more possible behaviours while still being fit, akin to a versatile tool. Their lack of specificity allows them to address a wide range of future scenarios, enhancing adaptation. Weak policies are the key to generalisation. I'll prove it. Recall from the definition of learning: * A proxy $<$<yap-speak>less than</yap-speak> is a binary relation on statements, and the set of all proxies is $Q$<yap-speak>Q</yap-speak>. * $<_{w}$<yap-speak>the weakness proxy</yap-speak> is the **weakness** proxy[^372]. For statements $l_{1}, l_{2}$<yap-speak>l one and l two</yap-speak> we have $l_{1} <_{w} l_{2}$<yap-speak>l one is weaker than l two</yap-speak> iff $|E_{l_{1}}| < |E_{l_{2}}|$<yap-speak>the cardinality of the extension of l one is less than the cardinality of the extension of l two</yap-speak>. * $<_{d}$<yap-speak>the simplicity proxy</yap-speak> is the **description length** or **simplicity** proxy[^373]. We have $l_{1} <_{d} l_{2}$<yap-speak>l one is simpler than l two</yap-speak> iff $|l_{1}| > |l_{2}|$<yap-speak>the length of l one is greater than the length of l two</yap-speak>. Now consider the follow two proofs: ### Theorem 1 (sufficiency) Assume $\alpha \sqsubset \omega$<yap-speak>alpha is a child of omega</yap-speak>. The weakness proxy sufficient to maximise the probability that a parent $\omega$<yap-speak>omega</yap-speak> is learned from a child $\alpha$<yap-speak>alpha</yap-speak>[^374]. **Proof 1** You're given the definition of $\upsilon$-task<yap-speak>upsilon task</yap-speak> $\alpha$<yap-speak>alpha</yap-speak> from which you infer a hypothesis $\pi \in \Pi_{\alpha}$<yap-speak>pi in the set of policies for alpha</yap-speak>. To learn $\omega$<yap-speak>omega</yap-speak>, you need $\pi \in \Pi_{\omega}$<yap-speak>pi in the set of policies for omega</yap-speak>: 1. For every $\pi \in \Pi_{\alpha}$<yap-speak>pi in the set of policies for alpha</yap-speak> there exists a $\upsilon$-task<yap-speak>upsilon task</yap-speak> $\gamma_{\pi} \in \Gamma_{\upsilon}$<yap-speak>gamma pi in the set of tasks</yap-speak> s.t. $O_{\gamma_{\pi}} = E_{\pi}$<yap-speak>the outputs of gamma pi equal the extension of pi</yap-speak>, meaning $\pi$ permits only correct outputs for that task regardless of input. We'll call the highest level task $\gamma_{\pi}$ s.t. $O_{\gamma_{\pi}} = E_{\pi}$ the **policy task** of $\pi$. 2. $\omega$<yap-speak>omega</yap-speak> is either the policy task of a policy in $\Pi_{\alpha}$<yap-speak>the set of policies for alpha</yap-speak>, or a child thereof[^375]. 3. If a policy $\pi$ is correct for a parent of $\omega$, then it is also correct for $\omega$. Hence we should choose $\pi$ that has a policy task with the largest number of children. As tasks are uniformly distributed, that will maximise the probability that $\omega$ is $\gamma_{\pi}$ or a child thereof. 4. For the purpose of this proof, we say one task is **equivalent**[^376] to another if it has the same correct outputs. 5. No two policies in $\Pi_{\alpha}$ have the same policy task[^377]. This is because all the policies in $\Pi_{\alpha}$ are derived from the same set inputs, $I_{\alpha}$. 6. The set of statements which might be outputs addressing inputs in $I_{\omega}$ and not $I_{\alpha}$, is $\overline{E_{I_{\alpha}}} = \{l \in L_{\upsilon} : l \notin E_{I_{\alpha}}\}$[^378]<yap-speak>the complement of E I alpha, defined as the set of statements l in L upsilon such that l is not in E I alpha</yap-speak>. 7. For any given $\pi \in \Pi_{\alpha}$, the extension $E_{\pi}$ of $\pi$ is the set of outputs $\pi$ implies. The subset of $E_{\pi}$ which fall outside the scope of what is required for the known task $\alpha$ is $\overline{E_{I_{\alpha}}} \cap E_{\pi}$[^379]<yap-speak>the intersection of the complement of E I alpha and E pi</yap-speak>. 8. $L_{\upsilon} = E_{I_{\alpha}} \cup \overline{E_{I_{\alpha}}}$ and for all $\pi \in \Pi_{\alpha}, E_{\pi} \subset L_{\upsilon}$. Apart from the inputs and correct outputs of $\alpha, \overline{E_{I_{\alpha}}}$ contains only outputs which would be incorrect according to both $\alpha$ and $\omega$. Put another way, $E_{I_{\alpha}} \cap E_{\pi} = O_{\alpha}$ for every possible choice of $\pi$ in $\Pi_{\alpha}$. Hence the only way $|E_{\pi}|$ can increase is if $|\overline{E_{I_{\alpha}}} \cap E_{\pi}|$ increases. It follows that $|\overline{E_{I_{\alpha}}} \cap E_{\pi}|$ increases with $|E_{\pi}|$. [^372]: By the weakness of a statement, we mean the cardinality of its extension. By the weakness of an extension we mean its cardinality. [^373]: When we speak of simplicity with regards to a policy $\pi \in \Pi_{\alpha}$ we mean the cardinality of the smallest correct policy $\pi' \in \Pi_{\alpha}$ s.t. $E_{\pi'} = E_{\pi}$. The complexity of an extension is the **simplest** statement of which it is an extension. [^374]: Assume there exist correct policies for $\omega$, because otherwise there would be no point in trying to learn it. [^375]: I'd like to give credit here to Nora Belrose for pointing out an error. Nora pointed out I was miscounting the number of tasks. As a result I realised I was not counting tasks, I was in fact counting policy tasks and had entirely neglected to mention this fact. This was a significant error which has now been corrected, with several additional steps added to account for equivalence. [^376]: This is because switching from $\beta$ to $\zeta$ s.t. $I_{\beta} \neq I_{\zeta}$ and $O_{\beta} = O_{\zeta}$ would be to pursue the same goal in different circumstances. This is because inputs are *subsets* of outputs, so both sets of inputs are implied by the outputs. $O_{\zeta}$ implies $I_{\beta}$ and $O_{\beta}$ implies $I_{\zeta}$ [^377]: Every policy task for policies of $\alpha$ is non-equivalent from the others. [^378]: This is because $\overline{E_{I_{\alpha}}}$ contains every statement which is a correct output or an incorrect output, and $\overline{E_{I_{\alpha}}}$ contains every statement which could possibly be in $I_{\omega}, E_{I_{\omega}}$ and thus $O_{\omega}$. [^379]: This is because $\overline{E_{I_{\alpha}}}$ is the set of all conceivable outputs by which one might attempt to complete $\alpha$, and so the set of all outputs that can’t be made when undertaking $\alpha$ is $\overline{E_{I_{\alpha}}}$ because those outputs occur given inputs that aren’t part of $I_{\alpha}$. 9. $2^{|E_{I_\alpha} \cap E_\pi|}$<yap-speak>Two to the power of the size of the intersection of E sub I alpha and E sub pi</yap-speak> is the number of non-equivalent parents of $\alpha$<yap-speak>alpha</yap-speak> to which $\pi$<yap-speak>pi</yap-speak> generalises. It increases monotonically with the weakness of $\pi$<yap-speak>pi</yap-speak>. 10. Given $\mathfrak{v}$-tasks<yap-speak>v-tasks</yap-speak> are uniformly distributed and $\Pi_\alpha \cap \Pi_\omega \neq \varnothing$<yap-speak>the intersection of Pi alpha and Pi omega is not empty</yap-speak>, the probability that $\pi \in \Pi_\alpha$<yap-speak>pi in Pi alpha</yap-speak> generalises to $\omega$<yap-speak>omega</yap-speak> isp(\pi \in \Pi_\omega \mid \pi \in \Pi_\alpha, \alpha \sqsubset \omega) = \frac{2^{|E_{I_\alpha} \cap E_\pi|}}{2^{|E_{I_\alpha}|}}
undefinedp(T = \text{true} \mid H = \text{true}) = 1
<yap-speak>the probability that T is true given H is true equals one</yap-speak> Now this is technically true based on my observations. It is stupid, but it is true, because it completely ignores causality. Lets assume after renaming my dog Thor I start to try to *intervene* in the environment to cause thunder. This means instead of waiting for it to rain and just passively observing my dog happening to howl when there is thunder, I start trying to make my dog howl hoping thunder will follow. This *intervention* is represented by a do operator applied to the variable $H$<yap-speak>H</yap-speak> as $\text{do}(H = \text{true})$<yap-speak>do H equals true</yap-speak>. Using this operator, I can now represent the difference between observing my dog howl when it thunders, and making my dog howl hoping I get thunder: [^427]: Judea Pearl and Dana Mackenzie. *The Book of Why: The New Science of Cause and Effect*. Basic Books, Inc., New York, 1st edition, 2018p(T = \text{true} \mid \text{do}(H = \text{true})) = p(T = \text{true}) \neq p(T = \text{true} \mid H = \text{true})
<yap-speak>the probability of thunder being true given an intervention on howling is just the probability of thunder, which is not equal to the probability of thunder given the observation of howling</yap-speak> **What this captures is the direction of causality.** Thunder causes my dog to howl. My dog’s howling does not cause thunder.  **The do operator is equivalent to a variable that influences the value of $H$<yap-speak>H</yap-speak>.** So really, we can represent any intervention by just expanding the graph as follows.  **This particular fact** was pointed out by Dawid in 2002, but by the time someone told me this I had already gone to the trouble of proving it. The proof is in the appendix. Nevertheless I now build on all this in an important way. All of this talk about intervening to *cause* an event and me passively *observing* an event is rather narcissistic. I am not the only agency capable of causing events. The question isn’t whether *I* caused an event. The question is *what* caused an event. Passive observation is just another way of saying something other than me caused the event. This can be resolved by just adding more variables. I can make my dog howl. You can make my dog howl. The thunder can make my dog howl. I’ll represent these with binary variables $\text{you do} \in \{\text{true}, \text{false}\}$<yap-speak>you do in the set true or false</yap-speak> and $\text{me do} \in \{\text{true}, \text{false}\}$<yap-speak>me do in the set true or false</yap-speak>.  **It gets complicated when I do this.** The fact that we’re dealing in high level abstractions like me and you means we lose the acyclicality of the graph. It becomes bidirectional We end up with cases like where you intervene, and in doing so thwart my attempted intervention[^428]. Or the opposite.  [^428]: For example, if you intervene to set $H$<yap-speak>H</yap-speak> to true and I try to set it to false, and you whack me over the head with my dog, making him howl and me fail in my attempted intervention. For science, of course. WHY THIS BIDIRECTIONALITY? Because computational dualism! Because these variables ignore important information. I can cause you to intervene or not, and you can cause me to intervene or not. The do operator gets away with acyclic graphs because it conveniently ignores the fact that there is more than one causal agency in the environment. Great for science, which was what it was proposed for. Bad for designing artificial intelligence. In my formalism variables and values like $H = \text{true}, T = \text{true}$<yap-speak>H equals true, T equals true</yap-speak> and $\text{me do}$<yap-speak>me do</yap-speak> are just *aspects* of the environment. By presupposing the world is divided up into variables, we presuppose there are certain dividing lines between aspects of the environment. That would undermine any claim I might then make, because assuming objects and properties amounts to assuming an abstraction layer. As I showed earlier complexity is determined by the abstraction layer. Why is a chair a chair? Because it affords us something[^429]. We work in terms of what is *relevant*[^430] to our survival[^431]. If I want to properly capture causality, I need to capture *relevance*. I need to explain how I get from an environment made up of unlabelled aspects and programs, to something that has an *identity* that *causes* something. That intervenes. A *causal-identity*. Normally a causality researcher would start with variables and learn the causal relations, but here we have no variables. We need to learn the variables. This led me to ask: if I had a causal relation, could I then learn the variables that fit that causal relation? Yes! I would just need to learn a policy that classifies the causal relation. THIS IS WHERE THE EXISTENTIAL OUGHT from earlier comes in handy here. It gives me attraction and repulsion from physical states. Valence[^432]. That simple dichotomy is enough to get us everything else. It is my foundational causal relation. Instead of assuming a set of objects and properties and trying to learn the causal relations between then, I can flip the problem. I can assume the causal relation, and learn the objects and properties. The causal relation is valence. If I am an adaptive system, then some aspects of the environment will attract me, and some will repel me. Instead of starting with two variables and learning the causal arrow between them, I can start with the arrow and learn the variables. [^429]: James J. Gibson. *The Ecological Approach to Visual Perception*. Houghton Mifflin, 1979 [^430]: What is relevant is determined by the cosmic ought. It preserves and destroys aspects, and each aspect forms an abstraction layer at a higher level of abstraction. [^431]: John Vervaeke, Timothy Lillicrap, and Blake Richards. Relevance realization and the emerging framework in cognitive science. *J. Log. Comput.*, 2012; John Vervaeke and Leonardo Ferraro. *Relevance, Meaning and the Cognitive Science of Wisdom*. Springer Netherlands, Dordrecht, 2013a; and John Vervaeke and Leonardo Ferraro. Relevance realization and the neurodynamics and neuroconnectivity of general intelligence. In Inman Harvey, Ann Cavoukian, George Tomko, Don Borrett, Hon Kwan, and Dimitrios Hatzinakos, editors, *SmartData*, NY, 2013b. Springer Nature [^432]: The transition of the environment from one state to another selects for aspects that preserve themselves. Things that preserve themselves are attracted to circumstances that preserve them. This attraction is otherwise known as valence. We get valence from the simple fact of change. ### THE PURE UNMEDIATED EXPERIENCE OF BEING I have proofs of optimal adaptability. A system which identifies the weakest policies is optimal. To do so, an adaptive system may delegate control to construct an optimal abstraction layer in which to construct the optimal policy. Any system that adapts optimally must correctly identify cause and effect[^433]. It must correctly discriminate between observation and intervention, so let me define exactly what I mean by causal intervention here. **Definition 12 (intervention)** *Intuitively, if $int$<yap-speak>int</yap-speak> and $obs$<yap-speak>obs</yap-speak> are "events" which have happened, then we say that $int$<yap-speak>int</yap-speak> has **caused** $obs$<yap-speak>obs</yap-speak> if $obs$<yap-speak>obs</yap-speak> would not have happened in the absence of $int$<yap-speak>int</yap-speak> (counterfactual). In our formalism, an **event** is a statement in $L_v$<yap-speak>L v</yap-speak>, and an event **happens** or is **observed** iff it is a true statement. If $obs \in L_v$<yap-speak>obs in L v</yap-speak> is sensorimotor activity we interpret as an "observed event", and $int \in L_v$<yap-speak>int in L v</yap-speak> is in **intervention** (by an organism or other agency, in the sense described by Pearl[^434]) to cause that event, then $obs \subset int$<yap-speak>obs is a subset of int</yap-speak> (because $int$<yap-speak>int</yap-speak> could not be said to cause $obs$<yap-speak>obs</yap-speak> unless $obs \subset int$<yap-speak>obs is a subset of int</yap-speak>).* By learning policies in response to attraction and repulsion from environmental states, a system must construct policies that classify those parts of the environment which intervene to *cause* that valence. I call these policies **causal-identities**. For example, to know that I have been bitten by a dog, I must have a causal-identity for that dog. That causal-identity is how I react to the dog. The dog is what it *affords* me[^435], rather than something platonic. A dog means something very different to a flea than it does to me. **Definition 13 (causal identity)** *If[^436] $obs \in L_v$<yap-speak>obs in L v</yap-speak> is an observed event, and $int \in L_v$<yap-speak>int in L v</yap-speak> is in intervention causing $obs$<yap-speak>obs</yap-speak>, then intuitively $c \subseteq int - obs$<yap-speak>c is a subset of int minus obs</yap-speak> "identifies" or "names" the intervening agency if $c \neq \emptyset$<yap-speak>c is not empty</yap-speak>. We call $c$<yap-speak>c</yap-speak> a **causal identity** corresponding to $int$<yap-speak>int</yap-speak> and $obs$<yap-speak>obs</yap-speak>. Suppose $INT$<yap-speak>I N T</yap-speak> and $OBS$<yap-speak>O B S</yap-speak> are sets of statements, and we assume $OBS$<yap-speak>O B S</yap-speak> contains observed events and $INT$<yap-speak>I N T</yap-speak> interventions, then a causal identity corresponding to $INT$<yap-speak>I N T</yap-speak> and $OBS$<yap-speak>O B S</yap-speak> is $c \neq \emptyset$<yap-speak>c is not empty</yap-speak> s.t. $\forall i \in INT(c \subset int)$ and $\forall obs \in OBS(c \cap obs = \emptyset)$<yap-speak>such that for all i in I N T, c is a subset of int, and for all obs in O B S, the intersection of c and obs is empty</yap-speak> (we can attempt to construct a causal identity for any $INT$<yap-speak>I N T</yap-speak> and $OBS$<yap-speak>O B S</yap-speak>). If a policy is a causal identity, then the associated task is to classify interventions.* [^433]: Judea Pearl and Dana Mackenzie. *The Book of Why: The New Science of Cause and Effect*. Basic Books, Inc., New York, 1st edition, 2018; [Jonathan Richens and Tom Everitt. Robust agents learn causal world models. In The Twelfth International Conference on Learning Representations, 2024.](https://openreview.net/forum?id=pOoKI3ouv1); and Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In *Artificial General Intelligence*. Springer Nature, 2023b [^434]: Judea Pearl and Dana Mackenzie. *The Book of Why: The New Science of Cause and Effect*. Basic Books, Inc., New York, 1st edition, 2018 [^435]: James J. Gibson. *The Ecological Approach to Visual Perception*. Houghton Mifflin, 1979 [^436]: (EXAMPLE) Suppose we have organisms a (Alice) and b (Bob). The inputs Alice has experienced so far $I_{h<t_a}$ can be divided into those in which Bob affected Alice $I_a^b$ and those in which Bob did not $I_a^{\neg b} = I_{h<t_a} - I_a^b$. By affecting Alice, Bob has intervened in Alice's experience. Alice can construct a causal identity $b$ for Bob corresponding to interventions $INT = I_a^b$ and observations $OBS = I_a^{\neg b}$. The objects which "exist" in Alice's experience are those for which she constructs a causal identity, so this is how Bob comes to exist as a distinct "object" which Alice experiences, rather than in parts of other objects). Causal-identities are the effect of the environment upon a body. A sensory datum. A body is always being attracted or repelled by its particular surroundings. This causes it to express statements. Those statements are causal-identities for what attracted or repelled the body. They are prelinguistic classifiers, each one denoting a particular cause of valence. Weaker causal-identities classify more commonly encountered causes of valence. This is just like how a weaker policy applies to more situations. If I have a causal-identity for my experiences of red, it is weaker than my causal-identity for red lobster. By w-maxing, a living system divides the world up into a hierarchy of policies. Some more specific. Some weaker. I suggest this is why and how a contentless environment can be divided up into objects and properties[^437]. I call this The Psychophysical Principle of Causality[^438]. There are two preconditions which must be satisfied before a system will express a causal-identity for an object. First there must be an incentive, for example the object is relevant to survival. This is the incentive precondition. It aligns with the idea of affordances[^439], in that a causal-identity is only constructed for an object to the extent that it affords a system some advantage to be able to recognise that object. The cosmic ought deems it so. Second, the system's abstraction layer must be capable of expressing a causal-identity for an object, that discriminates between events caused by the object and events which are not. This is the scale precondition, because the vocabulary of the abstraction layer must be of sufficient scale to represent the causal-identity. In other words I have formalised objects as their behaviour, and behaviour as tasks. A task only describes a coherent object if there exist correct policies for that task, which depends on the vocabulary. Hence the vocabulary must be of sufficient scale to represent and store a causal-identity. **Definition 14 (preconditions)** If $o$<yap-speak>o</yap-speak> is an organism, and $c$<yap-speak>c</yap-speak> is a causal identity: * the **representation** precondition is met iff $c \in L_{v_o}$<yap-speak>c is an element of L v o</yap-speak>, and * the **incentive** precondition is met if $o$<yap-speak>o</yap-speak> must learn $c$<yap-speak>c</yap-speak> to remain "fit"[^440]. [^437]: Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In *Artificial General Intelligence*. Springer Nature, 2023b [^438]: Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. *Why Is Anything Conscious?* Preprint, accepted to and presented at ASSC27 and MoC5, 2024 [^439]: James J. Gibson. *The Ecological Approach to Visual Perception*. Houghton Mifflin, 1979 [^440]: It is possible an organism might construct $c$ even if it is not required for the organism to remain fit, hence 'if' instead of 'iff'. Incentive is a sufficient precondition in conjunction with representation, but it is not strictly necessary. A learning system which w-maxes will construct a causal-identity for anything which meets these preconditions. Things which do not meet these preconditions will effectively not exist, as far as the learning system is concerned. In one paper, I even used this fact to explain the Fermi paradox[^441], arguing that our ability to recognise intelligence and thus life is contingent on that life being "like us". Life and intelligence could be all around us, but its behaviour falls outside the scope of what we're wired to notice and understand[^442]. Put another way, we rationalise what we see. When we construct a causal-identity for something, we construct a model of its *intent*. **Definition 15 (purpose, goal or intent)** *We consider a policy $c$<yap-speak>c</yap-speak> which is a causal identity corresponding to INT and OBS to be the **intent, purpose** or **goal** ascribed to the interventions. $c$<yap-speak>c</yap-speak> is what the interventions share in common, meaning the "name" or "identity" of behaviour is the "intent", "goal" or "purpose" of behaviour. Just as an intervention caused an observation, the particular intent which motivated the agency undertaking the intervention is what caused it (to correctly infer intent, one must infer a causal identity that implies subsequent interventions).* [^441]: Michael Timothy Bennett. Compression, the fermi paradox and artificial super-intelligence. In *Artificial General Intelligence*. Springer Nature, 2022b [^442]: Michael Timothy Bennett. On the computation of meaning, language models and incomprehensible horrors. In *Artificial General Intelligence*. Springer Nature, 2023c # THE SELF I am about to introduce the concept of self. To that end it is helpful to know *what* exactly is getting a self, so here is a formal definition of an organism. It is not central to this thesis, but for the sake of explaining what is being learned as a causal-identity, and for the latter chapters on "protosymbols" and meaning, it is helpful. I describe the circumstances of an organism[^443] $o$<yap-speak>o</yap-speak> as $\langle v_o, \mu_o, p_o, <_o \rangle$<yap-speak>v sub o, mu sub o, p sub o, and the binary relation less than sub o</yap-speak> where: * $O_{\mu_o}$<yap-speak>O sub mu sub o</yap-speak> contains every output which qualifies as "fit" according to natural selection. * $p_o$<yap-speak>p sub o</yap-speak> is the set of policies an organism knows, s.t. $p_o \subset p_{n.s.} \cup p_{\mathfrak{h}_{<t_0}}$<yap-speak>p sub o is a subset of the union of p n s and p h before time t zero</yap-speak> and: * $p_{n.s.} \subset \mathcal{L}_{v_o}$<yap-speak>p n s, a subset of L v o</yap-speak> is **reflexes** hard coded from birth by natural selection. * $p_{\mathfrak{h}_{<t_0}} = \bigcup_{\zeta \in \mathfrak{h}_{<t_0}} \Pi_\zeta$<yap-speak>p h before time t zero, defined as the union over zeta in h of capital pi sub zeta</yap-speak> is the set of policies it is possible to **learn** from a history of past interactions represented by a task $\mathfrak{h}_{<t_0}$<yap-speak>h before time t zero</yap-speak>. * If $\Pi_{\mathfrak{h}_{<t_0}} \not\subset (p_o - p_{n.s.})$<yap-speak>If pi h is not a subset of p o minus p n s</yap-speak> then the organism has **selective memory**. It can "forget" outputs, possibly to productive ends if they contradict otherwise good policies. * $<_o$<yap-speak>The relation less than sub o</yap-speak> is a binary relation over $\Gamma_{v_o}$<yap-speak>gamma v o</yap-speak> we call **preferences**. <yap-speak>To survive in a complex interactive setting as humans do, one must be able to tell the difference between events one has caused, and events one has merely observed. This implies the construction of causal-identities for one’s self. A do operator. Just because a do operator doesn’t encompass the full breadth of causal agencies because it only accounts for the agency of one, doesn’t mean it isn’t critically important. One must represent one’s self!</yap-speak> <yap-show> TO SURVIVE IN A COMPLEX interactive setting as humans do, one must be able to tell the difference between events one has caused, and events one has merely observed. This implies the construction of causal-identities for one’s self. A do operator. Just because a do operator doesn’t encompass the full breadth of causal agencies because it only accounts for the agency of one, doesn’t mean it isn’t critically important. One must represent one’s self! </yap-show> **Definition 16 (first order self)** *If $c$<yap-speak>c</yap-speak> is the lowest level causal identity corresponding to INT and OBS, and INT is every intervention an organism could make (not just past interventions, but all potential future interventions), then we consider $c$<yap-speak>c</yap-speak> to be the system’s **first order self**. If $c \in p_o$<yap-speak>c is an element of p sub o</yap-speak> then an organism has constructed a first order self. A first order self for an organism $o$<yap-speak>o</yap-speak> is denoted $o^1$<yap-speak>o one</yap-speak>. An organism has at most one first order self.* <yap-speak>This is equivalent to reafference. It lets one determine what one has done. It is the self that is part of every intervention one undertakes. Without a first-order-self, there is nothing to tie together past actions in memory. A fly has a first-order-self. There is a good</yap-speak> <yap-show> THIS IS EQUIVALENT TO REAFFERENCE. It lets one determine what one has done. It is the self that is part of every intervention one undertakes. Without a 1ST-order-self, there is nothing to tie together past actions in memory. A fly has a 1ST-order-self. There is a good </yap-show> [^443]: (INTUITIVE SUMMARY) Strictly speaking an organism $o$ would be a policy, but we can describe the circumstances of its existence as a task $\mu$ that describes all "fit" behaviour for that organism. We can also identify policies the organism "knows", because these are implied by the policy that is the organism. Likewise, we can represent lossy memory by having the organism "know" fewer policies than are implied by its history of interactions. Finally, preferences are the particular "protosymbol" the organism will use to "interpret" an input in later definitions. reason for this. Imagine a fly on my shoulder. If I move, the room moves around the fly. If the fly moves, the room moves around the fly. If the fly can’t tell the difference between these two things, it is going to end up very dead. Flies can tell the difference between these two things. A system may have hard-wired policies that do not contain one’s 1ST-order-self. Involuntary reflexes, for example. In those cases, those responses are not triggered by the self as I define it. One has a 1ST-order-self when one can classify one’s own interventions. It amounts to an overall ‘intent’ or goal which predicts my actions. This is because a causal-identity is an statement made by the environment, and it has an extension. By constructing a self, one defines a constraint from which future behaviour may be abducted. One can derive all of one’s possible agentic behaviours from a 1ST-order-self. <yap-show>  </yap-show>Figure 5: Illustration of a 1ST-order-self. **However, this does not mean** everything is preordained. A 1ST-order-self is just a very weak constraint. When system $\mathfrak{a}$<yap-speak>a</yap-speak> constructs a causal-identity for $\mathfrak{b}$<yap-speak>b</yap-speak>, what it is doing is creating a classifier of what $\mathfrak{b}$<yap-speak>b</yap-speak> *affords* $\mathfrak{a}$<yap-speak>a</yap-speak>. It is a sort of prediction of that narrow part of $\mathfrak{b}$’s 1ST-order-self which is relevant to $\mathfrak{a}$<yap-speak>a</yap-speak>, if $\mathfrak{b}$<yap-speak>b</yap-speak> happens to have one. What happens when I go a step further? What happens when $\mathfrak{a}$<yap-speak>a</yap-speak> predicts $\mathfrak{b}$<yap-speak>b</yap-speak> in so much detail that it predicts $\mathfrak{b}$’s prediction of $\mathfrak{a}$<yap-speak>a</yap-speak>? A sort of 2ND order of self, constructed in relation to another? **Definition 17 (chain notation)** *Suppose we have two organisms, $\mathfrak{a}$<yap-speak>a</yap-speak> (Alice) and $\mathfrak{b}$<yap-speak>b</yap-speak> (Bob). $c_{\mathfrak{a}}^{\mathfrak{b}}$<yap-speak>c sub a super b</yap-speak> denotes a causal identity for $\mathfrak{b}$<yap-speak>b</yap-speak> constructed by $\mathfrak{a}$<yap-speak>a</yap-speak> (what Alice thinks Bob intends). Subscript denotes the organism who constructs the causal identity, while superscript denotes the object. The superscript can be extended to denote chains of predicted causal identity. For example, $c_{\mathfrak{a}}^{\mathfrak{ba}} \subset c_{\mathfrak{a}}^{\mathfrak{b}}$<yap-speak>c sub a super b a is a subset of c sub a super b</yap-speak> denotes $\mathfrak{a}$’s prediction of $\mathfrak{b}$’s prediction of $\mathfrak{a}^1$ (what Alice thinks Bob thinks Alice intends). The superscript of $c_{\mathfrak{a}}^*$ can be extended indefinitely to indicate recursive predictions, however the extent recursion is possible is determined by $\mathfrak{a}$’s vocabulary $\mathfrak{v}_{\mathfrak{a}}$. Finally, Bob need not be an organism. Bob can be anything for which Alice constructs a causal identity.* **Definition 18 ($n^{\text{th}}$ order self)**<yap-speak>Definition 18, n-th order self</yap-speak> *An $n^{\text{th}}$ order self for $\mathfrak{a}$ is $\mathfrak{a}^n = c^{*\mathfrak{a}}_{\mathfrak{a}}$ where $*$ is replaced by a chain, and $n$ denotes the number of reflections. For example, a second order self $\mathfrak{a}^2 = c^{\mathfrak{ba}}_{\mathfrak{a}}$, and a third order self $\mathfrak{a}^3 = c^{\mathfrak{baba}}_{\mathfrak{a}}$. We use $\mathfrak{a}^2$ to refer to any second order self, and chain notation to refer to a specific second order self, for example $c^{\mathfrak{ba}}_{\mathfrak{a}}$. The union of two $n^{\text{th}}$ order selves is also considered to be an $n^{\text{th}}$ order self, for example $\mathfrak{a}^3 = c^{\mathfrak{baba}}_{\mathfrak{a}} \cup c^{\mathfrak{dada}}_{\mathfrak{a}}$, and the weaker or higher level a self is in the generational hierarchy, the more selves there are of which it is part.*<yap-speak>An n-th order self for a is a to the n, defined as c star a of a, where star is replaced by a chain, and n denotes the number of reflections. For example, a second order self a squared equals c b a of a, and a third order self a cubed equals c b a b a of a. We use a squared to refer to any second order self, and chain notation to refer to a specific second order self, for example c b a of a. The union of two n-th order selves is also considered to be an n-th order self, for example a cubed equals the union of c b a b a of a and c d a d a of a; and the weaker or higher level a self is in the generational hierarchy, the more selves there are of which it is part.</yap-speak> A <yap-show>2ND-order-self</yap-show><yap-speak>second-order-self</yap-speak> requires causal-identities for other objects. It is my prediction of your prediction of my <yap-show>1ST-order-self</yap-show><yap-speak>first-order-self</yap-speak>, or a narrow contextually relevant part thereof, what what I think is your perspective. This would be needed to anticipate and avoid predation. Conversely, it could help one herd and capture prey. One very distinct capability a <yap-show>2ND-order-self</yap-show><yap-speak>second-order-self</yap-speak> conveys but a <yap-show>1ST-order-self</yap-show><yap-speak>first-order-self</yap-speak> does not, is the ability to represent and reason about one’s own destruction. If I can predict your prediction of me, then I can predict you observing my destruction. This makes death conceivable. It makes it possible to reason about yourself in general, and to engage in basic deception. <yap-cap>Figure 6: Illustration of a <yap-show>2ND-order-self</yap-show><yap-speak>second-order-self</yap-speak>.</yap-cap> Finally, a 3RD-order-self permits one to predict one’s own 2ND-order-selves, which would be useful in even more complex social or multi-agent environments. It makes it possible to reason about the reasoning of others about themselves, and to plan complex interactions taking into account this self-aware reasoning capability in others. This is makes complex deception possible, where I can reason about your reasoning about my reasoning about your reasoning about me. I can act to influence your reasoning about my reasoning about your reasoning about me, so that you think I have an interpretation of your behaviour that I do not. That I view you in a manner that I do not, and that I think you’re going to do something in relation to me that I do not. However, if we can all predict each other so well, can we not also predict deception? The next chapter will thus deal in language, meaning and co-operation. <yap-cap>Figure 7: Illustration of a 3RD order self.</yap-cap> **Theorem 8 (n$^{\text{th}}$<yap-speak>n-th</yap-speak> order self convergence)** *An organism that uses weakness as its proxy will learn an n$^{\text{th}}$<yap-speak>n-th</yap-speak> order self if the incentive and representation preconditions are met for that order of self.* **Proof 8** Assume we have an organism $\mathfrak{o}$<yap-speak>o</yap-speak> that learns using "weakness" as a proxy. A $\mathfrak{v}_{\mathfrak{o}}$-task $\mathfrak{h}_{<t_\mathfrak{o}}$<yap-speak>v o task h sub t o</yap-speak> represents the history of $\mathfrak{o}$<yap-speak>o</yap-speak> (meaning $\mathfrak{h}_{<t_\mathfrak{o}} \sqsubset \mu_{\mathfrak{o}}$ and $\mathfrak{h}_{<t_\mathfrak{o}}$<yap-speak>h sub t o is a subset of mu o, and h sub t o</yap-speak> is an ostensive definition of $\mu_{\mathfrak{o}}$<yap-speak>mu o</yap-speak>, by virtue of the fact that $\mathfrak{o}$<yap-speak>o</yap-speak> remains alive). The organism explores the environment, intervening to maintain homeostasis. As it does so, more and more inputs and outputs are included in $\mathfrak{h}_{<t_\mathfrak{o}}$<yap-speak>h sub t o</yap-speak>. It follows that: 1. From the representation precondition we have that there exists a n$^{\text{th}}$<yap-speak>n-th</yap-speak> order self $\mathfrak{o}^n \in L_{\mathfrak{v}_{\mathfrak{o}}}$<yap-speak>o n in L v o</yap-speak>. 2. To remain fit, $\mathfrak{o}$<yap-speak>o</yap-speak> must "generalise" to $\mu_{\mathfrak{o}}$<yap-speak>mu o</yap-speak> from $\mathfrak{h}_{<t_\mathfrak{o}}$<yap-speak>h sub t o</yap-speak>. According to the incentive precondition, generalisation to $\mu_{\mathfrak{o}}$<yap-speak>mu o</yap-speak> requires $\mathfrak{o}$<yap-speak>o</yap-speak> learn the n$^{\text{th}}$<yap-speak>n-th</yap-speak> order self, which is when $\mathfrak{o}^n \in \mathfrak{p}_{\mathfrak{o}}$<yap-speak>o n in p o</yap-speak>. 3. From this ref[^444] we have proof that weakness is the optimal choice of proxy to maximise the probability of generalisation from child to parent is the weakest policy. It follows that $\mathfrak{o}$<yap-speak>o</yap-speak> will generalise from $\mathfrak{h}_{<t_\mathfrak{o}}$<yap-speak>h sub t o</yap-speak> to $\mu_{\mathfrak{o}}$<yap-speak>mu o</yap-speak> given the smallest history of interventions with which it is possible to do so (meaning the smallest possible ostensive definition, or cardinality $|\mathcal{O}_\alpha|$<yap-speak>the cardinality of O alpha</yap-speak>). Were we to assume learning under the above conditions does not construct an n$^{\text{th}}$<yap-speak>n-th</yap-speak> order self for $\mathfrak{o}$<yap-speak>o</yap-speak>, then one of the three statements above would be false and we would have a contradiction. It follows that the proposition must be true. $\square$ [^444]: Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In *Artificial General Intelligence*. Springer Nature, 2023a # X. LANGUAGE CANCER This chapter is about language. It is a combination of my Mirror Symbol Hypothesis[^445], and my subsequent papers on symbol emergence[^446] and the formalisation of Gricean pragmatics[^447]. Normativity expressed in natural language is not the same as the *existential* normativity I spoke about earlier. Normativity in natural language is social normativity. What we think things mean, and what values we hold. Our interpretations. Meaning. The cosmic *ought* is normative only in the sense that it discriminates, seeming to judge some things worthy of existence and not others. Meaning comes in many forms. The *semantic* meaning of a proposition is its truth conditions[^448]. For example, the meaning of "Larry's cat is green" is true when Larry has a cat, and it is green. Of course, this can lead one in circles. We end up defining the semantic meaning of the first sentence in terms of two other sentences. We end up endlessly deferring semantic meaning. Something is always missing. This is what Derrida called "differance"[^449]. Fortunately, this idea fits beautifully with Stack Theory. Semantic theories of meaning specify what the semantic truth conditional meaning of language is, so each such theory would be an abstraction layer. Another sort of theory is a *foundational* theory of meaning. A foundational theory descirbes a system that is a level of abstraction below. A foundational theory says what the system is that produces the semantic theory. [^445]: Michael Timothy Bennett and Yoshihiro Maruyama. Philosophical specification of empathetic ethical artificial intelligence. *IEEE Transactions on Cognitive and Developmental Systems*, 14(2): 292–300, 2022a [^446]: Michael Timothy Bennett. Symbol emergence and the solutions to any task. In *Artificial General Intelligence*. Springer Nature, 2022a [^447]: Michael Timothy Bennett. On the computation of meaning, language models and incomprehensible horrors. In *Artificial General Intelligence*. Springer Nature, 2023c [^448]: J. Speaks. Theories of Meaning. In Edward N. Zalta, editor, *The Stanford Encyclopedia of Philosophy*. Stanford University, Stanford, Spring 2021 edition, 2021 [^449]: Jacques Derrida. *Writing and difference*. U of Chicago P, 1978 # GRICEAN PRAGMATICS The **Gricean foundational theory** is that meaning is determined by intent[^450]. Assume speaker $\alpha$<yap-speak>alpha</yap-speak> means $m$<yap-speak>m</yap-speak> by saying $u$<yap-speak>u</yap-speak>. By saying $u$<yap-speak>u</yap-speak>, $\alpha$<yap-speak>alpha</yap-speak> intends that: 1. his audience come to believe $m$<yap-speak>m</yap-speak>, 2. his audience recognise this intention [called m-intention], and 3. (1) occurs on the basis of (2). **This is also called pragmatic meaning.** Basically, if you and I speak, then there are two meanings. There is the meaning I intend, and the meaning you interpret. The meaning you interpret is what you ascribe to my words and actions. The meaning I intend is what I *want* you to ascribe. We have understood each other if you ascribe the meaning I want you to ascribe. It is a prediction problem. 1. To *intend* a meaning, I need to predict what you think I think. 2. To *interpret* my meaning, you need to predict what I predicted you thought I thought. **This fits neatly within the frame of selves.** Assume I am $a$<yap-speak>a</yap-speak> and you are $b$<yap-speak>b</yap-speak>: 1. To *intend* a meaning, I need to meet the scale and incentive preconditions to construct the causal-identity $c_a^{ba}$<yap-speak>c sub a super b a</yap-speak>. 2. Then, to *interpret* my meaning, you need to meet the preconditions for the causal-identity $c_b^{aba}$<yap-speak>c sub b super a b a</yap-speak>. **In other words, we both need to have 2nd-order-selves!** Whatever information is represented in my 2nd-order-selves, I can communicate to you. The is analogous to attention. In trying to predict you and survive, I make a prediction of myself from your perspective. That prediction informs my communication with you. [^450]: Paul Grice. Meaning. *The Philosophical Review*, 66(3):377–388, 1957; and Paul Grice. Utterer’s meaning and intention. *The Philosophical Review*, 78(2):147–177, 1969 **ATTENTION IS ALL I NEED** If I am speaking to you, the meaning of my words is whatever I intend. You have understood me if you know that by saying $x$<yap-speak>x</yap-speak> that I am trying to get you to think $y$<yap-speak>y</yap-speak>. This is what a 2ND-order-self allows agents to do[^451]. A 2ND-order-self lets me predict what you think I think. I can use that to predict what you think I am trying to achieve by saying $x$<yap-speak>x</yap-speak>. Hence if I want you to believe $y$<yap-speak>y</yap-speak>, I can derive $x$<yap-speak>x</yap-speak> from my 2ND-order-self. I can simultaneously have many different 2ND-order-selves. I can have one for everyone I talk to. I can have aggregates thereof. I can even construct one from my own point of view. However for the sake of communication what is important is that I am have one from the perspective of the person I am talking to, so I can tailor my words to each of them to convey my intended meaning. Conversely, if I want to know what you mean when you say $x$<yap-speak>x</yap-speak>, I can derive $y$<yap-speak>y</yap-speak> from my prediction of your prediction of my prediction of you. My 2ND-order-self’s prediction of you. The 2ND-order-selves neatly encapsulate those things I pay attention to. They are contextual access. They offer a simple explanation of why some information is available to me at some times, and not others. Moreover, it is obviously impossible to communicate anything *not* in 2ND and higher order selves. At least, it is impossible to communicate *meaning* like a human can. As stated earlier, consciousness is oft cut into access and phenomenal aspects. The phenomenal is the special snowflake. Access is relegated the unglamorous role of “information processing”. Bland and unmysterious. However access consciousness is usually formally defined as the information available for reasoning and report. Communication. Presumably, that means communication in the human sense. If I accept that, then I must also accept that the only information available to access consciousness is that within 2ND and higher order selves. In other words, I propose a radically different interpretation of access consciousness than is typically used. I argue this is the only acceptable interpretation if what we are trying to describe is human-like consciousness, with human-like meaning. I will delve into this more deeply later, but for now it is important to make note of this point. [^451]: Michael Timothy Bennett. On the computation of meaning, language models and incomprehensible horrors. In *Artificial General Intelligence*. Springer Nature, 2023c # SEMIOTICS. WHAT MEAN? Organisms that can communicate can co-operate to achieve complex goals. Now that we know how intent can be communicated it is easy to see how signalling conventions or language might evolve. To that end, here is a formal definition of organism. **Definition 19 (organism)** I describe the circumstances of an organism[^452] $o$<yap-speak>o</yap-speak> as $\langle v_o, \mu_o, \mathfrak{p}_o, <_o \rangle$<yap-speak>v sub o, mu sub o, p sub o, and the relation less than o</yap-speak> where: * $O_{\mu_o}$<yap-speak>O sub mu sub o</yap-speak> contains every output which qualifies as "fit" according to natural selection. * $\mathfrak{p}_o$<yap-speak>p sub o</yap-speak> is the set of policies an organism knows, s.t. $\mathfrak{p}_o \subset \mathfrak{p}_{n.s.} \cup \mathfrak{p}_{\mathfrak{h}_{<t_o}}$<yap-speak>p sub o is a subset of the union of the natural selection policies and the learnable policies</yap-speak> and: * $\mathfrak{p}_{n.s.} \subset L_{v_o}$<yap-speak>p n s, a subset of L v o,</yap-speak> is **reflexes** hard coded from birth by natural selection. * $\mathfrak{p}_{\mathfrak{h}_{<t_o}} = \bigcup_{\zeta \in \mathfrak{h}_{<t_o}} \Pi_\zeta$<yap-speak>p sub h before time t zero, equals the union of pi sub zeta for all zeta in h before t zero,</yap-speak> is the set of policies it is possible to **learn** from a history of past interactions represented by a task $\mathfrak{h}_{<t_o}$<yap-speak>h before time t zero</yap-speak>. * If $\Pi_{\mathfrak{h}_{<t_o}} \not\subset (\mathfrak{p}_o - \mathfrak{p}_{n.s.})$<yap-speak>If pi sub h before t zero is not a subset of p o minus p n s</yap-speak> then the organism has **selective memory**. It can "forget" outputs, possibly to productive ends if they contradict otherwise good policies. * $<_o$<yap-speak>The relation</yap-speak> is a binary relation over $\Gamma_{v_o}$<yap-speak>gamma v sub o</yap-speak> we call **preferences**. For meaning, I must now define what I call a protosymbol system. It is a set of tasks based on the causal-identities an organism has learned. **Definition 20 (protosymbol system)** Assume an organism $o$<yap-speak>o</yap-speak>. For each policy $p \in \mathfrak{p}_o$<yap-speak>p in p sub o</yap-speak> there exists a set $\mathfrak{s}_p = \{ \alpha \in \Gamma_{v_o} : p \in \Pi_\alpha \}$<yap-speak>s sub p, which is the set of tasks alpha in gamma v sub o such that p is a valid policy for alpha</yap-speak> of all tasks for which $p$<yap-speak>p</yap-speak> is a correct policy. The union of all such sets is\mathfrak{s}o = \bigcup{p \in \mathfrak{p}o} { \alpha \in \Gamma{v_o} : p \in \Pi_\alpha }
undefinedFootnotes
-
These ended up as one big research project, starting with fractal compression and ending with consciousness. It has been a rather tumultuous ride due to contretemps like a global plague, university restructuring, and my stubborn refusal to heed most advice… anyway at the time I am writing this, my supervisory panel is officially listed in the university system as Sean Welsh, Anna Ciaunica, Yoshihiro Maruyama, Colin Klein and Samuel Allen Alexander. ↩
-
Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In Artificial General Intelligence. Springer Nature, 2023a; and Michael Timothy Bennett. A formal theory of optimal learning with experimental results. Forthcoming, IJCAI 2025, 2025e ↩
-
Michael Timothy Bennett. Symbol emergence and the solutions to any task. In Artificial General Intelligence. Springer Nature, 2022a; and Michael Timothy Bennett. On the computation of meaning, language models and incomprehensible horrors. In Artificial General Intelligence. Springer Nature, 2023c ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b ↩
-
Michael Timothy Bennett. Compression, the fermi paradox and artificial super-intelligence. In Artificial General Intelligence. Springer Nature, 2022b ↩
-
Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. The artificial scientist: Logicist, emergentist, and universalist approaches to artificial general intelligence. In Artificial General Intelligence. Springer Nature, 2022b ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. Philosophical specification of empathetic ethical artificial intelligence. IEEE Transactions on Cognitive and Developmental Systems, 14(2): 292–300, 2022a ↩
-
Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
Michael Timothy Bennett. What the f*ck is artificial general intelligence? Under Review, 2025b ↩
-
Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a ↩
-
Ashitha Ganapathy and Michael Timothy Bennett. Cybernetics and the future of work. In 2021 IEEE 21CW, 2021. DOI: 10.1109/21CW48944.2021.9532561 ↩
-
Michael Timothy Bennett. Computable Artificial General Intelligence. Under Review, 2022c ↩
-
Gabrielle S. Adams, Benjamin A. Converse, Andrew H. Hales, and Leidy E. Klotz. People systematically overlook subtractive changes. Nature, 2021 ↩
-
I have written 21 papers total. 12 of these are published or forthcoming in peer reviewed books and journals. By August 2025, I expect that number will rise to 19 out of 21. To validate my progress I have made sure to publish my results as I have progressed through my PhD. ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. Philosophical specification of empathetic ethical artificial intelligence. IEEE Transactions on Cognitive and Developmental Systems, 14(2): 292–300, 2022a ↩
-
Michael Timothy Bennett. Symbol emergence and the solutions to any task. In Artificial General Intelligence. Springer Nature, 2022a; and Michael Timothy Bennett. On the computation of meaning, language models and incomprehensible horrors. In Artificial General Intelligence. Springer Nature, 2023c ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b; and Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. The artificial scientist: Logicist, emergentist, and universalist approaches to artificial general intelligence. In Artificial General Intelligence. Springer Nature, 2022b ↩
-
Michael Timothy Bennett. What the f*ck is artificial general intelligence? Under Review, 2025b ↩
-
Pei Wang. On defining artificial intelligence. Journal of Artificial General Intelligence, 10(2):1–37, 2019 ↩
-
Richard Sutton. The bitter lesson. University of Texas at Austin, 2019 ↩
-
Ben Goertzel et al. Opencog hyperon: A framework for agi at the human level and beyond. Technical report, OpenCog Foundation, 2023 ↩
-
Eric Nivel et al. Autocatalytic endogenous reflective architecture. Technical report, Reykjavik University, School of Computer Science, 2013 ↩
-
Patrick Hammer and Tony Lofthouse. ‘opennars for applications’: Architecture and control. In Ben Goertzel, Aleksandr I. Panov, Alexey Potapov, and Roman Yampolskiy, editors, Artificial General Intelligence, pages 193–204, Cham, 2020. Springer Nature ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a. Which I am proud to say won an award at the 17th International Conference on Artificial General Intelligence, in Seattle. ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a; Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c; and Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a. ↩
-
Computers are often described as a stack. For example, a video game runs on a game engine that runs on an operating system that runs on a game console. Each one is just code inside the level below, like Matryoshka dolls. ↩
-
Hardware is a sort of body. ↩
-
For example, the idea that our reality is a simulation running in another reality amounts to claiming there are yet more abstraction layers to below . ↩
-
Stack Theory in turn provides the foundation for formalising enactivism in what I call Pancomputational Enactivism. ↩
-
Gualtiero Piccinini and Corey Maley. Computation in Physical Systems. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, Stanford, Sum. 21 edition, 2021. ↩
-
To ‘express’ is to physically realise, manifest or call into existence an object. ↩
-
Some may object this conflates description with verbalisation. ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a; and Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a ↩
-
As I have defined it for the purpose of this thesis. ↩
-
L. J. Savage. The Foundations of Statistics. John Wiley & Sons, NY, USA, 1954 ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a; and Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b; and Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
Later in the thesis, use this to define arithmetic operations on binary strings and run experiments. ↩
-
Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In Artificial General Intelligence. Springer Nature, 2023a; Michael Timothy Bennett. A formal theory of optimal learning with experimental results. Forthcoming, IJCAI 2025, 2025e; and Michael Timothy Bennett. Computable Artificial General Intelligence. Under Review, 2022c ↩
-
Simp-maxing being simplicity maximisation based on Ockham’s Razor. ↩
-
Inherited, hard-wired from birth. ↩
-
During the organism’s lifetime. ↩
-
All else being equal. ↩
-
This allows us to avoid asserting particular objects or properties exist. For example, why do we consider a stool to be something that exists instead of four legs and a seat?. Everything is really just an aspect of the environment. We need make this distinction so that we can examine exactly what is needed for an object exist in chapter 11. ↩
-
Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In Artificial General Intelligence. Springer Nature, 2023a; and Michael Timothy Bennett. A formal theory of optimal learning with experimental results. Forthcoming, IJCAI 2025, 2025e ↩
-
To frame it as an epistemological razor: “Explanations should be no more specific than necessary.” ↩
-
Recall simp-maxing is preferring simpler hypotheses in line with Ockham’s Razor. ↩
-
Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a; and Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a ↩
-
Elliott Sober. Ockham’s Razors: A User’s Manual. Cambridge Uni. Press, 2015. DOI: 10.1017/CBO9781107705937 ↩
-
Jacob D. Bekenstein. Universal upper bound on the entropy-to-energy ratio for bounded systems. Phys. Rev. D, 23: 287–298, Jan 1981 ↩
-
It says a bounded system can contain only a finite amount of information. ↩
-
To an extent determined by selection pressures. ↩
-
Conversely, in a static stack like a stable environment, weak constraints can take complex forms. This is used to explain the origins of life in chapter XI. ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b ↩
-
Which I am proud to say won an award at the 16th International Conference on Artificial General Intelligence, in Stockholm. ↩
-
Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
Valence. ↩
-
Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
e.g. It must be able to see and discriminate between it, and not it. ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. Philosophical specification of empathetic ethical artificial intelligence. IEEE Transactions on Cognitive and Developmental Systems, 14(2): 292–300, 2022a ↩
-
Michael Timothy Bennett. Symbol emergence and the solutions to any task. In Artificial General Intelligence. Springer Nature, 2022a ↩
-
Michael Timothy Bennett. On the computation of meaning, language models and incomprehensible horrors. In Artificial General Intelligence. Springer Nature, 2023c ↩
-
Paul Grice. Meaning. The Philosophical Review, 66(3):377–388, 1957; and Paul Grice. Utterer’s meaning and intention. The Philosophical Review, 78(2):147–177, 1969 ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. Philosophical specification of empathetic ethical artificial intelligence. IEEE Transactions on Cognitive and Developmental Systems, 14(2): 292–300, 2022a ↩
-
This is where I propose The Mirror Symbol Hypothesis from my first publication, to explain empathy. ↩
-
P C W Davies and C H Lineweaver. Cancer tumors as metazoa 1.0: tapping genes of ancient ancestors. Physical Biology, 8(1), feb 2011; and Michael Levin. Bioelectrical approaches to cancer as a problem of the scaling of the cellular self. Progress in Biophysics and Molecular Biology, 2021. Cancer and Evolution ↩
-
Chris Fields, Mahault Albarracin, Karl Friston, Alex Kiefer, Maxwell JD Ramstead, and Adam Safron. How do inner screens enable imaginative experience? applying the free-energy principle directly to the study of conscious experience. Neuroscience of Consciousness, 2025 ↩
-
Michael L. Wong, Carol E. Cleland, Daniel Arend, Stuart Bartlett, H. James Cleaves, Heather Demarest, Anirudh Prabhu, Jonathan I. Lunine, and Robert M. Hazen. On the roles of function and selection in evolving systems. Proceedings of the National Academy of Sciences, 120(43):e2310223120, 2023. DOI: 10.1073/pnas.2310223120. URL https://www.pnas.org/doi/abs/10.1073/pnas.2310223120 ↩
-
This is an explanation of life proposed by others. ↩
-
Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a ↩
-
Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c ↩
-
Michael Timothy Bennett. Compression, the fermi paradox and artificial super-intelligence. In Artificial General Intelligence. Springer Nature, 2022b ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b; and Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. The artificial scientist: Logicist, emergentist, and universalist approaches to artificial general intelligence. In Artificial General Intelligence. Springer Nature, 2022b ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b; and Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
Computed concurrently in one step. ↩
-
Computed sequentially in many steps. ↩
-
Meaning the latter is the weaker standard for consciousness. ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. Philosophical specification of empathetic ethical artificial intelligence. IEEE Transactions on Cognitive and Developmental Systems, 14(2): 292–300, 2022a ↩
-
Michael Timothy Bennett. Symbol emergence and the solutions to any task. In Artificial General Intelligence. Springer Nature, 2022a; and Michael Timothy Bennett. On the computation of meaning, language models and incomprehensible horrors. In Artificial General Intelligence. Springer Nature, 2023c ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b; and Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
Jaegwon Kim. Philosophy of Mind. Routledge, New York, 3rd ed. edition, 2011 ↩
-
Seeing a scan of brain activity is not the same as actually experiencing particular brain activity. It is this experience that we cannot observe in another. ↩
-
The subject of explanation is called an explanandum. The explanation is itself is called the explanans. Philosophers study the explanandum, and engineers the explanans. ↩
-
An interpreter is something that translates one thing to another; for example French to Spanish, or from computer code to the movements of a mechanical arm. ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a ↩
-
David Wallace. The Emergent Multiverse: Quantum Theory according to the Everett Interpretation. Oxford University Press, 2012. ISBN 9780199546961. DOI: 10.1093/acprof:oso/9780199546961.001.0001. URL https://doi.org/10.1093/acprof:oso/9780199546961.001.0001 ↩
-
A position now called occasionalism. ↩
-
Who’s a good boy? ↩
-
At least, that is what Sean Welsh once called me. ↩
-
Hilary Putnam. Psychological predicates. In William H. Capitan and Daniel Davy Merrill, editors, Art, mind, and religion, pages 37–48. University of Pittsburgh Press, 1967 ↩ -
Pei Wang. A constructive explanation of consciousness. Journal of Artificial Intelligence and Consciousness, 07(02):257–275, 2020; Piotr Boltuc. The engineering thesis in machine consciousness. Techné: Research in Philosophy and Technology, 2012; and Manuel Blum and Lenore Blum. A theoretical computer science perspective on consciousness. J. Artif. Intell. Conscious., 8:1–42, 2020 ↩ -
Recall the subject of explanation is called an explanandum. The explanation is itself is called the explanans. We are here trying to describe the explanandum. ↩
-
Anil Seth and Tim Bayne. Theories of consciousness. Nature Reviews Neuroscience, 2022; and Georg Northoff. Unlocking The Brain, Vol. II: Consciousness, volume 2. Oxford University Press, USA, 2014 ↩
-
Ned Block. On a confusion about a function of consciousness. Brain and Behavioral Sciences, 1995 ↩
-
Report just means you can consciously set out to communicate it to other people. ↩
-
David Chalmers. Facing up to the problem of consciousness. Journal of Consciousness Studies, 1995; Ned Block. On a confusion about a function of consciousness. Brain and Behavioral Sciences, 1995; Thomas Nagel. What is it like to be a bat? Philosophical Review, 1974; Shaun Gallagher and Dan Zahavi. The Phenomenological Mind. Routledge, New York, NY, 2021; and Thomas Fuchs. Ecology of the Brain: The phenomenology and biology of the embodied mind. Oxford University Press, 2017 ↩
-
Homeostasis basically just means “staying alive”. I remain alive because I have “static” internal state; physical processes that keep me from being dead. ↩
-
Ned Block. On a confusion about a function of consciousness. Brain and Behavioral Sciences, 1995 ↩
-
Thomas Nagel. What is it like to be a bat? Philosophical Review, 1974 ↩
-
David Chalmers. Facing up to the problem of consciousness. Journal of Consciousness Studies, 1995; and Ned Block. On a confusion about a function of consciousness. Brain and Behavioral Sciences, 1995 ↩
-
Piotr Boltuc. The engineering thesis in machine consciousness. Techné: Research in Philosophy and Technology, 2012 ↩
-
Bjorn Merker. The liabilities of mobility: A selection pressure for the transition to consciousness in animal evolution. Consciousness and Cognition, 2005. Neurobiology of Animal Consciousness; Bjorn Merker. Consciousness without a cerebral cortex: A challenge for neuroscience and medicine. Behavioral and Brain Sciences, 2007; and Andrew B. Barron and Colin Klein. What insects can tell us about the origins of consciousness. Proceedings of the National Academy of Sciences, 2016 ↩
-
Judea Pearl and Dana Mackenzie. The Book of Why: The New Science of Cause and Effect. Basic Books, Inc., New York, 1st edition, 2018 ↩
-
Stan Franklin, Bernard J Baars, Uma Ramamurthy, Gilbert Harman, Antonio Chella, Michael Wheeler, Terrell Ward Bynum, and John Barker. Apa newsletters, 2008; and Piotr Boltuc. The engineering thesis in machine consciousness. Techné: Research in Philosophy and Technology, 2012 ↩
-
Pei Wang. A Constructive Explanation of Consciousness and its Implementation. World Scientific, 2023 ↩
-
Realised just means “made real” or “produced” or “created”. ↩
-
Piotr Boltuc. The engineering thesis in machine consciousness. Techné: Research in Philosophy and Technology, 2012; and Piotr Bołtuć. Consciousness for agi. Procedia Computer Science, 2020. BICA 2019 ↩
-
Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
At least those parts I consider important; why there is “something it is like”, the construction of selves, access consciousness and meaning. ↩
-
Which I already published in one of those aforementioned papers. ↩
-
Alain Morin. Levels of consciousness and self-awareness: A comparison and integration of various neurocognitive views. Consciousness and Cognition, 2006 ↩
-
I will argue that if access conscious contents are those available for report, then they are available for report in the sense of human exchanges of meaningful intent. I will show that the exchange of communicative intent requires reflectivity, and so access consciousness cannot exist without self awareness. ↩
-
David M. Rosenthal. Consciousness and Mind. Oxford University Press UK, New York, 2005; and Richard Brown, Hakwan Lau, and Joseph E. LeDoux. Understanding the higher-order approach to consciousness. Trends in Cognitive Sciences, 23(9):754–768, 2019. doi: 10.1016/j.tics.2019.06.009 ↩
-
John Morrison. Perceptual confidence. Analytic Philosophy, 57(1):15–48, 2016. DOI: 10.1111/phib.12077; and Megan Peters. Towards characterizing the canonical computations generating phenomenal experience, 04 2021 ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b ↩
-
Bernard Baars. In the Theater of Consciousness: The Workspace of the Mind. 1997 ↩
-
Anil Seth and Tim Bayne. Theories of consciousness. Nature Reviews Neuroscience, 2022 ↩
-
Manuel Blum and Lenore Blum. A theoretical computer science perspective on consciousness. J. Artif. Intell. Conscious., 8:1–42, 2020 ↩
-
Gerald M Edelman and Joseph A Gally. Reentry: a key mechanism for integration of brain function. Front Integr Neurosci, 7:63, August 2013 ↩
-
Anil K Seth, Jeffrey L McKinstry, Gerald M Edelman, and Jeffrey L Krichmar. Visual binding through reentrant connectivity and dynamic synchronization in a brain-based device. Cereb Cortex, 2004 ↩
-
Victor Lamme. Towards a true neural stance on consciousness. Trends in cognitive sciences, 2006; and Victor Lamme and Pieter Roelfsema. The distinct modes of vision offered by feedforward and recurrent processing. Trends in neurosciences, 2000 ↩
-
Giulio Tononi. An information integration theory of consciousness. BMC Neuroscience, 5(1):42, 2004; and Giulio Tononi, Melanie Boly, Marcello Massimini, and Christof Koch. Integrated information theory: from consciousness to its physical substrate. Nature Reviews Neuroscience, 17(7):450–461, Jul 2016. ISSN 1471-0048. DOI: 10.1038/nrn.2016.44. URL ↩
-
Anil Seth and Tim Bayne. Theories of consciousness. Nature Reviews Neuroscience, 2022 ↩
-
W. R. Ashby. Principles of the self-organizing dynamic system. Journal of General Psychology, 1947; and H. von Foerster. On self-organizing systems and their environments. In Self-Organizing Systems. Pergamon Press, 1960 ↩
-
Scott Camazine, Nigel Franks, J Sneyd, Eric Bonabeau, Jean-Louis Deneubourg, and Guy Theraulaz. Self-Organization in Biological Systems. Princeton University Press, NJ, 2001; Thomas D. Seeley. When is self-organization used in biological systems? The Biological Bulletin, 2002; and Fernando Rosas, Pedro A.M. Mediano, Martín Ugarte, and Henrik J. Jensen. An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems. Entropy, 2018 ↩
-
Hermann Haken. Advanced Synergetics: Instability Hierarchies of Self-Organizing Systems and Devices. Springer-Verlag, Berlin, 1983 ↩
-
Scott Camazine. Patterns in nature. Natural history, 2003; and Martha Ann Bell and Kirby Deater-Deckard. Biological systems and the development of self-regulation: Integrating behavior, genetics, and psychophysiology. Journal of developmental and behavioral pediatrics, 2007 ↩
-
Scott Kelso. Dynamic Patterns: The Self-Organization of Brain and Behavior. MIT Press, Boston, 1997; Karl Friston. The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2):127–138, 2010; and Emmanuelle Tognoli and J A Scott Kelso. Enlarging the scope: grasping brain complexity. Front Syst Neurosci, 2014 ↩
-
Chris Fields and Michael Levin. Scale-free biology: Integrating evolutionary and developmental thinking. BioEssays, 42, 06 2020; and Patrick McMillen and Michael Levin. Collective intelligence: A unifying concept for integrating biology across scales and substrates. Communications Biology, 2024 ↩
-
Karl Friston. The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2):127–138, 2010 ↩
-
Naturalist meaning as a consequence of natural selection. ↩
-
Friston K., FitzGerald T., Rigoli F., Schwartenbeck P., O. Doherty J., and Pezzulo G. Active inference and learning. Neurosci Biobehav Rev., pages 862–879, 2016 ↩
-
Karl Friston. The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2):127–138, 2010; and Karl Friston. Life as we know it. Journal of The Royal Society Interface, 10(86):20130475, 2013. DOI: 10.1098/rsif.2013.0475 ↩
-
Mark Solms. The Hidden Spring. Profile Books, London, 2021 ↩
-
Bjorn Merker. The liabilities of mobility: A selection pressure for the transition to consciousness in animal evolution. Consciousness and Cognition, 2005. Neurobiology of Animal Consciousness; and Bjorn Merker. Consciousness without a cerebral cortex: A challenge for neuroscience and medicine. Behavioral and Brain Sciences, 2007 ↩
-
A creature without a self is just a reflection of the world around it. This has some interesting implications. ↩
-
Erich von Holst and Horst Mittelstaedt. Das reafferenzprinzip. Naturwissenschaften, 37(20):464–476, Jan 1950. ISSN 1432-1904. DOI: 10.1007/BF00622503 ↩
-
Andrew B. Barron and Colin Klein. What insects can tell us about the origins of consciousness. Proceedings of the National Academy of Sciences, 2016 ↩
-
I was unaware of reafference at the time I initially published my findings. My theory found its origins in artificial general intelligence and Pearlean causality, rather than a biologically inclined empirical perspective. ↩
-
Ricard Solé, Melanie Moses, and Stephanie Forrest. Liquid brains, solid brains. Philosophical Transactions of the Royal Society B: Biological Sciences, 374(1774):20190040, 2019. DOI: 10.1098/rstb.2019.0040. URL https://royalsocietypublishing.org/doi/abs/10.1098/rstb.2019.0040; Ricard Solé and Luís F Seoane. Evolution of brains and computers: The roads not taken. Entropy, 24(5):665, 2022; and Ricard Solé et al. Fundamental constraints to the logic of living systems. Interface Focus, 2024 ↩
-
Brett P. Andersen, Mark Miller, and John Vervaeke. Predictive processing and relevance realization: exploring convergent solutions to the frame problem. Phenomenology and the Cognitive Sciences, 2022 ↩
-
John Vervaeke, Timothy Lillicrap, and Blake Richards. Relevance realization and the emerging framework in cognitive science. J. Log. Comput., 2012; John Vervaeke and Leonardo Ferraro. Relevance, Meaning and the Cognitive Science of Wisdom. Springer Netherlands, Dordrecht, 2013a; John Vervaeke and Leonardo Ferraro. Relevance realization and the neurodynamics and neuroconnectivity of general intelligence. In Inman Harvey, Ann Cavoukian, George Tomko, Don Borrett, Hon Kwan, and Dimitrios Hatzinakos, editors, Smart Data, NY, 2013b. Springer Nature; and Johannes Jaeger, Anna Riedl, Alex Djedovic, John Vervaeke, and Denis Walsh. Naturalizing relevance realization: Why agency and cognition are fundamentally not computational. Frontiers in Psychology, 15, 2024 ↩
-
Anna Ciaunica, Evgeniya V. Shmeleva, and Michael Levin. The brain is not mental! coupling neuronal and immune cellular processing in human organisms. Frontiers in Integrative Neuroscience, 2023 ↩
-
Evan Thompson. Mind in Life: Biology, Phenomenology, and the Sciences of Mind. Harvard University Press, Cambridge MA, 2007 ↩
-
Patrick McMillen and Michael Levin. Collective intelligence: A unifying concept for integrating biology across scales and substrates. Communications Biology, 2024 ↩
-
Note that though I formalise enactive cognition, I do so by formalising the formation of an interpreter rather than presupposing it. This is useful to combine enactivism with computationalism. ↩
-
L. J. Savage. The Foundations of Statistics. John Wiley & Sons, NY, USA, 1954 ↩
-
Francisco Varela, Evan Thompson, Eleanor Rosch, and Jon Kabat-Zinn. The Embodied Mind: Cognitive Science and Human Experience. 2016; and Giovanni Rolla and Nara Figueiredo. Bringing forth a world, literally. Phenomenology and the Cognitive Sciences, 2021 ↩
-
Gualtiero Piccinini. Physical Computation: A Mechanistic Account. Oxford University Press, UK, 2015 ↩
-
Johannes Jaeger, Anna Riedl, Alex Djedovic, John Vervaeke, and Denis Walsh. Naturalizing relevance realization: Why agency and cognition are fundamentally not computational. Frontiers in Psychology, 15, 2024 ↩
-
Gualtiero Piccinini and Corey Maley. Computation in Physical Systems. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, Stanford, Sum. 21 edition, 2021 ↩
-
Elliott Sober. Ockham’s Razors: A User’s Manual. Cambridge Uni. Press, 2015. DOI: 10.1017/CBO9781107705937 ↩
-
Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c ↩
-
JJC Smart. Sensations and brain processes. Philosophical Review, 68(April): 141–56, 1959. DOI: 10.2307/2182164 ↩
-
Gilbert H. Harman. The inference to the best explanation. The Philosophical Review, 74(1):88–95, 1965. ISSN 00318108, 15581470. URL http://www.jstor.org/stable/2183532 ↩
-
Bas C. van Fraassen. Laws and Symmetry. Oxford University Press, 1989 ↩
-
Jacques Derrida. Writing and difference. U of Chicago P, 1978 ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a ↩
-
Thanks Elija Perrier for help with the phrasing here. ↩
-
It is probably better known as Hume’s Law, but I prefer Hume’s Guillotine. Sharper. More of an edge. ↩
-
Paul Grice. Meaning. The Philosophical Review, 66(3):377–388, 1957; and Paul Grice. Utterer’s meaning and intention. The Philosophical Review, 78(2):147–177, 1969 ↩
-
Utterance is philosophical jargon for “something said aloud”. ↩
-
I cite precedent for the use of profanity in the chapter title. A respected PLoS medical journal permitted the word “shit” in a paper title. My use of censored profanity seems a little tame in comparison. ↩
-
Stefanie J Krauth, Jean T Coulibaly, Stefanie Knopp, Mahamadou Traoré, Eliézer K N’Goran, and Jürg Utzinger. An in-depth analysis of a piece of shit: distribution of Schistosoma mansoni and hookworm eggs in human stool. PLoS Neglected Tropical Diseases, 6(12): e1969, 12 2012. ISSN 1935-2727. DOI: 10.1371/journal.pntd.0001969. ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. The artificial scientist: Logicist, emergentist, and universalist approaches to artificial general intelligence. In Artificial General Intelligence. Springer Nature, 2022b; and Michael Timothy Bennett. What the f*ck is artificial general intelligence? Under Review, 2025b ↩
-
Stuart Russell. Artificial Intelligence and the Problem of Control, pages 19–24. Springer Nature, 2022 ↩
-
Kristinn R. Thorisson. A New Constructivist AI: From Manual Methods to Self-Constructive Systems, pages 145–171. Atlantis Press, Paris, 2012; Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, MA, 2018; and P. Wang. Rigid Flexibility: The Logic of Intelligence. Applied Logic Series. Springer Nature, 2006 ↩
-
Judea Pearl and Dana Mackenzie. The Book of Why: The New Science of Cause and Effect. Basic Books, Inc., New York, 1st edition, 2018 ↩
-
Ben Goertzel. Generative ai vs. agi: The cognitive strengths and weaknesses of modern llms, 2023. arXiv ↩
-
Marcus Hutter. Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer Nature, Heidelberg, 2010 ↩
-
Shane Legg and Marcus Hutter. Universal intelligence: A definition of machine intelligence. Minds and Machines, pages 391–444, 2007 ↩
-
Jan Leike and Marcus Hutter. Bad universal priors and notions of optimality. Proceedings of The 28th Conference on Learning Theory, in Proceedings of Machine Learning Research, pages 1244–1259, 2015 ↩
-
François Chollet. On the measure of intelligence, 2019 ↩
-
Nick Bostrom. The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds and Machines, 22(2): 71–85, May 2012. ISSN 1572-8641. DOI: 10.1007/s11023-012-9281-3; and Nick Bostrom. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, Oxford, UK, 2014. ISBN 9780199678112 ↩
-
Michael Timothy Bennett. Lies, damned lies, and the orthogonality thesis. Under Review, 2025c ↩
-
Pei Wang. On defining artificial intelligence. Journal of Artificial General Intelligence, 10(2):1–37, 2019 ↩
-
See definition 5 in the appendix. ↩
-
Ben Goertzel. Artificial general intelligence: Concept, state of the art. Journal of Artificial General Intelligence, 5(1):1–48, 2014 ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. The artificial scientist: Logicist, emergentist, and universalist approaches to artificial general intelligence. In Artificial General Intelligence. Springer Nature, 2022b ↩
-
Richard Sutton. The bitter lesson. University of Texas at Austin, 2019 ↩
-
Murray Campbell, A. Joseph Hoane, and Feng hsiung Hsu. Deep blue. Artificial Intelligence, 2002 ↩
-
Ashish Vaswani et al. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, NY, 2017. Curran ↩
-
Note that I will prove an upper bound on embodied intelligence in this thesis. ↩
-
Sutton actually says ‘search’ and ‘learning’, but those terms are a bit ambiguous because a search algorithm can be used to learn. Hence to make the distinction clearer I’ll call these ‘search’ and ‘approximation’. Symbolic methods like traditional reinforcement learning fall into the search bucket. Curve fitting of any kind falls into approximation. ↩
-
Tom B Brown et al. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ‘20, NY, 2020 ↩
-
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis. Highly accurate protein structure prediction with alphafold. Nature, 2021 ↩
-
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020 ↩ ↩2
-
Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019. Association for Computational Linguistics ↩
-
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ‘21, page 610–623, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383097 ↩
-
Gary Marcus. Deep learning: A critical appraisal, 2018 ↩
-
S. Russell and P. Norvig. Artificial intelligence: A modern approach, global edition 4th. Pearson, London, 2021 ↩
-
Peter E. Hart, Nils J. Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2):100–107, 1968. DOI: 10.1109/TSSC.1968.300136 ↩
-
Henry Kautz and Bart Selman. Planning as satisfiability. In IN ECAI-92, pages 359–363, New York, 1992. Wiley ↩
-
A. Newell and H. Simon. The logic theory machine–a complex information processing system. IRE Transactions on Information Theory, 2(3):61–79, 1956 ↩
-
Christian Schulte and Mats Carlsson. Chapter 14 - finite domain constraint programming systems. In Francesca Rossi, Peter van Beek, and Toby Walsh, editors, Handbook of Constraint Programming, Foundations of Artificial Intelligence. Elsevier, 2006; Stefan Edelkamp and Stefan Schrödl. Chapter 9 - distributed search. In Stefan Edelkamp and Stefan Schrödl, editors, Heuristic Search, pages 369–427. Morgan Kaufmann, San Francisco, 2012; and Yichao Zhou and Jianyang Zeng. Massively parallel a* search on a gpu. Proceedings of the AAAI Conference on Artificial Intelligence, (1), 2015 ↩
-
Henry Kautz and Bart Selman. Planning as satisfiability. In IN ECAI-92, pages 359–363, New York, 1992. Wiley ↩
-
Murray Campbell, A. Joseph Hoane, and Feng hsiung Hsu. Deep blue. Artificial Intelligence, 2002 ↩
-
Peter E. Hart, Nils J. Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2):100–107, 1968. DOI: 10.1109/TSSC.1968.300136 ↩
-
Alex Krizhevsky et al. Imagenet classification with deep convolutional neural networks. Commun. ACM, 2017; and Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016 ↩
-
Ashish Vaswani et al. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, NY, 2017. Curran ↩
-
Volodymyr Mnih et al. Human-level control through deep reinforcement learning. Nature, 2015 ↩
-
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929–1958, 2014. ↩
-
Alex Krizhevsky et al. Imagenet classification with deep convolutional neural networks. Commun. ACM, 2017; and Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016 ↩
-
Ashish Vaswani et al. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, NY, 2017. Curran ↩
-
Jacob Devlin et al. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 2019 ↩
-
Tom B Brown et al. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, NY, 2020 ↩
-
Volodymyr Mnih et al. Human-level control through deep reinforcement learning. Nature, 2015 ↩
-
John Schulman et al. Proximal policy optimization algorithms, 2017 ↩
-
Elija Perrier and Michael Timothy Bennett. Position: Stop acting like language model agents are normal agents, 2025. arXiv ↩
-
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “why should i trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ‘16, page 1135–1144, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342322 ↩
-
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, NY, 2017. Curran ↩
-
Tom B Brown et al. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ‘20, NY, 2020 ↩
-
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, page 3320–3328, Cambridge, MA, USA, 2014. MIT Press ↩
-
Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019. Association for Computational Linguistics ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. The artificial scientist: Logicist, emergentist, and universalist approaches to artificial general intelligence. In Artificial General Intelligence. Springer Nature, 2022b ↩
-
David Silver et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587): 484–489, 2016 ↩
-
Michael Timothy Bennett and Yoshihiro Maruyama. Philosophical specification of empathetic ethical artificial intelligence. IEEE Transactions on Cognitive and Developmental Systems, 14(2): 292–300, 2022a ↩
-
A. Garcez, M. Gori, L. C. Lamb, L. Serafini, M. Spranger, and S. N. Tran. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. 2019 ↩
-
Marta Garnelo, Kai Arulkumaran, and Murray Shanahan. Towards deep symbolic reinforcement learning, 2016 ↩
-
John E. Laird. The Soar Cognitive Architecture. MIT Press, MA, 2012 ↩
-
John R. Anderson, Daniel Bothell, Michael D. Byrne, Scott Douglass, Christian Lebiere, and Yulin Qin. An integrated theory of the mind. Psychological Review, 2004. Because apparently six authors are needed to figure out how your brain works ↩
-
Ben Goertzel et al. Opencog hyperon: A framework for agi at the human level and beyond. Technical report, OpenCog Foundation, 2023 ↩
-
Ben Goertzel. Actpc-chem: Discrete active predictive coding for goal-guided algorithmic chemistry as a potential cognitive kernel for hyperon and primus-based agi, 2024 ↩
-
Eric Nivel et al. Autocatalytic endogenous reflective architecture. Technical report, Reykjavik University, School of Computer Science, 2013; and Kristinn R. Thorisson. A New Constructivist AI: From Manual Methods to Self-Constructive Systems, pages 145–171. Atlantis Press, Paris, 2012 ↩
-
P. Wang. Rigid Flexibility: The Logic of Intelligence. Applied Logic Series. Springer Nature, 2006 ↩
-
Elija Perrier and Michael Timothy Bennett. Position: Stop acting like language model agents are normal agents, 2025. URL arXiv ↩
-
Actually I proposed it in the papers and I rehash it here. ↩
-
Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In Artificial General Intelligence. Springer Nature, 2023a; Michael Timothy Bennett. A formal theory of optimal learning with experimental results. Forthcoming, IJCAI 2025, 2025e; Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a; and Michael Timothy Bennett. What the f*ck is artificial general intelligence? Under Review, 2025b ↩
-
Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, and Manfred K. Warmuth. Occam’s razor. Information Processing Letters, 1987 ↩
-
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929–1958, 2014. URL http://jmlr.org/papers/v15/srivastava14a.html ↩
-
Jorma Rissanen. Modeling by shortest data description. Automatica, 1978 ↩
-
Jürgen Schmidhuber. Discovering neural nets with low kolmogorov complexity and high generalization capability. Neural Networks, 10(5):857–873, 1997; and Marcus Hutter, David Quarel, and Elliot Catt. An Introduction to Universal Artificial Intelligence. Chapman and Hall/CRC, 1st edition, 2024. DOI: 10.1201/9781003460299 ↩
-
A.N. Kolmogorov. On tables of random numbers. Sankhya: The Indian Journal of Statistics, A:369–376, 1963 ↩
-
Gregory J. Chaitin. On the length of programs for computing finite binary sequences. J. ACM, 1966 ↩
-
Jorma Rissanen. Modeling by shortest data description. Automatica, 1978 ↩
-
J. Ziv and A. Lempel. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory, 23(3):337–343, 1977. DOI: 10.1109/TIT.1977.1055714 ↩
-
R.J. Solomonoff. A formal theory of inductive inference. part i. Information and Control, 7(1):1–22, 1964 ↩
-
Marcus Hutter. Universal Algorithmic Intelligence: A Mathematical Top-Down Approach, pages 227–290. Springer Berlin Heidelberg, Berlin, Heidelberg, 2007; and Marcus Hutter. Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer Nature, Heidelberg, 2010 ↩
-
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929–1958, 2014. URL http://jmlr.org/papers/v15/srivastava14a.html ↩
-
Patrick Hammer and Tony Loft-house. ‘opennars for applications’: Architecture and control. In Ben Goertzel, Aleksandr I. Panov, Alexey Potapov, and Roman Yampolskiy, editors, Artificial General Intelligence, pages 193–204, Cham, 2020. Springer Nature ↩
-
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929–1958, 2014. JMLR ↩
-
Marcus Hutter, David Quarel, and Elliot Catt. An Introduction to Universal Artificial Intelligence. Chapman and Hall/CRC, 1st edition, 2024. DOI: 10.1201/9781003460299 ↩
-
Jorma Rissanen. Modeling by shortest data description. Automatica, 1978 ↩
-
The last I propose in this thesis. ↩
-
D.H. Wolpert and W.G. Macready. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1):67–82, 1997. DOI: 10.1109/4235.585893 ↩
-
I show why simplicity and generalisation are correlated in this thesis, in chapter 14 ↩
-
Jan Leike and Marcus Hutter. Bad universal priors and notions of optimality. Proceedings of The 28th Conference on Learning Theory, in Proceedings of Machine Learning Research, pages 1244–1259, 2015 ↩
-
Ming Li and Paul M. B. Vitányi. An Introduction to Kolmogorov Complexity and its Applications (Third Edition). Springer Nature, New York, 2008 ↩
-
Jan Leike and Marcus Hutter. Bad universal priors and notions of optimality. Proceedings of The 28th Conference on Learning Theory, in Proceedings of Machine Learning Research, pages 1244–1259, 2015 ↩
-
Shane Legg and Marcus Hutter. Universal intelligence: A definition of machine intelligence. Minds and Machines, pages 391–444, 2007; and Shane Legg. Machine Super Intelligence. PhD thesis, Uni. of Lugano, 2008 ↩
-
Ming Li and Paul M. B. Vitányi. An Introduction to Kolmogorov Complexity and its Applications (Third Edition). Springer Nature, New York, 2008 ↩
-
L. A. Levin. Universal sequential search problems. Problems of Information Transmission, 9(3):265–266, 1973 ↩
-
Laurent Orseau. Asymptotic non-learnability of universal agents with neural networks. In Joscha Bach, Ben Goertzel, and Matthew Iklé, editors, Artificial General Intelligence: 5th International Conference, AGI 2012, pages 234–243, Berlin, Heidelberg, 2012. Springer Nature ↩
-
Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c ↩
-
L. A. Levin. Universal sequential search problems. Problems of Information Transmission, 9(3):265–266, 1973; Gregory J. Chaitin. On the length of programs for computing finite binary sequences. J. ACM, 1966; and Jorma Rissanen. Modeling by shortest data description. Automatica, 1978 ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a ↩
-
J. A. Fodor. Methodological solipsism considered as a research strategy in cognitive psychology. Behavioral and Brain Sciences, 3(1):63–73, 1980. DOI: 10.1017/S0140525X00001771 ↩
-
Laurent Orseau and Mark Ring. Space-time embedded intelligence. In Joscha Bach, Ben Goertzel, and Matthew Iklé, editors, Artificial General Intelligence, pages 209–218, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg. ISBN 978-3-642-35506-6 ↩
-
Laurent Orseau. Asymptotic non-learnability of universal agents with neural networks. In Joscha Bach, Ben Goertzel, and Matthew Iklé, editors, Artificial General Intelligence: 5th International Conference, AGI 2012, pages 234–243, Berlin, Heidelberg, 2012. Springer Nature ↩
-
Zoe Kleinman and Chris Vallance. AI ’godfather’ Geoffrey Hinton warns of dangers as he quits Google. BBC News, May 2023. URL https://bbc.com/news/world-us-canada-65452940. Accessed: 2025-03-13 ↩
-
Geoffrey Hinton. The forward-forward algorithm: Some preliminary investigations, 2022 ↩
-
Hubert L. Dreyfus. What Computers Can’t Do: A Critique of Artificial Reason. Harper & Row, 1972 ↩
-
Oron Shagrir. Why we view the brain as a computer. Synthese ↩
-
Daniel Hutto and Erik Myin. Radical enactivism: Basic minds without content, 2013 ↩
-
Gualtiero Piccinini and Corey Maley. Computation in Physical Systems. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, Stanford, Sum. 21 edition, 2021 ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a; Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c; and Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a ↩
-
Hubert L. Dreyfus. What Computers Can’t Do: A Critique of Artificial Reason. Harper & Row, 1972; Hubert L. Dreyfus. Why heideggerian ai failed and how fixing it would require making it more heideggerian. Philosophical Psychology, 20(2):247–268, 2007. DOI: 10.1080/09515080701239510; and Michael Wheeler. Martin Heidegger. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, Fall 2020 edition, 2020 ↩
-
Bas C. van Fraassen. Laws and Symmetry. Oxford University Press, 1989 ↩
-
Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a ↩
-
Ben Goertzel. The Hidden Pattern: A Patternist Philosophy of Mind. Brown-Walker Press, USA, 2006 ↩
-
Bas C. van Fraassen. Laws and Symmetry. Oxford University Press, 1989 ↩
-
Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c ↩
-
W.V.O. Quine. Philosophy of Logic: Second Edition. Harvard University Press, Cambridge MA, 1986. ISBN 9780674665637. http://www.jstor.org/stable/j.ctvk12scx ↩
-
Jacques Derrida. Writing and difference. U of Chicago P, 1978 ↩
-
Oron Shagrir. Why we view the brain as a computer. Synthese ↩
-
Jacques Derrida. Writing and difference. U of Chicago P, 1978; and J. Speaks. Theories of Meaning. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, Stanford, Spring 2021 edition, 2021 ↩
-
To avoid ambiguity, note that Pan-computational Enactivism refers to the formalism of enactive cognition based on Stack Theory. ↩
-
Evan Thompson. Mind in Life: Biology, Phenomenology, and the Sciences of Mind. Harvard University Press, Cambridge MA, 2007; John Vervaeke, Timothy Lillicrap, and Blake Richards. Relevance realization and the emerging framework in cognitive science. J. Log. Comput., 2012; John Vervaeke and Leonardo Ferraro. Relevance, Meaning and the Cognitive Science of Wisdom. Springer Netherlands, Dordrecht, 2013a; John Vervaeke and Leonardo Ferraro. Relevance realization and the neurodynamics and neuroconnectivity of general intelligence. In Inman Harvey, Ann Cavoukian, George Tomko, Don Borrett, Hon Kwan, and Dimitrios Hatzinakos, editors, SmartData, NY, 2013b. Springer Nature; and Daniel Hutto and Erik Myin. Radical enactivism: Basic minds without content, 2013 ↩
-
Gualtiero Piccinini. Physical Computation: A Mechanistic Account. Oxford University Press, UK, 2015; and Gualtiero Piccinini and Corey Maley. Computation in Physical Systems. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, Stanford, Sum. 21 edition, 2021 ↩
-
Hence I often refer to it as Pancomputational Enactivism. ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a; Michael Timothy Bennett. Is complexity an illusion? In Artificial General Intelligence. Springer Nature, 2024c; and Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
Realised meaning it is made real, or brought into existence. ↩
-
It can be referred to as Stack Theory because it has to be true no matter how far down the stack we go. ↩
-
Gualtiero Piccinini and Corey Maley. Computation in Physical Systems. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, Stanford, Sum. 21 edition, 2021 ↩
-
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, MA, 2018 ↩
-
A present state, or a point in time etc. Truth is reference dependent here. ↩
-
Johannes Jaeger, Anna Riedl, Alex Djedovic, John Vervaeke, and Denis Walsh. Naturalizing relevance realization: Why agency and cognition are fundamentally not computational. Frontiers in Psychology, 15, 2024 ↩ -
Realised meaning it is made real, or brought into existence. ↩
-
Gualtiero Piccinini. Physical Computation: A Mechanistic Account. Oxford University Press, UK, 2015 According to Gualtiero Piccinini in Physical Computation ↩ -
David Wallace. The Emergent Multiverse: Quantum Theory according to the Everett Interpretation. Oxford University Press, 05 2012. ISBN 9780199546961. DOI: 10.1093/acprof:oso/9780199546961.001.0001. URL https://doi.org/10.1093/acprof:oso/9780199546961.001.0001 As discussed by David Wallace in The Emergent Multiverse ↩ -
(notation)
E with a subscript is the extension of the subscript. For example,E sub l is the extension ofl . (intuitive summary)L v is everything which can be realised in this abstraction layer. The extensionE sub x of a statementx is the set of all statements whose existence impliesx , and so it is like the sub-table ofx ’s truth table for whichx is true. ↩ -
Forgive the abuse of notation, for the purpose of this line think of nand as a function in . ↩
-
For example, contains all the states where and ↩
-
For example states where the gate is off or destroyed. ↩
-
() ↩
-
() ↩
-
() ↩
-
Note that in the above example, none of contain the aspect . This will become important in later chapters when I introduce causal-identities. ↩
-
Again, I have violated my own rule and written out contents for these states for your intuition. ↩
-
R. Landauer. Irreversibility and heat generation in the computing process. IBM Journal of Research and Development, 5(3):183–191, 1961; and Seth Lloyd. Ultimate physical limits to computation. Nature, 406(6799): 1047–1054, 2000 ↩ -
Stevan Harnad. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1):335– 346, 1990. ISSN 0167-2789. doi: https://doi.org/10.1016/0167-2789(90)90087-6. URL https://www.sciencedirect.com/science/article/pii/0167278990900876; and Jacques Derrida. Writing and difference. U of Chicago P, 1978 ↩ -
David Wallace. The Emergent Multiverse: Quantum Theory according to the Everett Interpretation. Oxford University Press, 05 2012. ISBN 9780199546961. doi: 10.1093/acprof:oso/9780199546961.001.0001. URL https://doi.org/10.1093/acprof:oso/9780199546961.001.0001 ↩ -
Ilya Prigogine. From Being to Becoming: Time and Complexity in the Physical Sciences. W.H. Freeman, 1980 ↩ -
Andy Clark. Being There: Putting Brain, Body, and World Together Again. MIT Press, 1997 ↩ -
John Searle. Minds, Brains, and Programs. Behavioral and Brain Sciences, 3:417–457, 1980 ↩ -
David Wallace. The Emergent Multiverse: Quantum Theory according to the Everett Interpretation. Oxford University Press, 2012. ISBN 9780199546961. DOI: 10.1093/acprof:oso/9780199546961.001.0001. URL https://doi.org/10.1093/acprof:oso/9780199546961.001.0001 ↩
-
Robin Gandy. Church’s thesis and principles for mechanisms. In The Kleene Symposium. North-Holland, 1980; Ricard Solé et al. Fundamental constraints to the logic of living systems. Interface Focus, 2024; and Oron Shagrir. Why we view the brain as a computer. Synthese ↩
-
Patrick McMillen and Michael Levin. Collective intelligence: A unifying concept for integrating biology across scales and substrates. Communications Biology, 2024 ↩
-
C. Horsman, S. Stepney, R. C. Wagner, and V. M. Kendon. When does a physical system compute? Proceedings of the Royal Society A, 470(2169):20140182, 2014 ↩ -
James J. Gibson. The Ecological Approach to Visual Perception. Houghton Mifflin, 1979 ↩ -
Joshua Bongard and Michael Levin. There’s plenty of room right here: Biological systems as evolved, overloaded, multi-scale machines. Biomimetics, 8(1), 2023 ↩ -
Jerry A. Fodor. The Language of Thought. Harvard University Press, 1975 ↩ -
Johannes Jaeger, Anna Riedl, Alex Djedovic, John Vervaeke, and Denis Walsh. Naturalizing relevance realization: Why agency and cognition are fundamentally not computational. Frontiers in Psychology, 15, 2024 ↩ -
L. J. Savage. The Foundations of Statistics. John Wiley & Sons, NY, USA, 1954; and Ramon Ferrer i Cancho and Ricard Solé. The small world of human language. Proceedings of the Royal Society B: Biological Sciences, 268(1482):2261–2265, 2001. DOI: 10.1098/rspb.2001.1800 ↩
-
John Vervaeke, Timothy Lillicrap, and Blake Richards. Relevance realization and the emerging framework in cognitive science. J. Log. Comput., 2012; John Vervaeke and Leonardo Ferraro. Relevance, Meaning and the Cognitive Science of Wisdom. Springer Netherlands, Dordrecht, 2013a; John Vervaeke and Leonardo Ferraro. Relevance realization and the neurodynamics and neuroconnectivity of general intelligence. In Inman Harvey, Ann Cavoukian, George Tomko, Don Borrett, Hon Kwan, and Dimitrios Hatzinakos, editors, Smart Data, NY, 2013b. Springer Nature; and Johannes Jaeger, Anna Riedl, Alex Djedovic, John Vervaeke, and Denis Walsh. Naturalizing relevance realization: Why agency and cognition are fundamentally not computational. Frontiers in Psychology, 15, 2024 ↩ -
Gualtiero Piccinini and Corey Maley. Computation in Physical Systems. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, Stanford, Sum. 21 edition, 2021 ↩ -
Ben Goertzel. The Hidden Pattern: A Patternist Philosophy of Mind. Brown-Walker Press, USA, 2006 ↩ -
Mutually exclusive within a ‘world’ or timeline. ↩ -
Kevin J. Mitchell. Free Agents: How Evolution Gave Us Free Will. Princeton University Press, Princeton, NJ, 2023. ISBN 9780691226231 ↩ -
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a; and Michael Timothy Bennett. Are biological systems more intelligent than artificial intelligence? Forthcoming, 2025a ↩
-
Michael Timothy Bennett. The optimal choice of hypothesis is the weakest, not the shortest. In Artificial General Intelligence. Springer Nature, 2023a; and Michael Timothy Bennett. A formal theory of optimal learning with experimental results. Forthcoming, IJCAI 2025, 2025e ↩
-
Michael Timothy Bennett. Emergent causality and the foundation of consciousness. In Artificial General Intelligence. Springer Nature, 2023b; and Michael Timothy Bennett, Sean Welsh, and Anna Ciaunica. Why Is Anything Conscious? Preprint, accepted to and presented at ASSC27 and MoC5, 2024 ↩
-
David Hume. A Treatise of Human Nature. 1739 ↩
-
Alfred North Whitehead. Process and Reality. 1929 ↩
-
Charles Darwin. On the Origin of Species. 1859 ↩
-
Ilya Prigogine. From Being to Becoming: Time and Complexity in the Physical Sciences. W.H. Freeman, 1980 ↩
-
John Maynard Smith. Evolution and the Theory of Games. Cambridge University Press, 1982 ↩
-
Stuart A. Kauffman. The Origins of Order: Self-Organization and Selection in Evolution. Oxford University Press, 1993 ↩
-
Daniel C. Dennett. Darwin’s Dangerous Idea: Evolution and the Meanings of Life. Simon & Schuster, 1995 ↩
-
Seth Lloyd. Ultimate physical limits to computation. Nature, 406(6799): 1047–1054, 2000 ↩
-
David Deutsch. The Fabric of Reality: The Science of Parallel Universes–and Its Implications. Penguin Books, 1997 ↩
-
Michael Timothy Bennett. Lies, damned lies, and the orthogonality thesis. Under Review, 2025c ↩
-
Michael Timothy Bennett. Computational dualism and objective superintelligence. In Artificial General Intelligence. Springer Nature, 2024a ↩
-
Stack Theory is the idea that everything is an infinite state of abstraction layers. Pancomputational Enactivism is the formalisation of enactivism within Stack Theory. ↩
-
For the purpose of defining intelligence, we need some notion of value. I’ll get to where this comes from in the next section. ↩
-
(notation) If , then we will use subscript to signify parts of , meaning one should assume even if that isn’t written. (intuitive summary) To reiterate and summarise the above: An input is a possibly incomplete description of a world. An output is a completion of an input. A correct output is a correct completion of an input. ↩
-
(further intuitive summary) A -task is a formal, behavioural description of an aspect of the environment. For example, a self-organising biological system could be described as a task enumerating all behaviour in which it remains alive. It begins alive in circumstances given by inputs , and remains alive in circumstances given by outputs , and is dead in circumstances given by . Likewise, we could describe the game chess played from the perspective of white. We could say contains a state corresponding to each and every move of each and every possible game of chess, contains every possible sequence of moves in which the game has not ended and it remains possible for white to win, and contains every possible sequence ending in a move that means white has won. Tasks are behavioural descriptions of systems in the philosophical sense of the word, and we will next relate these ideas to machine functionalism. ↩
-
To repeat the above definition in set builder notation: $$
\Pi_\alpha = { \pi \in L_v : E_{I_\alpha} \cap E_\pi = O_\alpha } ↩