Your Brain Doesn't Command Your Body. It Predicts It. [Max Bennett]

What's really interesting about this book, Max, is, obviously, I've read loads and loads of books in the space and there's people like Hinton and Hawkins and Damasio and Friston. I mean, even like Sutton, and what's interesting is it's a bit like the blind men and the elephant. So, they've all got a completely different story to tell. And I think the magic that you have pulled off with this book is somehow you've woven it together into a coherent story. What do you think about that?

Well, first, I'm very appreciative of the kind words. I think I came from a very unique perspective just because I was a complete outsider. And I didn't come to it with the objective of writing an academic book at all. I came to it from the objective I was just learning on my own. And I just started building this corpus of notes because I was so independently curious. And I kind of stumbled on this idea really for myself of how do I make sense of all of these disparate opinions and really this complete lack of information about how the brain actually works.

And I had my own set of, I think, biases coming from sort of the technology entrepreneurial world where we tend to think about things as ordered modifications. When you think about product strategies or how to roll things out, we like to think about things as what's step one, then what's step two, what's step three. So I think I did have sort of a cognitive bias to when presented with an incredible amount of complexity to try and make sense of it in a similar type of way. But yeah, I think as an outsider, I felt very free to explore and cross the boundaries between fields.

I mean, I look at the book as a merging of three fields. One is comparative psychology. So trying to understand what are the different intellectual capacities of different species. Evolutionary neuroscience. So what do we know about the past brains of humans and the ordered set of modifications of how brains came to be? And then AI, which is how do we ground sort of the high flutin conceptual discussions about how the brain works in what works in practice? I think is a really important grounding principle because it helps really hold us accountable towards the principles that we think work. If we can't implement them in AI systems, it should make us question if we actually have the ideas right.

So yeah, I think being an outsider in some ways comes with disadvantages, but there are some advantages also, which is you're kind of free to borrow from a variety of different fields and think freshly about things.

Yes. I mean, if you can point to any particular ideas that you found really, really difficult to reconcile, what would those be?

One thing that's really challenging is if we were to actually lay out what's the data richness of comparative psychology studies across species. If you put that on a whiteboard and looked at it, you would realize we have so little data on what intellectual capacities different animals in fact have. For example, the lamprey fish, which is sort of the canonical animal that's used as a model organism for the first vertebrates. Because it's the of all vertebrates alive today, it's one of our most distant vertebrae cousins. To my knowledge, there are absolutely no studies examining the map-based navigation of the lamprey fish. So, we have no idea if it's in fact capable of recognizing things in 3D space.

Now, when we look at other vertebrates like tio fish, it seems like they're eminently capable of doing that. We look at lizards, they're eminently capable of doing that. So, we sort of infer it seems likely that the first vertebrates were able to do this. We know the brain structures from which it emerges and in reptiles and tilios fish are present in the lamprey. So, we sort of back into an inference that, okay, probably the lamprey fish can do that. But this is all sort of in some sense guessing and trying to put the pieces together from very little information. So I think that's one challenging aspect to reconcile.

The other one that's really hard is in neuroscience, there's a lot of really interesting ideas about how the brain might work that have not really been tested in the wild from the an AI perspective, and then there's a lot of AI systems that work really well that have diverged substantially from at least the evidence, what the evidence suggests is how brains work. So how do you bridge the gap between these two things? I think is a really fascinating space to operate in, which is like what can we learn about the brain if anything from the success of transformers as an example. What can we learn if anything from the success of generative models in general? What can we learn from the success of and failures of modern reinforcement learning?

I mean, in some ways reinforceable learning has been a success, in other ways it's really fallen short of what a lot of people hoped it would be. So I think the gap between neuroscience and AI is still a challenging one to bridge in a lot of ways. For example, Carl Friston has all these incredible ideas in active inference. In a hundred years, will we look back on this and be like Carl Friston was on to something? If you look at the AI systems today, there's very little usage of active inference principles working in practice. So that could mean that the ideas don't have legs, or it could mean that there's a breakthrough behind the corner where we're actually missing some of the key principles that he's devising. And these are questions we don't have the answers to.

I think there might possibly be some breakthroughs around the corner. I don't know if you know, but I'm Carl Friston's personal publicist. I do all of his stuff. I probably interviewed him more than anyone else, but I love my friend. He's an amazing guy.

Yeah, he's an amazing man. Honestly, he is the man. He is the man. And so kind.

I know. For me as well, he has so much time to explain things. But yeah, you could cynically argue that the effective active inference agent is just a reinforcement learning agent of a particular variety. I think it's equivalent to a like an inverse reinforcement learning maximum entropy agent or something. But there's so much more than that. There's so much richness and explanatory power of kind of modeling this thing as as a generative model that can generate policies and plans of actions and and so on. And also we want to have agents that you know we understand what they're doing and and like with steerability and being able to do the simulations that you talk to so eloquently in in your in your paper but just I love for instance we'll do we'll do him properly later but to come back to what you were saying though so you were saying oh you know there might be a parallel between transformers and AI models in in your book actually on this page you you sort of like g you you kind of analogize modelbased reinforcement learning and the neoortex.

And of course I interviewed Hawkins back in the day and the main criticism of his book is that you know the triune brain type argument. So he's kind of giving giving the explanation that the brain developed a bit like geological strata with one layer and then another layer rather than kind of co-evolving together. And it's so hard not to think like that because you give so many beautiful examples in your book, not only just morphologically, but but in terms of capability, how the neoortex, I mean, you said like with stroke victims, for example, it's not like the brain recovers those dead cells. It learns to kind of repurpose those functions in other parts of the brain. So, it seems like Mount Castle was correct in the neoortex is this just magic general purpose learning system. I mean, what do you think?

So I think well there's two different there's two different ways to look at the neoortex enabling things like mental simulation and model free or model based reinforcement learning. One is that that function and algorithm is being implemented in the neoortex. But another which is a slightly less strong claim which is the one I would make is the addition of the neoortex enables the overall system to engage in this process. So it is not saying that the entire process is implemented in the neoortex. I think it seems very clear that the phalamus and basil ganglia are essential aspects of enabling the pausing the mental simulation the modeling of one's own intentions the evaluation of the results etc. But it is possibly to say which is what I'm argue in the book is in the absence of the neoortex that process does not happen.

And so I think where my ideas would synergize with what Hawkins is saying is the neoortex builds a very rich model of the world and a model of sufficient richness that you can explore it in the absence of sensory input. And that's a really essential aspect of model-based reinforcement learning because if I have a a model of the world that has sufficient richness that I can mentally simulate actions that I'm not actually taking and it at least somewhat accurately predicts the real consequences of those actions. And that model is really useful because I can now imagine outcomes before having them. I can flexibly adjust to new situations.

And of course there's so many really deep interesting questions that are yet to be answered about that. For example, just because you can render a simulation of the world doesn't answer the question, okay, what do you simulate? I mean, this is one of the one of the hardest problems of model based reinforcement learning is fine, you can have a model of the world, but how do you prune the search space of which aspects of that model you explore before evaluating outcomes? And so that's another really hard challenge that I think there's a lot of good evidence is actually a partnership between the neoortex and the and the basil ganglia, which is a much older structure.

So yeah, I'm not really of the, you know, I think the triune brain has been, you know, amongst evolutionary neuroscientists largely discredited. And I think in part somewhat unfairly if you actually read McClean's writings, he was he actually is very open about the fact that, you know, this is an approximation. It's not exactly accurate. Like he he couches his claims much more than the popular culture than just converted into a dogma. But I think the the popular interpretation of the triune brain is not accurate which is it is clearly not the case that the brain evolves in three key layers. It's not the case that a reptile brain doesn't have anything limbic like if you the a reptile brain absolutely has a cortex that does a lot of what our limbic structures do etc etc. So yeah that those would be my thoughts on that.

Yeah it it's fascinating because we as humans need to have models to explain and understand the thing itself just like active inference for example. I mean, I I'll get to planning and agency and goals in a little while, but a lot of these things are instrumental fictions. I'm not saying that our brains don't plan, but you know, like the way that the abstract mathematical way that we understand planning, it's probably not how the brain works. It's it's much more complicated than that. But why don't we just rewind to the beginning? So, you know, we're going to be talking about this chapter on on, you know, simulation if you like. And you kind of lead by saying that what the neoortex does is it does learning by imagining. And Hawkins spoke about this as well. He said we've got the matrix inside our brains, right? We're always just doing all of these simulations of of future things. And we're using that to kind of help us understand the world. And you you give this really interesting example of some of the features of the brain that kind of lead you to believe that we are basically living in a simulation. And it's almost like rather than perceiving things, we're testing if our simulation is correct, but that means that we can only simulate one thing at a time. So we can't we can't see two things. We can only see one thing. So can can you talk through that?

Sure. So one of the first sort of introspections and explorations into how perception works in the human mind happened in the late 19th century with all of these explorations of visual illusions that you see in pretty much every neuroscience textbook or book that you open. So listeners will be familiar with you've probably seen examples of triangles where you actually perceive a triangle in a picture when there is in fact no triangle. Or yes that isn't that yeah you can find that picture. Yeah sorry I hope I'm not distracting you.

No no no. So that's a you know a standard finding that was observed in the 19th century which is this idea that clearly the brain observes the presence of things even though they're not actually there. So we perceive a triangle there, we perceive a sphere. We perceive sort of a bar. we perceive the word editor when in fact if you actually examine that the word E is not there. There's evidence that suggests E is there by virtue of showing the shadows but we did not actually write the letter E there but the brain regularly observes that. So that finding led this scientist Herman von Helmholtz to sort of come up with this concept that what the brain what you actually consciously perceive is not your sensory stimuli. You are not receiving sensory input and experiencing the sensory input. What's happening is your brain is making an inference as to what is true in the world, what's actually there and then the evidence is just the sensory input is giving evidence to your brain as to what's there. And so you start from this prior and then that prior maintains itself until you get sufficient evidence to the contrary and then you change your mind.

And so it's not hard to imagine why this would be extremely useful in any sort of environment that an animal might evolve in. You know, suppose you have a mouse running across a tree branch at night. First I see the tree branch and the moonlight. So I build a mental model of the tree branch. As I move forward, I lose moonlight. I no longer see the tree branch. As long as as I'm stepping forward, the evidence is consistent with my prior of the tree branch. it makes way more sense for me to maintain the mental model of that tree branch as opposed to all of a sudden the tree branch disappears because I no longer see the sensory stimuli of it. So because sensory stimuli is very noisy, it makes a lot of sense that we in integrate it over time, build a prior and then until something gives us evidence to the contrary, we maintain our prior about the world. So so that was sort of the first idea that there's some form of inference. there's some ch there's some difference between sensory input and some model of the world that we infer and then thus perceive.

What's interesting that is not as discussed but is also present in the discussions amongst scientists in the late 20th century about this late 19th century about this is this idea that you can't actually render a simulation of two things at once. So there's lots of really interesting sort of visual illusions around this where you can see something. The famous one is it's either a duck or a rabbit.

Yep. Exactly. And it's interesting. Yeah. And then so you can see that staircase is either moving up to the left or you're under the staircase looking upwards and it's actually a ceiling that's jagged. And why can't the brain perceive both of those things at the same time? Well, it would make sense if you have a model that there are such things as ducks, there are such things as rabbits, there are such things as 3D shapes that operate under certain assumptions. And if that's true, then you cannot see a duck and a rabbit at the same time because there's no such thing. It cannot be the case that the staircase is looking from above and below at the same time. So what your brain is not doing is just perceiving the sensory stimuli. It's trying to infer what is a real 3D thing in the world that I am aware about that this sensory stimuli is suggesting is true and that is the thing that I'm going to render in your mind.

And I think one way this parallels nicely to some of Hawkins's ideas actually is if you hold the thousand brains theory to be true, which is the the neoortex has sort of all of these redundant overlapping models of objects, then it would make a lot of sense that we want to synergize these models to render one thing at a time. You don't want to have 15 different things rendered because then it's really hard to evaluate them and vote between these different columns. So it makes sense that the brain does is say, "Let me integrate all the input across sensory stimuli and render one sort of symphony of of models in my mind so I can see one thing at a time." So that's this idea of perception by inference. At the time no one really connected that to the idea of planning. So this was just this idea that what we perceive is different from the sensory stimuli we we get.

But later on sort of as a the world of AI started thinking about things from the perspective of perception by inference what we end up realizing is this idea of perception by inference if you're going to train a model to do that it comes with this notion of generation so because the way it self-supervises is it takes the prior and it tries to make predictions and it compares the predictions in the world to the sensory stimuli and as long as those predictions are below a threshold I maintain my prior. So a famous version of this is the Helm Holtz machine which Hinton devised. I think that was in the 80s. It could be later I forget.

Yep. And so this is this basic idea that you can build a model he called, you know, a lot of people use the term latent representation. Some people don't like that term for a variety of philosophical reasons. But it builds a representation of things by virtue of building a model. In other words, perception by inference. And the way you build that model is you're constantly comparing generating predictions from that model to what actually occurs. And this also has synergies to a lot of Hawkins's ideas where we think about intelligence as prediction. And so what that means is if you if you build a model of perception by inference by virtue of generation then it's relatively easy to say okay well what happens if I just turn off sensory stimuli and I start exploring the latent representation. Well now we're exploring a simulated world. I'm able to cut off sensory stimuli, close my eyes, and imagine a chair, rotate the chair, change the color of the chair.

And because this model is a relatively good, has relatively rich features about how the world actually works, I can model things without ever having experienced them, without ever having done them and relatively reasonably predict what what is real is would actually happen if I were to do those things. So what I think is interesting and perhaps somewhat of a novel proposal in the book is I think a lot of people think about the neoortex as having adaptive value because of how good it is at recognizing things in the world. Right? So a lot you know if you read a standard textbook a lot of what people will talk about the neoortex is how good it is at perceiving things object recognition. Some of the best studied parts of the brain are this visual neoortex. So we understand reasonably well how we're building models of visual objects etc. But from an evolutionary perspective this is a little bit hard to to find convincing because if you actually examine the object recognition of vertebrates it's incredibly accurate. I mean a fish can recognize human faces. A fish can oneot recognize an object when rotated in 3D space. So, it's hard to find a dividing line between object recognition in animals with a neoortex and object recognition in objects without a neoortex and with with brain structures that seem more similar to early vertebrates.

Did you have a question?

Yeah. Well, well, yeah, I just wanted to touch on a couple things there. So Hawkins said that the reason I mean he said first of all that we overcome the binding problem by having this kind of profusion of individual sensory motor models rather than having this kind of you know feed forward enrichment of representations and that was really interesting but he also said the reason why having let's say we've got 150,000 mini cortical columns that are just wired to different sensory motor signals and he said it's the it's the kind of diversity and sparsity that gave the robustness of recognition And then you can kind of think okay well well what do the representations look like? So if our brain builds some kind of a a model of the world some kind of topological model it it must be a representation not necessarily it's not like a a hunulous or it's not like a stapler inside the brain if I'm modeling a stapler it's actually some weird structure of the stapler but as seen by every way I can touch feel here and you know lick a stapler or whatever. So it's difficult for us to imagine what that is. But the reason I'm the reason I'm going down this road though because that's a bit weird, isn't it? So we have this very weird representation of things and then I come to Hinton's Helm Holtzian generative model and you can get it to generate let's say a number eight if it's trained to do numbers right and Hinton would argue and I would disagree with him that that the model understands what an eight is. Now, this is weird, isn't it? Because we understand what a mouse is, but intuitively we feel that a neural network doesn't understand what an eight is. And I would argue that we're getting into semantics here. So, I think the reason we understand things is because there's a relational component to understanding. So, semantics is is about the ontology. you know our rep the way we feel about things where the thing came from what was the intention guiding the creation of the thing what was the provenence of the thing it's almost like the interconnectedness of the thing tells you more about the meaning of the thing than the actual thing itself in a weird way what what do you think?

Well I do think I think uh this is where uh the word understanding I think can mean different things to different people can understand the word understanding in different ways I think there is absolutely uh something to the idea that uh just because you can recognize something. So that would be a feed forward network that can observe a stapler. uh alone is insufficient to what most people would mean when they use the word understanding. And just to talk about interrelatedness, if I have a feed forward network that you know let's say it's just a binary classifier, is this thing a stapler or not? Now I can't ask lots of things of that feed forward network that I would expect of a an agent or a model that understands what a stapler is. I couldn't ask it, for example, what would happen if I burned the stapler? What would happen if I opened it? What would you see inside of it? I can't ask what does a stapler do? I can't say show show a human holding a stapler and a set of objects in front of them and what do you think the person's going to do next with this stapler? And so and so clearly understanding most of our intuitive understanding of the word understanding contains some richness that's not included in just classifying or recognizing the presence of objects. So I think that's absolutely the case. My intuitions sort of fall in the direction of what we mean typically when we use the word understanding it comes to having something that can be mentally explored. And so I think that could that requires what you're describing, which is the interrelatedness of things because when I see someone holding a stapler and you ask me the question, what are the things they would likely do? Well, I start imagining what they might do with that stapler and then I can evaluate which ones seem plausible to me. And so in the imagination and evaluation of which things seem plausible, there's an interconnectedness between the thing and the the world around it. But yes I if I absolutely agree with you that just recognizing objects clearly lacks something that we mean when we say understanding.

Yeah. Do do you take into consideration the memephere? So you know we've got this kind of we've got ideas that are quite collective as well. I guess I was going to explore this with you later but maybe maybe let me frame it like this. I I have this intuition that a lot of our intelligence is outside of the brain. So, it's almost like if if I was in the wilderness and, you know, disconnected from society, I would be much, you know, a lesser human being. In a weird way, I might have more agency, but, you know, I I wouldn't have access to all of these rich cultural tools and knowledge and patterns and and so on. So, it it's almost like that's where a lot of our intelligence comes from rather than and and and also meaning as well rather than than just being able to plan in the brain and so on. And there must be some kind of interplay. So culture must shape the development of our brain and and vice versa. But culture seems to be more dynamic. But how how do you wrestle with that?

Well, I mean there it's that's undeniably true. I mean the the first example that comes to mind is just writing. I mean what would humans be if you removed the technology of writing? I mean we would we would all realize we're not that smart. I mean writing is a technology that externalizes a feature of the brain. Which is memory which brains are not that great at. We do a good job of like condensing aspects of a memory. So we uh for episodic like things and procedural memory we're relatively good at. But for semantic memories we're terrible. And so externalizing that with writing is one of the key technologies that enables us to be way smarter than we are in fact because we now have this external device that enables us to store largely infinite numbers of memories and translate them across generations. So that alone I think proves your point which is what humans are capable of is clearly some relationship between brains and external things that can be writing tools that can be other brains you know sharing ideas and getting challenged. So yeah you're absolutely I agree with you in intelligence is like the it's the dynamics isn't it? It's it's all of the low-level, you know, dynamics of things interacting with each other. And we can take a snapshot of language and and we might say, "Oh, well, yeah, you know, language isn't isn't the intelligence, but but language itself is a form of intelligence." I'm not just talking about the words and language models and so on. We're talking about the actual language in in our culture. And I guess I I think of that as as almost a distinct form of intelligence.

Yeah. Yeah, I mean I I think it we all it almost gets into a philosophical territory as where do you draw the bounding boxes over the things that are imbued with intelligence and the things that are supportive mechanisms that that allow those things to have intelligence. So through one lens you could you could think about brains as the physical entity in which intelligence is instantiated and then language is a supportive tool. You could take a very perhaps odd view which is language is the thing that's evolving and it's just instantiated in these brains that sort of produce the language, consume the language, but it's language that's that's evolving. The same way that you know we don't think about intelligence on the level of an individual neuron. We don't imbue a neuron with intelligence but we think on the scale of the 86 billion neurons something has emergently appeared that we do deem intelligent. You know, there's some great sci-fi books, where, you know, intelligence gets instantiated in colonies of ants where each individual ants isn't not intelligent, but somehow the colony itself is capable of doing incredible abstractions and whatever. So, so yes, I think there's there's very interesting ways to and it's not obvious how one divides the lines between what are the physical entities in which these are instantiated.

That said, I have a particular interest in brains because I think if we're looking for for what is the physical manifestation that we can learn from and thus try to one to understand ourselves just I think all species have an interest in understanding ourselves. But then also if we want to try and borrow some ideas from how biological intelligence works into AI, I think the of all the physical things to examine, the brain to me seems clearly the one that is probably the most rich with insight. But I no I agree. I I think your point is well taken.

Interesting. Okay. I want to close the loop on what what you said about the brain being an imagination filling in machine. So you said that it does filling in. It's one at a time. it can't unsee visual illusions and evidence is seen in the wiring of the neoortex itself you say. So it's shown to have many properties consistent with a generative model. The evidence is seen in the surprising symmetry the ironclad inseparability between perception and imagination that is found in generative models in the neoortex. And you give examples like illusions and how humans succumb to hallucinations why we dream and sleep and even the inner workings of imagination itself. So it really seems plausible when when you kind of think of it in that way.

Yeah. I think I think there's there's a reason why so much of the neuroscience community I mean what I'm saying there is not really novel so much the neuroscience community has sort of rallied around this idea of predictive coding which is very related to active inference and generative models because there's just so much evidence that what's going on in the neoortex the imagination of things episodic memories I mean it's there's been some good evidence that epis sodic memory and in other words thinking about past and imagining the future are in fact the same underlying process happening in the neoortex which is again consistent with this idea that there's a generative model if we look at the connectivity patterns I didn't talk about this too deeply in the book is a little technical but what you would expect from a generative model is backwards connections would be much richer than forward connections because you're modulating downstream of course neo course is not perfectly hierarchal but things that are in general lower in in the hierarchy would have lots of of inputs from parts of the neoortex that are higher in the hierarchy. That's absolutely what we see. So yeah, there's a lot of evidence that these are two sides of the same coin which is there's some form of generative model being implemented.

I do think in AI one way in which this manifests is the the very clear success of self-s supervision. I mean this idea although the actual predictive coding algorithms that people have sort of devised as the neo as neoortex implementing when we've actually modeled them they haven't outperformed any of the stuff going on in in AI world the principle of self-s supervision which is can a system end up having really interesting emergent properties and generalize well when you only train it on predicting sensory input that it receives. And that clearly has become the case. I mean if the transformer I think is a great example of if you just give it a bunch of data and you train it through self-supervision i.e. masking so you hide certain data inputs it becomes remarkably accurate and good at generalizing across data that it hasn't seen before which is in principle all a gener what people are predicting or claiming that the neoortex is doing as generative model.

Yeah, it it feels to me that there is a bright difference though between let's say a transformer and and the neoortex. And I think the difference is maybe agency is not the right word, but that you can think of the the neurons I think as having some kind of autonomy. So they're sending messages to each other and then the other, you know, it's eventually consistent. So the other neuron will get the message and it will decide itself what it's going to do. and in a transformer just because of the way they're connected and the backrop algorithm and so on they they all um they all kind of ride a Mexican wave together to use an analogy. So it feels like a difference in kind to me.

Well I mean clear clearly as I argue in the book I don't think that the brain is just one big transformer. So I would agree with you in in the human brain unless you think there's something that's non-deterministic and sort of magical happening. Uh you know I think you would still say that there is either uh you know base firing rates of neurons uh that and then there's sensory input that flows up and then goes through the brain until eventually there's sort of it's affecting uh muscles until you're responding. So there is you know there is sort of a deterministic flow happening. it might not be as uh feed forward as what's happening in transformer which is definitely the case. But they might both be sort of uh you know deterministic in in a in a similar fashion.

I think a lot of people I've had this exact sort of argument with a lot of people and one counterargument that people have towards this idea I don't know if I fully agree with it but it's interesting is that attention heads really are doing something more magical than we give them credit for which is they are kind of dynamically rerouting and effectively resetting the network based on the context that the prompt is getting. And so although technically it's just a series of you know matrix multiplications etc. If in principle what's happening is these attention heads are doing something really clever where they're looking at the context of a prompt and then effectively dynamically reweing the network to decide what it cares about and what it doesn't. So there are people that think, you know, there is something really interesting happening in the transformer that might be analogous to certain things that are happening in the brain. But yeah, clearly, you know, these these feed forward networks are are not capturing everything that's going on in the brain.

Yeah, it's really interesting what you said because the way I read that is things like chatbt and language models, they are entropy smuggling or agency smuggling. So what that means is they kind of just do what you tell them to do and all of the kind of the agency. So my directedness comes from me. So I give it a prompt, it does the thing that I wanted it to do and then the kind of the the mapping that you were talking about, I interpret that a bit like a database query. So you know depending on the prompt you give it it'll activate a certain part of the representation space and it will give you a certain result back. But the the the brain has this thing where all of the neurons have their own directedness and and the weird thing is at the cosmic scale agency is it seems to emerge. So even transformer models that were acting autonomously could presumably in large enough scale give rise to something that we think of as directedness or goals or purpose or or whatever. But it's almost like in the natural world because there are so many levels, scales and scales of independent autonomous things just kind of mingling with each other independently and then like downstream mixing their information together and rinsing and repeating over many many different scales. That seems to be the thing that gives rise to all of these amazing things like agency and creativity and and etc.

Yeah. Yeah, I mean I think the the notion of agency is an interesting one where I really am amenable. I mean there is sort of a a I don't know if I would call it a schism in the field but there are there are there is a debate where between sort of the reinforcement learning world and the active inference world where how much of intelligence can be conceived as optimizing a reward function and sort of the re the hardcore reinforcement learning world is like everything is just a reward and then the active inference world is you know would argue that not all behavior is driven just by optimizing a single reward function. There is some uncertainty minimization. There is trying to satisfy your own model of yourself, fulfill your own predictions. These sort of things that seem very well aligned to behavior we see. But uh you know it's unclear which of these is right. It's probably some balance of the two. But to me agency people would conceive of agency differently in these two worlds, right? So I think some people in the RL world would say agency is just you give something a reward function and then it just learns over time try and optimize that reward. In the more active inference world which I do think has legs and I'm obviously amendable to the idea of agency is a little bit more. It's building a model of yourself and trying to infer what your goals are based on observing yourself and then trying to make predictions to fulfill those end goals. In other words, it's constructed. Goals are constructed.

And this is sort of one of my favorite Friston papers is predictions not commands. I don't know if you've read that paper, but I think it's a brilliant paper about how you