I’ve been thinking about representations a lot lately. More specifically, I have been thinking about the possibility of noncomputational representation. On first blush, this sounds strange because representationalism has for a long time been intimately connected with the Computational Theory of Mind, which basically says that the brain is some kind of computer, and that cognition is most basically the manipulation of abstract quasi-linguaform representations by means of a low-level syntactic realizer base. I’ve never been quite sure how this is supposed to work, but the gist of it is captured by the software/hardware distinction. The mind is the software of the computing brain. Representations, in virtue of their supposed quasi-linguaform nature, are often thought of in terms of propositions. For a brain to know that P, it must have a representation or belief to the effect of that P. As it commonly goes, computation is knowledge, knowledge is representational, the brain represents, the brain is a computer.
But in this post I want to explore the idea of noncomputational representation. The basic idea under question is whether we can say that the brain traffics in representations even though it is not a computer i.e. if the brain is not a computer, does it still represent things, if so, how and in what sense? Following Jeff Hawkins, I think it is plausible to suppose that the brain is not a digital computer. But if the brain is not computing like a computer in order to be so intelligent, what is it doing? Hawkins thinks that the secret of the brain’s intelligence is the neocortex. He thinks that the neocortex is basically a massive memory-prediction machine. Through experience, patterns and regularities flow into the nervous system in terms of neural patterns and regularities. These patterns are then stored in the brain’s neocortex as memory. It is a well-known fact that cortical memories are “stored” in the same place as where they were originally taken in and processed.
How is this possible? Hawkins’ idea is that the reason why we see memory as being “stored” in the original cortical areas is that the function of storing patterns is to aid in the prediction of future patterns. As we experience the world, the sensory details change based on things like our perspective. Take my knowledge of where my chair is in my office. After experiencing this chair from various positions in the room, I now have a memory of where the chair is in relation to the room, and I have a memory of where the room is in relation to the house, and the house in relation to the neighborhood, and the neighborhood to the city, and so on. In terms of the chair, what the memory allows me to do is to “know” things about the chair which are independent of my perspective. I can look at the chair from any perspective and recognize that it is my chair, despite each sensory profile displaying totally different patterns. How is this possible? Hawkins idea is that the neocortex creates an invariant representation of the chair which is based on the integration of lower-level information into a higher-order representation.
What does it mean to create an invariant representation? The basic idea here can be illustrated in terms of how information flows into and around the cortex. At the lowest levels, the patterns of regularities of my sensory experience of the chair are broken up into scattered and modality-specific information. The processing at the lowest levels is carried out by the lowest neocortical layers. Each small region in these layers has a receptive field that is very narrow and specific, such as firing only when a line sweeps across a tiny upper-right quadrant in the visual field. And of course, when the information comes into the brain it is processed by contralateral cortical areas, with the right lower cortical layers only responding to information streaming in from the left visual field, and vice-versa. As the modality specific and feature-oriented information flows up the cortical hierarchy, the receptive fields of the cells becomes broader, and more steady in the firing patterns. Whereas the lower cortical areas only respond to low-level details of the chair, the higher cortical areas stay active while in the presence of the chair under any experential condition. These higher cortical areas can thus be said to have created an invariant representation of the patterns and regularities which are specific to the chair. The brain is able to create these representations because the world actually is patterned and regular, and the brain is responding to this.
So what is the cash value of these invariant representations? To understand this, you have to understand how once the information flows to the “top” of the hierarchy (ending in the hippocampus, forming long-term memories), it flows back down to the bottom. Neuroanatomists have long known that 90% of the connections at the lower cortical layers are streaming in from the “top”, and not the “bottom”. In other words, there is a massive amount of feedback from the higher levels into the lower levels. Hawkins’ idea is that this feedback is the physical instantiation of the invariant representations aiding in prediction. Because my brain has stored a memory/representation of what the chair is “really” like abstracted from particular sensory presentations, I am able to predict where the chair will be before I even walk into the room. However, if I walked into the room and the chair was on the ceiling, I would be shocked, because I have nothing in my memory about my chair, or any chair, ever being on the ceiling. Except I might have a memory about people pulling pranks by nailing furniture to ceilings, so after some shock, I would “re-understand” my expectations about future perceptions of chairs, being less surprised next time I see my chair on the ceiling.
Hawkins think that it is this relation between having a good memory and the ability to predict the future based on that memory which is at the heart of intelligence. In the case of memories flowing down to the sensory cortices, the “prediction” is one that predicts what future patterns of sensory activity are like. For example, the brain learns Sensory Pattern A and creates a memory of this pattern throughout the cortical hierarchy. The most invariant representation in the hierarchy flows down to the lower sensory areas and fires the Pattern A again based on the memory-based prediction about when it will experience Pattern A again. If the memory-prediction was accurate, the incoming pattern will match Pattern A, and the memory will be confirmed and strengthened. If the pattern comes in is actually Pattern B, then the prediction will be incongruous with the incoming information. This will cause the new pattern to shoot up the hierarchy to form a new memory, which then feedbacks down to make predictions about future sensory experience. In the case of predictions flowing down into the motor cortices, the “predictions” are really motor commands. If I predict that if I walk into my office and turn right I will see my chair, and if the prediction is in the form of a motor commond, the prediction will actually make itself come true if the chair is where the brain predicted it will be. Predictive motor commands are confirmed when the prediction is accurate, and disconfirmed if inaccurate.
So, a noncomputational representation is based on the fact that the brain (particularly the neocortex) is organized in an hierarchical memory system based on neuronal patterns and regularities, which in turn are composed of synaptic mechanisms like long-term potentiation. According to Hawkins, it is the hierarchy from bottom to top and back which gives the brain its remarkable powers of intelligence. The intelligence of humans for Hawkins is really a product of having a very good memory and being able to anticipate and hence understand the future in incredibly complex ways. If you understand a situation, you will not be surprised because your memory is so accurate. If you do not understand it, you cannot predict what will happen next.
An interesting feature of Hawkins’ theory is that it predicts that the neocortex is fundamentally running a single algorithm: memory-prediction. So what gives the brain its adult modularity and specialization? It is the specific nature of the patterns and regularities of each specific sensory modality flowing into the brain. But the common currency of the brain is patterns of neuronal activity. Thus, every area of the cortex, could, in principle, “handle” any other neuronal pattern. Paul Bach-y-Rita’s research on sensory substitution is highly relevant here. Bach-y-Rita’s research has shown that the common currency of the perception is the detection and learning of sensory regularities. His research has, for example, allowed blind patients to “see” light by wiring a camera onto their tongues. This is to be expected if the neocortex is running a single type of algorithm. So what actually “wires” a cortical subregion is the type of information which streams in. Because auditory data and visual data always enter the brain from unique points, it is not surprising that specialized regions of the cortex “handle” this information. But the science shows that if any region is damaged, a certain amount of plasticity is capable of having other areas “take over” the input. This is especially true in childhood. What Micah Allen and I have tried to show in our recent paper is that higher-order functions of humans are based on the kinds of information available for humans to “work with”, namely, social-linguistic information. So the key to human unique cognitive control is not having an evolutionary unique executive controller in the brain. Rather, the difference is in what kinds of information can be funneled into the executive controller. For humans, a huge amount of the data streaming in is social-linguistic. Our memory-prediction systems thus operate with more complexity and specialization because of the unique social-linguistic nature of the patterns which stream into the executive. So to answer Daniel Wegner’s question of “Who is the controller of controlled processes?“, the uniqueness of “voluntary” control is based on the higher-level invariant memories being social-linguistic in nature. The uniqueness of the information guarantees that the predictions, and thus behavior, of the human cognitive control system will be unique. So we are not different from chimps insofar as we have executive control. The difference lies in what kinds of information that control has to work with in terms of its memory and predictive capacities.