Tag Archives: memory

Noncomputational Representation

I’ve been thinking about representations a lot lately. More specifically, I have been thinking about the possibility of noncomputational representation. On first blush, this sounds strange because representationalism has for a long time been intimately connected with the Computational Theory of Mind, which basically says that the brain is some kind of computer, and that cognition is most basically the manipulation of abstract quasi-linguaform representations by means of a low-level syntactic realizer base. I’ve never been quite sure how this is supposed to work, but the gist of it is captured by the software/hardware distinction. The mind is the software of the computing brain. Representations, in virtue of their supposed quasi-linguaform nature, are often thought of in terms of propositions. For a brain to know that P, it must have a representation or belief to the effect of that P. As it commonly goes, computation is knowledge, knowledge is representational, the brain represents, the brain is a computer.

But in this post I want to explore the idea of noncomputational representation. The basic idea under question is whether we can say that the brain traffics in representations even though it is not a computer i.e. if the brain is not a computer, does it still represent things, if so, how and in what sense? Following Jeff Hawkins, I think it is plausible to suppose that the brain is not a digital computer. But if the brain is not computing like a computer in order to be so intelligent, what is it doing? Hawkins thinks that the secret of the brain’s intelligence is the neocortex. He thinks that the neocortex is basically a massive memory-prediction machine. Through experience, patterns and regularities flow into the nervous system in terms of neural patterns and regularities. These patterns are then stored in the brain’s neocortex as memory. It is a well-known fact that cortical memories are “stored” in the same place as where they were originally taken in and processed.

How is this possible? Hawkins’ idea is that the reason why we see memory as being “stored” in the original cortical areas is that the function of storing patterns is to aid in the prediction of future patterns. As we experience the world, the sensory details change based on things like our perspective. Take my knowledge of where my chair is in my office. After experiencing this chair from various positions in the room, I now have a memory of where the chair is in relation to the room, and I have a memory of where the room is in relation to the house, and the house in relation to the neighborhood, and the neighborhood to the city, and so on. In terms of the chair, what the memory allows me to do is to “know” things about the chair which are independent of my perspective. I can look at the chair from any perspective and recognize that it is my chair, despite each sensory profile displaying totally different patterns. How is this possible? Hawkins idea is that the neocortex creates an invariant representation of the chair which is based on the integration of lower-level information into a higher-order representation.

What does it mean to create an invariant representation? The basic idea here can be illustrated in terms of how information flows into and around the cortex. At the lowest levels, the patterns of regularities of my sensory experience of the chair are broken up into scattered and modality-specific information. The processing at the lowest levels is carried out by the lowest neocortical layers. Each small region in these layers has a receptive field that is very narrow and specific, such as firing only when a line sweeps across a tiny upper-right quadrant in the visual field. And of course, when the information comes into the brain it is processed by contralateral cortical areas, with the right lower cortical layers only responding to information streaming in from the left visual field, and vice-versa. As the modality specific and feature-oriented information flows up the cortical hierarchy, the receptive fields of the cells becomes broader, and more steady in the firing patterns. Whereas the lower cortical areas only respond to low-level details of the chair, the higher cortical areas stay active while in the presence of the chair under any experential condition. These higher cortical areas can thus be said to have created an invariant representation of the patterns and regularities which are specific to the chair. The brain is able to create these representations because the world actually is patterned and regular, and the brain is responding to this.

So what is the cash value of these invariant representations? To understand this, you have to understand how once the information flows to the “top” of the hierarchy (ending in the hippocampus, forming long-term memories), it flows back down to the bottom. Neuroanatomists have long known that 90% of the connections at the lower cortical layers are streaming in from the “top”, and not the “bottom”. In other words, there is a massive amount of feedback from the higher levels into the lower levels. Hawkins’ idea is that this feedback is the physical instantiation of the invariant representations aiding in prediction. Because my brain has stored a memory/representation of what the chair is “really” like abstracted from particular sensory presentations, I am able to predict where the chair will be before I even walk into the room. However, if I walked into the room and the chair was on the ceiling, I would be shocked, because I have nothing in my memory about my chair, or any chair, ever being on the ceiling. Except I might have a memory about people pulling pranks by nailing furniture to ceilings, so after some shock, I would “re-understand” my expectations about future perceptions of chairs, being less surprised next time I see my chair on the ceiling.

Hawkins think that it is this relation between having a good memory and the ability to predict the future based on that memory which is at the heart of intelligence. In the case of memories flowing down to the sensory cortices, the “prediction” is one that predicts what future patterns of sensory activity are like. For example, the brain learns Sensory Pattern A and creates a memory of this pattern throughout the cortical hierarchy. The most invariant representation in the hierarchy flows down to the lower sensory areas and fires the Pattern A again based on the memory-based prediction about when it will experience Pattern A again. If the memory-prediction was accurate, the incoming pattern will match Pattern A, and the memory will be confirmed and strengthened. If the pattern comes in is actually Pattern B, then the prediction will be incongruous with the incoming information. This will cause the new pattern to shoot up the hierarchy to form a new memory, which then feedbacks down to make predictions about future sensory experience. In the case of predictions flowing down into the motor cortices, the “predictions” are really motor commands. If I predict that if I walk into my office and turn right I will see my chair, and if the prediction is in the form of a motor commond, the prediction will actually make itself come true if the chair is where the brain predicted it will be. Predictive motor commands are confirmed when the prediction is accurate, and disconfirmed if inaccurate.

So, a noncomputational representation is based on the fact that the brain (particularly the neocortex) is organized in an hierarchical memory system based on neuronal patterns and regularities, which in turn are composed of synaptic mechanisms like long-term potentiation. According to Hawkins, it is the hierarchy from bottom to top and back which gives the brain its remarkable powers of intelligence. The intelligence of humans for Hawkins is really a product of having a very good memory and being able to anticipate and hence understand the future in incredibly complex ways. If you understand a situation, you will not be surprised because your memory is so accurate. If you do not understand it, you cannot predict what will happen next.

An interesting feature of Hawkins’ theory is that it predicts that the neocortex is fundamentally running a single algorithm: memory-prediction. So what gives the brain its adult modularity and specialization? It is the specific nature of the patterns and regularities of each specific sensory modality flowing into the brain. But the common currency of the brain is patterns of neuronal activity. Thus, every area of the cortex, could, in principle, “handle” any other neuronal pattern. Paul Bach-y-Rita’s research on sensory substitution is highly relevant here. Bach-y-Rita’s research has shown that the common currency of the perception is the detection and learning of sensory regularities. His research has, for example, allowed blind patients to “see” light by wiring a camera onto their tongues. This is to be expected if the neocortex is running a single type of algorithm. So what actually “wires” a cortical subregion is the type of information which streams in. Because auditory data and visual data always enter the brain from unique points, it is not surprising that specialized regions of the cortex “handle” this information. But the science shows that if any region is damaged, a certain amount of plasticity is capable of having other areas “take over” the input. This is especially true in childhood. What Micah Allen and I have tried to show in our  recent paper is that higher-order functions of humans are based on the kinds of information available for humans to “work with”, namely, social-linguistic information. So the key to human unique cognitive control is not having an evolutionary unique executive controller in the brain. Rather, the difference is in what kinds of information can be funneled into the executive controller. For humans, a huge amount of the data streaming in is social-linguistic. Our memory-prediction systems thus operate with more complexity and specialization because of the unique social-linguistic nature of the patterns which stream into the executive. So to answer Daniel Wegner’s question of “Who is the controller of controlled processes?“, the uniqueness of “voluntary” control is based on the higher-level invariant memories being social-linguistic in nature. The uniqueness of the information guarantees that the predictions, and thus behavior, of the human cognitive control system will be unique. So we are not different from chimps insofar as we have executive control. The difference lies in what kinds of information that control has to work with in terms of its memory and predictive capacities.


Filed under Philosophy, Psychology

A crude theory of perception: thoughts on affordances, information, and the explanatory role of representations

Perception is the reaction to meaningful information, inside or outside the body. The most basic information is information specific to affordances. An affordance is a part of reality which, in virtue of its objective structure, offers itself as affording the possibility of some reaction (usually fitness-enhancing, but not necessarily so). A reaction can be understood at multiple levels of complexity and mechanism. Sucrose, in virtue of its objective structure, affords the possibility of maintaining metabolic equilibrium to a bacteria cell. Water, in virtue of its objective structure, affords the possibility of stable ground for the water strider. Water, in virtue of its objective structure, does not afford the possibility of a stable ground for a human being unless it is frozen. An affordance then is, as J.J. Gibson said, both subjective and objective at the same time. Objective, because what something affords is directly related to its objective structure; subjective, because what something affords depends on how the organism reacts to it (e.g. human vs. water strider)

The objective structure of a proximal stimulus can only be considered informationally meaningful if that stimulus is structured so as to be specific to an affordance property. If a human is walking on the beach towards the ocean, the ocean will have the affordance property it has regardless of whether the human is there to perceive information specific to it. The “success” or meaningfulness of the human’s perception of the ocean is determined by whether the proximal stimulus contains information specific to that affordance property. A possible affordance property might be “getting you wet”, which is usually not useful, but can be extremely useful if you are suddenly caught on fire. Under normal viewing conditions, the objective structure of the ambient array of light in front of the human contains information specific to the ocean’s affordance properties in virtue of its reflective spectra off the water and through the airspace. But if the beach was shrouded in a very thick fog, the ambient optic array would stimulate the human’s senses, but the stimulus wouldn’t be meaningful because it only conveys useless information about the ocean, even though that information is potentially there for the taking if the fog was cleared. An extreme version of “meaningless stimulus without perception” is the Ganzfeld effect. On these grounds, we can recreate, without appealing to any kind of representational theory, the famous distinction between primary and secondary qualities i.e. the distinction between mere sensory transduction of meaningless stimuli and meaningful perception.

Note too how perception is most basically “looking ahead” to the future since the affordance property specifies the possibility of a future reaction. This can be seen in how higher animals can “scan” the environment for information specific to affordances, but restrain themselves from acting on that information until the moment is right. This requires inhibition of basic action schemas either learned or hardwired genetically as instinctual. In humans, the “range” of futural cognition is uniquely enhanced by our technology of symbols and linguistic metaphor. For instance, a human can look at a flat sheet of colored paper stuck to a refrigerator and meaningfully think about a wedding to attend one year in the future. A scientist can start a project and think about consequences ten years down the road. Humans can use metaphors like “down the road” because we have advanced spatial analogs which allow us to consciously link disparate bits of neural information specific to sensorimotor pathways into a more cohesive, narratological whole so as to assert “top-down” control by a globally distributed executive function sensitive to social-cultural information.

This is the function which enables humans to effortlessly “time travel” by inserting distant events into the present thought stream or simulating future scenarios through conscious imagination. We can study the book in our heads of what we have done and what we will do, rehearse speech acts for a future occasion, think in our heads what we should have said to that one person, and use external symbolic graphs to radically extend our cognitive powers. Reading and writing, for example, has utterly changed the cognitive powers of humans. Math, scientific methodology, and computer theory have also catapulted humans into the next level of technological sophistication. In the last few decades, we have seen how the rise of the personal computer, internet, and cellphone has radically changed how humans cope in this world. We are as Andy Clark said, natural born cyborgs. Born into a social-linguistic milieu rich in tradition and preinstalled with wonderful learning mechanisms that soak up useful information like sponges, newborn humans effortlessly adapt to the affordances of the most simple environmental elements (like the ground) to the most advanced (the affordance of a book, or a website).

So although representations are not necessary at the basic level of behavioral reaction shared by the unicellulars (bacteria reacting to sucrose by devouring it and using it metabolically), the addition of the central nervous system allows for the storage of affordance information into representational maps. A representational map is a distributed pattern of brain activity which allows for the storage of informational patterns which can be utilized independently of the stimulus event which first brought you into contact with that information. For example, when a bird is looking right at a food cache, it does not need its representational memory to be able to get at the food; it simply looks at the cache and then reacts by means of a motor program for getting at the food sparked by a recognition sequence. However, when the cache is not in sight and the bird is hungry, how does the bird get itself to the location of the cache? By means of a re-presentation of the cache’s spatial location which was originally stored in the brain’s memory upon first caching the food. By accessing stored memory-based information about a place even when not actually at that place, the bird is utilizing representations to boost the cognitive prowess of its nonrepresentational affordance-reaction programs. Representations are thus a form of brain-based cognitive enhancement which allow for the reaction to information which is stored within the brain itself, rather than just contained in the external proximal stimulus data. By developing the capacity to react to information stored within itself, the brain gains the capacity to organize reactions into more complicated sequence of steps, delaying and modifying reactions and allowing for the storage of information for later retrieval and the capacity to better predict events farther into the future (like the bird predicting food will be at its cache even though it is miles away).


Filed under Consciousness, Philosophy

A thought about memory and intention

Imagine that you are reading at 3pm and you suddenly remember that you have a pressing appointment at 4pm. The thought simply pops into your mind, and completely surprises you. What’s curious about this example is that the process of remembering was carried out without your intending it to happen. You did not do the remembering as your conscious mind was focusing on reading and understanding the text. It was the nonconscious mind which did the remembering, and then forced the results of its processing into the input-tray for consciousness. As you were reading, your nonconscious mind informed the conscious mind that it had just carried out a remembering process and that the result of the process is the content that you have a pressing appointment at 4pm. Your consciousness then completely reroutes itself and starts to focus on the process of getting to the appointment on time.

I like this example because it illustrates how there are many mental processes being carried out below the surface of consciousness. We usually think that remembering is something we consciously do. In a way this could be analytically true if we simply defined remembering as requiring our consciousness to be involved. But if we didn’t define remembering in this way, then we get the interesting idea that crucially important mental activities like remembering can happen below the surface of consciousness. Often, the results of these nonconscious processes are never even introduced into consciousness, and directly influence the behavior of the body without our conscious awareness. But sometimes the end-products of the nonconscious mental processes are loaded into consciousness and our conscious mind becomes aware of the content and is now able to start performing conscious operations on the content, which often involve the narratization of the content into a folk psychological story (often causal, involving reasons), which then has top-down effects on the entire behavior of the system e.g. the narratization of the nonconscious remembering process about the appointment enables the conscious rumination of behavior possibilities and allows for a shortcut in the decision making process through the higher-order linguistic categorization of the appointment in terms of simpler, more abstract categorical structures and schemas like “4:00pm”,”get documents”, “find keys”,”get in car”,”take highway”, “second floor”.

These abstract categorical structures allow for the construction of a mental narrative-schema through which consciousness acts and is able to influence the world. We become capable of consciously thinking thoughts like “Shoot! My appointment is at 4pm, I better get ready now and take the highway so I can make it on time.” Thoughts like these provide decision-making shortcuts and start a chain-reaction of reciprocal information exchange between the conscious and nonconscious systems. The tight functional loops between these systems give rise to complex and fluid human behaviors, such as scrambling to get ready and driving a car to make a pressing social appointment.

Leave a comment

Filed under Consciousness