Reconciling Direct Realism?

Sometimes I sit in class and think about the nature of perception and reality. That sounds cliche, but I often find myself wondering whether I am really perceiving the professor as they give a lecture. What am I looking at? Am I merely perceiving representations, or ideas, in my head, or am I really looking at the external world? How can I reconcile the fact that visual information from the environment must be filtered through my nervous system before it is perceived with the sensation that I am directly looking at the world. On one hand, the representational theory of perception makes sense because it seems like there is always going to be this “gap” between my perception and reality, mediated through my sensory organs. On the other hand, it makes evolutionary sense that animals would develop a direct perceptual system in order to save cognitive resources. “Perception is cheap, representation is expensive.”

So what am I looking at when I perceive the world? Ideas in my head or real objects? James Gibson proposed a solution that he thought solved these dualistic paradoxes when he came up with the concept of the ambient optic array. Light is bouncing all around the environment, reflecting information about surfaces and textures, eventually settling into invariant “visual angles”. It is the information in this ambient optic array that we perceive. We don’t perceive the world. We don’t perceive representations in our head, projected onto a Cartesian theater. We directly pickup information from the invariant visual angles of light in the ambient optic array.

This is a mind/body/world system. It embedded and embodied. It is confusing to talk about sense-data stimulating the retina, and the brain “perceiving” this data, as if it was projected onto our cortex and the mind just mysteriously “reads” the data. This leads to conceptual muddles such as mind/body dualism and the representational theory of perception. Gibson thought it made more sense to talk about a ecologically embedded perceptual system picking up information directly from the environment. The distinction between this information pickup and the representational theory of perception is subtle. The difference lies in the fact that with the representational theory there is this impossible divide between between “internal” world of the mind and the “external” physical world. Somehow information crosses this metaphysical gap. Gibson thought it was much more parsimonious and evolutionarily sound to talk about perception in terms of direct pickup by a holistic agent in the environment. The information in the ambient optic array is structurally isomorphic to the firings of the nervous system, which is embedded in a whole body, capable of moving about in the world. By utilizing this ecological approach to perception, Gibson was able to drop the conceptual muddle of a “mind” perceiving ideas driven by the sense organs, but rather, a Self perceiving the environment through invariant structures in the light reflected in the environment. This is why the phenomenology of perception always puts the environment “out there”, in the world, as opposed to “inside” the internal chambers of the mind.

:: :: :: :: :: :: :: :: :: ::

Thoughts on Representation

I just read an interesting paper by Eric Dietrich and Arthur B. Markman entitled “Discrete Thoughts: Why Cognition Must Use Discrete Representations.” In the paper, they first give a definition of general mental representations and make a distinction between discrete and continuous representations. Then they outline seven arguments for why they think discrete representations are necessary for any system that discriminates between two or more states.

Their definition of general mental representation is I think robust and conceptually useful. They define a representation to be any internal state that mediates or plays a mediating role between a system’s inputs and outputs in virtue of that state’s semantic content. They define semantic content in terms of information that is causally efficacious and in terms of what that information is used for. What this means is that representations have to be a part of mental causation. This approach reminds me a lot of Hofstadter’s work, which I have talked about here. Hofstadter emphasizes how mental representations, which mediate between the environmental stimulus and the behavioral output by virtue of being causal at the appropriate level of analysis. I take Dietrich and Markman to mean the same thing when they say that mental representations must be “psychologically real”. In Hofstadter’s terminology, the symbols must be active.

Next, the authors offers a definition of discrete representation. “A system has discrete representations if and only if it can discriminate its inputs.” If a system categorizes, then it has discrete representations. In contrast, a continuous representation would be more tightly bound to its correspondence with the environment. It would be coupled in such a way that it wouldn’t have the ability to make distinctions between its inputs. This is illustrated by the examples of a watt governor and a thermostat. In a watt governor, the arm angles of continuous representations of the speed of the fly wheel, and in contrast, a thermometer must make an on/off discrimination of the continuous representation of the varying bimetal strip. The discrete representation supervenes on the continuous representation.

Finally, the authors give seven arguments why cognition requires discrete representations. I won’t go over the arguments in detail, I will just list a brief summary taken from the text.

1. Cognitive systems must discriminate among states in the represented world.
2. Cognitive systems are able to access specific properties of representations.
3. Cognitive systems must be able to combine representations.
4. Cognitive systems must have some compositional structure.
5. There are strong functional role connections among concepts in cognitive systems.
6. Cognitive systems contain abstractions.
7. Cognitive systems require non-nomic representations.

In their conclusion the authors discuss the claim that it follows from the presence of discrete representations in the cognitive system that the best paradigm for cognitive science must be computationalism. They argue that any system that utilizes discrete representations must be finite and has deterministic transitions between states which can be constructed into an algorithm. Thus, the mind can be described as a computationalism system. I think this is a clever argument and places computationalism into its proper role as the dominant paradigm in cognitive science. Until conflicting evidence shows that when it comes to general mental phenomena there is a better methodological framework, we shouldn’t deny computationalism’s place as the best explanatory paradigm.

:: :: :: :: :: :: :: :: :: ::

Pragmatic and Epistemic Action

In a 1994 paper entitled “On Distinguishing Epistemic From Pragmatic Action” published in Cognitive Science, David Kirsh and Paul Maglio make an fascinating distinction between actions that change the world(pragmatic) and actions that change the nature of our mental tasks(epistemic). That sounds interesting you say, but how did the researchers go about showing such a distinction? By playing Tetris! Or rather, watching other people play Tetris.

I am sure almost all of you are familiar with the game Tetris so I won’t bother going into too much detail describing how one plays it. Basically, various geometric shapes called “zoids” fall one at a time and you have to arrange them in a row. One is allowed to rotate the zoids to best fit them into the virtual environment. The key idea behind using Tetris as their methodological domain was that Tetris is requires real-time, split-second interactive cognitive and perceptual performance. This allowed the researchers to tease out how people offload cognitive computation onto the external world in order to ease up the difficulty of the mental task at hand. This sort of external manipulation is called epistemic action and as I mentioned above, is distinguished from an action that merely seeks to change the nature of the world. Epistemic actions improve cognition by doing the following:

Reducing the memory involved in mental computation, that is, space
complexity;

Reducing the number of steps involved in mental computation, that is,
time complexity;

Reducing the probability of error of mental computation, that is,
unreliability.

Kirsh and Maglio found that advanced Tetris players perform a variety of epistemic actions to reduce their internal computational effort. In contrast to less-advanced players who rotate the zoids in their head, advanced players would physically rotate the zoids. This seemingly simple action changes the way the mind handles the computational task of rotating the zoids in the game and thus allows the player to manipulate the virtual world with more reliability and speed.

Such data suggests that standard theoretical frameworks in cognitive science might not be enough to explain the full extant to which humans utilize the external environment in ways that alter their mental landscape to improve cognitive performance. Instead of breaking up the world into a dualism of physical space and information-processing space, it might be more theoretically useful to have a more unified and fluid space where both pragmatic and epistemic actions can take place. This approach gives more credence to the idea that we are fundamentally in the world, embedded and embodied, with a perceptual and cognitive repertoire that doesn’t make hard and fast distinctions between the inner and outer realms.

Reference:

Kirsh, D., & Maglio, P (1994) On Distinguishing Epistemic from Pragmatic Action. Cognitive Science: A Multidisciplinary Journal, Vol. 18, No. 4: pages 513-549

:: :: :: :: :: :: :: :: :: ::

Controllers?In my brain?

I have been hearing a lot of buzz lately about Asim Roy’s newly published paper Connectionism, controllers, and a brain theory. What is all the hype about? Well, Roy claims to be offering a “new theory for the internal mechanisms of the brain.” Sounds exciting doesn’t it? What could Roy be proposing that is so revolutionary? Roy proposes that…wait for it… some parts of the brain control other parts! I suppose most of you aren’t exactly blown away, and I certainly wasn’t either.

Roy’s rhetoric is obviously pretty overblown, but let us give him the benefit of the doubt and move on to his actual arguments. He starts off by claiming that connectionist theory “postulates that the brain does not have controllers in it.” He quotes Rumelhart, Hinton, and McClelland as saying “there is no central executive overseeing the general flow of processing.” Seems pretty non-controversial to me though. They seem to be merely saying that there isn’t a homunculus in the system, controlling everything with a immutable Will. Philosophy 101. So what is Roy actually arguing against? A straw man? Sort of, but not quite. Roy argues that in connectionist models, there is a “controller” in the system that controls the learning algorithms and thus connectionist theories are essentially rooted in control-theoretic modeling.

But, as peter over at conscious entities mentioned, connectionist theorists haven’t exactly gotten to the point where they are proposing a general architectural model of how the brain works. It seems entirely plausible that when connectionist models get to that point of complexity, they wouldn’t hesitate to propose that some modules control other modules. Otherwise, I don’t see how one could get a theory that modeled high-level cognition. The way Roy structured his arguments, I don’t think anyone would argue against the idea that “there are parts of the brain that control other parts.” Furthermore, Roy himself undermines his claim for proposing a “new paradigm” when he says things as trivially obvious as:

It should be pointed out that this theory does not posit that there is a single executive controller in the brain. [b]Instead it envisions “multiple distributed controllers” controlling various subsystems or modules of the brain[/b]. The main argument of the paper that connectionists use “executive controllers” is only pointing out that their algorithms use a “central controller.” But different modules in the brain using connectionist-type learning can have their separate controllers.

I’d also like to point out that Roy was beaten by at least ten years on his emphasis of controllers. In his 1997 book Being There, Andy Clark says:

The idea here is that the brain should not be seen as primarily a locus of inner descriptions of external states of affairs; rather, it should be seen as a locus of inner structures that act as operators upon the world via their role in determining actions…This perspective leads to a rather profound shift in how we think about mind and cognition-a shift I characterize as the transition from models of representation as mirroring or encoding to models of representation as control

So contrary to Roy’s strong rhetoric, people sympathetic to connectionist theory such as Clark have been thinking about the mind and the brain in terms of action-oriented controllers for many years. In conclusion, I agree with Roy’s essential argument that there are parts of the brain that control other parts of the brain, but I don’t think this is a revolutionary of a paradigm as he thinks it is. Roy himself quotes from all over the neuroscience literature showing that it is riddled with control-theoretic terms, and by his own argument, he shows that connectionist theory is also already steeped in control theory. Surely, the connectionists themselves understand this. So who is Roy arguing against here?

:: :: :: :: :: :: :: :: :: ::

Perception and Action: Context-specificity

Research by Adolph, Eppler, and Gibson on infant responses to slopes has provided insight into the interplay between perception and action. In the research, infants with different forms of mobility(crawlers or walkers) were encouraged to ascend and descend slopes with different degrees of steepness. The walkers were wary of slopes of 20 degrees or more whereas the crawlers fearlessly attempt slopes of 20 degrees or more. As the crawlers increased in experience, they learned to avoid descending the steeper slopes. However, when crawlers first begin to walk this avoidance pattern seems to disappear and they again plunged down the steep slope without hesitation.

These results seem to indicate that the perceptual knowledge that infants gain about the world is action-specific. Infants do not learn about slopes in general but rather, they learn about slopes-and-crawling and then slopes-and-walking. Research along these lines paints a picture of perception as being for specific action-routines. Thus, theoretical frameworks in cognitive science should be geared towards “motocentricity” rather can “visuocentricity”. This re-conceptualization ties in with what James Gibson posited almost 30 years ago in his book The Ecological Approach to Visual Perception: that perception is tied in what we can do with perceptual information. Our perception of a chair is intimately coupled with the fact that chairs are for sitting. Gibson claimed that these perceptual affordances for action are directly perceived in the environment around us. So when an infant looks at a slope, he perceives that the slope affords for falling. The only trouble is putting such information about the environment into use using the context-specific motor schemas available to the infant.

References:

Karen E. Adolph, Marion A. Eppler, Eleanor J. Gibson (1993) Crawling versus Walking Infants’ Perception of Affordances for Locomotion over Sloping Surfaces Child Development, Vol. 64, No. 4 (Aug., 1993), pp. 1158-1174

Tool Use: Part of the Body Schema

The Italian Neuroscience Mafia is at it again: Giacomo Rizzolatti and cohorts recorded the brain activity of macaque monkeys in the F5 and F1 areas while they were grasping with their hands and then when they were grasping with a pair of pliers. Remarkably, the same neurons fired in the same order when they were grasping with their hands as when they were grasping with the tool. Furthermore, the same neurons also fired in the same order when the monkeys used “reverse pliers” that required closing and then opening the hand in order to grasp the food. Because of this, the researchers concluded that “the capacity to use tools is based on an inherently goal-centered functional organization of primate cortical motor areas.”

Their evidence clearly shows that there are cortical neurons in the F5 and F1 area that code for for the goal of motor acts, instead of the motor act itself. These neurons are then connected to neurons that more specifically code for the motor act of opening and closing. Furthermore, the researchers show evidence that amidst the goal-directed neurons in the F5 area, mirror neurons are also involved, which code for goal-directed actions during the observation and execution of an act and are rich in the F5 area.

Cognition and Emotion

Jake Young over at Pure Pedantry has an interesting article on cognition and emotion. He summarizes a review paper by Luiz Pessosa that argues that cognition and emotion are not separate. He goes through a number of different arguments in favor of such a thesis, which I will not list here, but needless to say, seem quite compelling. However, since Antonio Damasio’s work in the 90s, I don’t think this is a very radical hypothesis( Damasio published Descarte’s Error in 1994). Fascinating article nevertheless.

Thinking about Libet, pt 2

In my last post, I briefly discussed the most pertinent results from Benjamin Libet’s 1982 experiment and some of the implications. In this post, I would like to put on my speculative hat and talk about an alternative to the dichotomy I set up in the previous post I linked.

If you don’t recall, the dichotomy was set up between what appears to happen when we hear a noise behind us and what Libet thought was going on. What seems to be going on is that we consciously hear a sound and then turn around, but Libet proposed that we unconsciously hear it first, turn around, and then our brain performs a “backwards subjective referral” of the event to make it seem like the first scenario is happening. In my last post I said this was a false dichotomy and now I will speculate on what I think is really going on. Bare with me.

Imagine that a human body is sitting in a classroom and attending to the lecturer. All of a sudden this attentive human becomes aware of a door opening behind his back and cocks his head around accordingly in order to see what just interrupted the lecture. All of this activity, including the awareness of the noise of the door opening and the innervation of the appropriate neck and back muscles, all happened within the more-or-less continuous, 3d perceptual space that we all are familiar with.

So, while there are hundreds of potential things to be attentive of in any normal classroom(including the awareness of your own body sitting down), the available cognitive processing power responsible for “shifting the attentional spotlight” allocated almost the entirety of its capacity to the perception of the door opening, probably because of the neural contrast/distinction of such a sudden noise but also because this human body knows from past experience that usually the only things that open doors are other humans and it is this implicit social knowledge imbedded into action-schemas that makes it hard not to notice sudden changes in the social environment.

This contextual social knowledge makes it obvious why more attentional capacity would be allocated to the perception of the door opening rather than say the perception of his body twisting around or the perception of practically anything else in the room, even though the overall body-system never stopped being aware of these things, it is just that the percentage of attentional capacity devoted to the door opening made that particular activity more vivid that anything else upon episodic recollection. With the situation set up in this way, let us try go back to explaining Libet’s half-second delay.

Under my conceptualization, the reason why the neural isomorphic representation of the door opening seems to “echo”(thanks Dennett) around the brain for an “extra” half second past the “necessary” evoked motor potentials is because the brain is essentially “telling a story to itself” about the event. The functionality of this generated “after-the-fact” story comes from the fact that the door-opening-event is now easily fed into a variety of different cognitive systems thanks to the considerably long(in brain terms) half-second of processing necessary to turn such “noisy” sensory data into higher-level “conceptual” representations that implicitly include such linguistic conceptual relativities as self/other, internal/external, etc. important for metaphor-based story telling.

Furthermore, I speculate that evolutionarily speaking, the most advantageous way of putting these high-level representations of low-level sensory-action-data to use would be through an advanced memory/prediction/empathy system. This interrelated triad would be of great advantage in a social atmosphere. Perhaps I will elaborate on this triad a later post, but for now I am done wildly speculating from my armchair.

:: :: :: :: :: :: :: :: :: ::

Thinking about Libet

In 1982, Benjamin Libet carried out a remarkable study on consciousness that is still being debated by contemporary philosophers and scientists. In today’s post I would like to briefly highlight the results and spell out some implications. Here are the most pertinent results as far as I can see them:

• In order to “consciously experience” a sensation, it must apparently bounce around the somatosensory cortex, or some other “high-level” area of the cortex for about half a second, probably isolated to the frontal areas(Libet, 1982)
• “A touch on the skin that the subjects would otherwise have reported feeling was retroactively masked up to half a second later by a stimulation to the cortex”(Blackmore, 2004)

Okay, so how can phenomenological consciousness “drag” half of a second behind the real world when clearly we are able to react much faster than that? The most obvious idea is to say that consciousness has no causal power, it is merely a resultant and not a force (in James’ terms). However, this is at odds with the “hard problem” of consciousness because if our “unconscious” does all the important work, such as reacting to dangerous stimuli in split-second situations, there would have been no evolutionary pressure for phenomenal consciousness to tag along and “dangle” half a second behind the real important things going on in the world, such as a stepping on a snake or braking for a red light.

I believe that I can sketch out a framework that can reasonably explain how consciousness could happen “after the fact”, yet still have enough function that it could easily have evolved in the way that it did given the close-knit social structures of our early hominid ancestors.

Let us look at Blackmore’s example of turning around to look who just opened a door while you are sitting in a classroom. This is what seems to happen:(from blackmore)

Scenario 1

1. Consciously hear sound
2. Turn around to look

According to Libet, it should be more like this:

Scenario 2

1. Unconsciously “hear” sound
2. Turn around to look
3. Backwards subjective referral of consciousness to make it seem like Scenario 1 is what actually happened

So how do we extricate ourselves from this mess? I think the first step is to recognize that you are setting up a false dichotomy of sorts by trying to directly reconcile scenarios 1 and 2 as the only two options. Furthermore, we should follow Dennett’s advice and use extreme conceptual caution when using the terms “conscious” and “unconscious”, because the nature of our language necessarily forces an implicit acceptance of the Cartesian Theater whenever we use the language of conscious/unconscious, and it is this intuitive dichotomy that makes it impossible to solve these kinds of philosophical problems using ordinary conceptual frameworks.

However, if we use the framework of enactive perception and attentional theories of consciousness, we will get a better understanding of why trying to decide between either Scenario 1 or 2 will only result in frustration and headaches. In my next post I will discuss an alternative way of looking at this problem. Stay tuned!

References
Libet, B. 1982 Brain Stimulation in the study of neuronal functions for conscious sensory experiences. Human Neurobiology 1, 235-42

Blackmore, S. 2004 Consciousness: An Introduction

:: :: :: :: :: :: :: :: :: ::

Connectomes: Mapping the Circuitry of the Brain

Wired is running an interesting article on the new research field called “connectomics”.

Basically, Harvard researchers built a machine that slices thin layers of brain tissue and then takes high-resolution pictures with an electron microscope, hoping to form detailed diagrams of the actual circuitry of the brain i.e. “connectomes”. This process stands to generate a huge amount of data on how the brain is actually wired. Exciting times!