The Myth of Sensory Immediacy – Why Berkeley Was Wrong



Theories are often supported by unquestioned assumptions. One of the oldest unquestioned assumptions in philosophy concerns the nature of perceptual stimuli, namely, that sense-data are immediately perceived and that sensation consists in the sequential processing of  “motions” immediately pressed upon the eye. Originating in Plato’s Theaetetus, this idea reached maturation in Descartes and was then taken up by Locke under the concept of primary and secondary qualities. According to these thinkers, sensible ideas (sense-data) are immediate impressions upon the eye which communicate the motion of light particles through the nerve conduits so that they may be processed and deposited onto the internal mental theater for subjective viewing. It is important to note that the conduit metaphor assumes that perception can only perceive the atomic sequence of impacts upon the eye; any nonimmediate mental perception is the result of “inference”. In other words, there is a “bottleneck” of immediacy that cannot be overcome without higher-order cognitive acts. Moreover, sense-data are private in that the primary qualities “appear” differently for different people in accordance with their individuality.  The logic of this conduit metaphor was radicalized by Berkelely in his extreme sense-data empiricism. In the First Dialogue, he writes:

Philonous. It seems then, that by sensible things you mean those only which can be perceived immediately by sense.

Hylas. Right.

Phil. Doth it follow from this, that though I see one part of the sky red, and another blue, and that my reason doth thence evidently conclude that there must be some cause of that diversity of colours, yet that cause cannot be said to be a sensible thing, or perceived by the sense of seeing?

Phil. This point then is agreed between, that sensible things are those only which are immediately perceived by sense. You will further inform me, whether we immediately perceive by sight any thing beside light, and colours, and figures: or by hearing any thing but sounds: by the palate, any thing besides tastes: by the smell, besides odours: or by the touch, more than tangible qualities.

Phil. Sensible things therefore are nothing else but so many sensible qualities [immediately perceived], or combinations of sensible qualities.

Our discourse proceeded altogether concerning sensible things, which you defined to be the things we immediately perceive by our senses. Whatever other qualities therefore you speak of, as distinct from these, I know nothing of them, neither do they at all belong to the point in dispute.

As a good empiricist, Berkeley radicalized the logic of the conduit metaphor for visual perception. If there is nothing “in” the mind-container other than impressions formed by the immediate perception of sensible things, and if sensible things cannot exist without a perceiving mind, then we have no rational recourse for knowing anything about an extrasensory material world composed of primary qualities not dependent on the mind. Thus, the ultimate substance of reality is Mind or spirit, for we can never escape the confines of our mental container.

The essence of the conduit/container metaphor consists in understanding the perceptual process in terms of an immediate communication of sense-data across the nerve-conduits into the “container” or “theater” of the mind. Accordingly, the very nature of the perceptual stimulus ensures that the mind can never rationally proceed beyond the causal immediacy of private sense-data. Because the perceptual stimulus is assumed to consist of immediate impressions upon the retina, any perception of motion must be inferred from the two-dimensional patterns of light, which are then transduced into sense-data. For Berkelely, the radicalization of this logic leads to the proposal that we have no rational recourse for getting “outside” of the internal theater of sense-data, for what we “perceive” is not the primary real, but rather, the secondary quality of “how it appears” subjectively.

Upon reflection however, we can see that the flatness or “immediacy” of sensory input is usually presupposed only after thinking about perception in terms of a frozen snapshot of reality. Ecological information, however, does not exist exclusively in an instantaneous slice of time for, as J.J. Gibson points out, “Animals and men [directly] perceive motions, events, episodes, and whole sequences” (1966, p. 276). Thus, the information we perceive in the environment has both successive and adjacent order. For Gibson, it is a mistake to think of persisting patterns as being a separate stimulus; biologically speaking, “Transformations of pattern are just stimulating as patterns are…motion is immediately detected by animals, not secondarily deduced from change of position”( ibid., p. 40). Accordingly, the brain is not in the business of continuously constructing a mind bogglingly detailed phenomenal model from spots of sensations differing in brightness and color. If this were true, Gibson sardonically notes that “the fact of perception [would be] almost miraculous”. Instead, Gibson theorizes that the nervous system directly “picks up” or behaviorally “resonates” to the ecological information available in the environment, particularly in respect to changes in the layout of surfaces, changes in the color and texture of surfaces, and changes of existence of surfaces. As Mark Rowlands nicely puts it,

information is simply optical structure-together with the deformation in this structure generated in a nomothetic way from the environmental layout and events. This optical structure is not similar in any way to the environment, but it is specific to it. That is, optical structure is nominally dependent upon environmental structure. Because the structure in the optic array is specific to its environmental sources, an observer whose perceptual system detects some optical structure is therefore aware of what this specifies. Thus, the perceiver is aware of the environment not the array. Therefore, once we describe the input for perception in terms of a structured optic array, we are committed to the idea that there is enough information directly available in the organism’s visual input to give that organism useful knowledge about the nature of its environment. Postulation of additional information processing would, to this extent, be superfluous. (1995, p. 9)

The logic of Gibsonian information processing goes counter to the thesis of radical sensory immediacy. Under the Gibsonian framework, perception is not constituted by the processing of sense-data through the bottleneck of retinal immediacy. Instead, the perceptual system is capable of a first-order perception of whole sequences in the environment. By proposing that the transformations of pattern within the ambient optic array contain both sequential and adjacent order, the notion that perception consists of immediate detection and transduction of motion across the two-dimensional retina loses its status as an unquestioned assumption. Accordingly, the Gibsonian framework argues that perception is grounded by ecological information, not sensory immediacy. And because ecological information is temporally extended, the classic model of immediacy is overcome by supposing that animals are capable of directly resonating to such information. As it turns out then, we are not trapped within the theater of our minds; access to reality is quite pedestrian.



Filed under Philosophy, Psychology, Uncategorized

18 responses to “The Myth of Sensory Immediacy – Why Berkeley Was Wrong

  1. Victor Panzica

    When I reach out with my hand and grab the handle of the coffee cup on my desk, my hand is like my retina and the information travels through my hand and up my spinal cord to the base of my brain. The feel of the handle is a gross perception by my hand but if I want to be analytic, I can use my fingers to feel the edges and curves of the handle. Likewise my overall retinal function also makes gross perceptions of the environment, however different fields of the retina and subsequent brain functions detect analytic qualities of perceived objects.

    Dr David Hubel has an interesting web site on his vision work

  2. Gary Williams

    It could also be said that tactile sensation is not “bottlenecked” given that if I am holding a ball in my hand, I don’t somehow feel 5 different parts of the ball but rather, feel the whole ball at once. In the same way, visual perception is able to take the “whole” scene at once and perceive events and higher-order sequences. This must be some sort of top-down driven process wherein our sensation is integrated with a linguistically structured categorial intuition that provides “labels” or “tags” upon those higher-order events/objects such that the world becomes cut at its joints in terms social background knowledge.

  3. Victor Panzica

    “…Locke said that the mind is forever trapped behind a veil of Ideas.” But to me it appears that the mind is trapped behind a veil of rules.

    My conception of seeing is that our vision cortex is not a two dimensional but three dimensional construction, so the mechanism of our brain posits the outside world as an object would be frozen inside of a block of ice.

    I don’t know if you’ve read Chalmers PhilPapers discussion thread on the Explanatory Gap

  4. That is a wonderful picture, did you make it?

  5. Eric,

    I found the picture several years ago on wikipedia under the heading, “Cartesian theater”.

  6. Mike

    I was surprised that you didn’t mention Hegel’s argument against “sense-certainty” (one of the early chapters of the phenomenology) where he too argues against empiricism (although in a very different way and for a different purpose.

  7. Gary Williams


    Hegel is, unfortunately, one of my great blind spots. I just don’t know enough about him to work him into my thinking, although I am aware of the basic gist of his ideas and how they influenced philosophy.

  8. N

    Hi Gary,

    “One of the oldest unquestioned assumptions in philosophy concerns the nature of perceptual stimuli, namely, that sense-data are immediately perceived and that sensation consists in the sequential processing of ”motions” immediately pressed upon the eye…”

    While I appreciate the thrust of your post, here, I do take issue with your description of direct realism as something that overturns “one of the oldest unquestioned assumptions in philosophy”. It is true that certain British empiricists adopted this model, and that the notion of an inner “Cartesian theatre” has remained a strong idea in much modern philosophy. But Thomas Reid immediately rebuked both the empiricists and Descartes, and Kant followed by (productively) adopting entirely different assumptions about the nature of experience.

    The allegedly “unquestioned” Lockean model has in fact been rigorously attacked almost since its very inception. Let’s hope Gibson has driven the final nail into its coffin.

  9. Gary Williams


    I should say, rather, that such assumptions are usually unquestioned in Western philosophy. As you point out, people like Thomas Reid put forward a style of direct realism in contraposition to Descartes. And I actually think Aristotle was an early progenitor of direct realism. This is why I think Heidegger was so enamored of Aristotle’s account of perception and uncovering.

    But I should add that sensation-based theories of perception dominate the contemporary cognitive science circles. There are a few Gibsonians here and there (Rodney Brooks being the biggest example), but from what I can tell cognitivist representationalism based on the assumptions of sense-data theory is still taught in most undergraduate “sensation and perception” psychology courses.

  10. Gibsonian psychologist checking in…

    N, Gary is entirely correct about how modern cognitive psychology treats perception. The only new bit comes from the increasingly sophisticated attempts to provide internal representation with the tools it would need if sense-data was all you had (Bayesian this and that, etc). Gibson’s brilliance was that instead of trying to solve the wrong problem cleverly, he stepped back and saw it was the wrong problem; this move has yet to be properly appreciated, in part I think because it meant Gibson started talking about perception in such a radically different way.

  11. Pingback: Playtonic Dialogues Is A Semifinalist/ Top Vote-Getter For The 3 Quarks Daily 2010 Prize In Philosophy! « Playtonic Dialogues

  12. I’d be more sympathetic to these Gibsonian theories of perception if they could better deal with things like dreams, phantom limbs and other hallucinations, binocular rivalry and other bistable percepts, perception-specific effects of localized brain damage (e.g., achromatopsia), conscious but paralyzed humans, etc..

    Such facts that spawned a representational theory of mind are very tough for the neobehaviorists/Gibsonians to deal with, no matter how many affordances you pack into the equation.

    Also, you seem to be conflating an antiquated sense-datum theory (that was pretty much killed by Sellars) with more modern representationalist theories that don’t talk of ‘sensations’ much at all, especially not in the theoretically suspect way that the Logical Positivists did.

    I am a sensory neuroscientist, and neither I nor any of my colleagues ever talk about ‘sense data’. Represenatation-talk, though, that is everywhere. But representational theories of perception are not committed to ‘immediacy’ in any straightforward way, so I’m not convinced your post attacks anything real unless you really mean to attack representationalism generally.

    In which case, you have a tough road ahead as representationalism handles data like the list I gave at the start of my post quite naturally, unlike the Gibsonian and neobehaviorist theories in which ‘skilled interactions’ are all there is to perception.

    • Gary Williams


      I don’t think Gibsonian psychology is incapable of dealing with those phenomena, but Gibson would say that studying perception under those conditions doesn’t capture the way in which animals naturally stay in touch with the environment. All those conditions presuppose that the animal is in some way cut off from the information in the environment and thus in need of constructing a stable percept out of unstable or ambiguous sensory information (sense-data).

      Gibson never denied that hallucinations and rival-percepts are possible. He just didn’t think they were cases of perception proper. For him, perception is a technical term which means the detection of meaningful stimulus information. Accordingly, dreaming, hallucinations, phantom limbs, rivalry, etc. are not examples of perception, but rather, “mere stimulation”. Gibsonians are always careful to separate stimulation of sensory nerves from meaningful perception. The phenomena of “mere sensory stimulation” are examples of what happens when the perceptual process is unable to effectively detect meaningful stimulus information because either not enough information was available in the stimuli (ambiguity) or the process of detection is itself somehow physiological corrupt (localized brain damage).

      Gibson would say that the study of perception under such unnatural conditions is perfectly fine if you want to study anatomy, physiology, and neuroscience. But such studies are not going to provide a foundation for developing a proper psychology of how animals naturally stay in touch with their surroundings under normal evolutionary conditions.

      I don’t think these kinds of phenomena are “tough” for Gibsonians to handle. I could elaborate further how on the study of perception under artificial conditions often leads to the utilization of anatomical facts to explain psychological facts, but I won’t press the point here because it would then require a discussion of why the SR-theory is still taken seriously by input-output computational representationalists why this is problematic for developing a scientific account of psychological meaning.

      Also, just because neuroscientists do not use the word “sense-data”, that does not mean that they are not implicitly using the concept under a different name. Sense-data is for the most part synonymous with the term “proximal stimulus” or “raw input”. As far as I am aware, most neuroscientists are committed to telling a story of perception wherein the task is to construct an internal representation of the distal stimulus from the inadequate and ambiguous information available in the proximal stimulus. Another way to say it is that the brain is in the business of constructing mental models of the environment from the information contained in the sense-data.

      Sense-data is basically the “raw input” in the classic computational input-process-output model. But Gibsonian psychology has a problem with the very notion of “raw input”. And it gets the representationalist no where to critique Gibson by saying ecological psychology has no account of how “processing” works. This is not true. When Gibson rejected information processing he did not reject the idea that the brain is in the business of processing information. Far from it. Instead, Gibson rejected the notion that the information being processed was “physiological data” like wavelengths and photons. He thought this view was mistakenly starting from physiological facts and then moving to psychological facts. But changing the way in which we understand the “input” or “stimulus”, Gibsonian psychology becomes a new model of information processing rather than a rejection of information processing altogether. But we must do away with talk about “raw data” and instead start talking about the detection of information that specifies affordances.

      And I would be careful about constructing too much of a strawman of Gibson. His theory goes above and beyond “skilled interaction”. Skilled interaction is what Gibsonians call “performatory” behavior. But there is also “exploratory” behavior wherein the animal is “scanning” for information and detecting affordances without utilizing them for action. So Gibsonian psychology isn’t about pure “action”. There is also exploratoration and discovery. Action and perception can come apart, but most critics of Gibson miss this point and accuse him of developing a theory of just “skilled interaction” instead of perception itself.

  13. To deal with things like hallucinations, bistable percepts, dreaming, and phantom limbs you would say those aren’t cases of perception? Then you have a different explanatory target than pretty much everyone studying perception and should use a different word.

    There is no evidence that in illusions (e.g., bistable perception) that the brain somehow knows to switch its mode of operation to a nonGibsonian style, and then in nonillusory perception, it switches to a mode in which the Gibsonian theoretical apparatus will become applicable. You’d need an argument that illusions don’t reveal something real about the underlying architecture of normal perception.

    The common rejoinder, that such things come up in conditions that are somehow “unnatural” or “artificial” doesn’t help. That’s what biologists do. We tweak variables to see how the system functions. We look at people with brain damage, we change sodium concentrations in neuronal environments, we add drugs, we show the system strange stimuli to see how it responds at a perceptual level.

    Gibson did help highlight the power of the environment and behavior to some of the work traditionally ascribed to factors inside the brain, but ultimately the representationalist theories are better fit by the neuronal data, behavioral data, psychological data. Gibson’s success is suggesting that we trim down the number of representational states required to explain behavior, and pointing out some sources of information available that people hadn’t considered enough before (e.g., optic flow).

    Note also I don’t think that Gibson thinks of perception as mere skilled interaction (they are allies, but not identical), but there is a large contingent of people that do believe that. They are wrong, but they do exist.

    Finally, ‘sense data’ is a specialized term within philosophy, pushed especially hard within the Logical Positivist tradition, that was largely killed by Sellars in his essay Empiricism and the Philosophy of Mind. Modern theories of perception do not advocate such a theory. If you are equating representationalism with sense datum theory, that is a mistake. There is a good entry on this technical term here. Nobody in neuroscience advocates, either in word or in spirit, such an outmoded doctrine of perception. It was never a scientific theory, but pat of a strange foundationalist philosophical theory that aimed to ground all knowledge in sensory experience. That is clearly not the neuroscientific approach. The neuroscientific approach is the nonfoundationalist approach of the ‘Second Philosopher’ as described by Penelope Maddy.

    At any rate, your entry is very well written, even if all of the conclusions are wrong. 🙂

  14. In case it becomes lost in the scuffle, my considered view on Gibson is that he made two very important contributions to perceptual psychology. First, he pointed some sources of information available from environment/organism interactions that people hadn’t considered enough before. E.g., optic flow, haptic interactions with physical objects. These are now standard considerations in the study of perception.

    Second, based on the first types of considerations, he responded to runaway speculative models in cognitive psychology by pointing out that we could trim down the number and complexity of representational states required to explain behavior. This is also now a standard consideration in perceptual psychology.

    Ultimately, it is an empirical question how much detail the brain builds into its representations of the world. The data suggests it is more rich than it has to be. However, it is also less rich than those who think that our brain constructs a full-blown detailed 3-d model of everything in our vicinity.

    For some reason there is an almost mystical respect for Gibson among his followers, the kind of overblown respect as you’d find among many Wittgensteinians. While I frankly don’t understand it, I do respect and admire his positive contributions to the study of perception. It’s just that the worthy bits have largely already been absorbed into contemporary studies of perception, and now we have philosophers pushing other parts that have not been so absorbed because they do not help explain the data (indeed, they do the opposite).

    That said, it is good that people continue to mine his work for further helpful tidbits. Perhaps there is an undiscovered affordance for perceptual science buried in there, one that has not yet been expressed clearly enough for the practitioners to understand and absorb into their work.

    • Gary Williams


      I really wish it were true that contemporary neuroscience has assimilated and appropriated all of Gibson’s criticisms and insights. You say that perceptual psychologists have learned their lesson about setting up the problem of perception such that the complexity of the tasks is reduced by taking into account environment considerations, but I’m just not sure. I think it comes down to some pretty fundamental assumptions that just aren’t shared between the two camps.

      Now, don’t get me wrong. I think the two camps should try and learn from each other, appropriate their respective arguments, and then from this synthesis a new perceptual foundation can emerge. But as it stands right now, computational neuroscience is incompatible with many of the claims of ecological psychology and I don’t think we can simply “pick and choose”. For example, an ecological approach works by emphasizing reciprocal causality whereas (from what I understand) computational neuroscience is still wedded to a view of the perceptual stimulus as an efficient cause. For ecological psychology, the stimulus does not *cause* behavior. It is an opportunity for behavior.

      I mean, the two camps are using two fundamentally different metaphors. The metaphor of orthodox theory is “construction” i.e. the construction of a representation. The metaphor of ecological theory is “regulation” i.e. the regulation of a perturbation upon a dynamic, autonomous system. Construction vs. regulation.

      Now, we could of course develop a regulation-model of cognition that takes insights of representational constructivism and applies them to high-level functions but maintains a dynamic systems core for the low-level regulation of informational perturbations. This is the approach I favor.

      There is a lot of the research experiments that orthodox perceptual theorists use to demonstrate the existence of “construction”. I do not deny these results. I just think they tell us little about basic perception. Rather I think they tell us a lot about higher-order perception and top-down influences. In this respect, I agree 100% with orthodox theory in saying that past experience shapes our perceptual experience and in this sense perception is infused with representations/expectations/inferences, etc. But I see a huge difference between the low-level memory effects of the perceptual learning of affordances (retrospectivity) and the autobiographical memory effects of higher-order human perception. I’m not saying that all construction-artifacts are the result of these higher-order processes, but often I think the results of experiments on visual illusions merely emphasize the way in which reflective, introspective gazing on perceptual experience triggers unique phenomena not associated with the basic detection and utilization of an affordance (something an earthworm can do).

      I don’t have a mystical respect for Gibson, although I do respect him greatly. I think he set the stage for ecological psychology but he was by no means the last word. There is still A LOT of work to do. After all, psychology is still a young field. But I think Gibson laid the foundation for what a psychology of low-level perception/behavior/awareness should look like. But he left a lot of room open for what a psychology of both basic perception and higher-order cognition would look like. Computational neuroscientists want to explain the low-level stuff in terms of the high-level stuff whereas ecological psychology wants to do the opposite. But it is a mistake to say that a true science of mind shouldn’t attempt to do both simultaneously at multiple levels of analysis.

  15. Nick Layson

    hey Eric i love you and im a man ❤

  16. Nick Layson

    i love you too gary ❤

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s