Representationalism and the Scene-Immediacy of Visual Experience:
A Journey to the Fringe and Back.
Abstract: Both visual experience and conscious thought represent external objects, but in visual experience these objects seem present before the mind and available for direct access in a way that they don’t in conscious thought. In this paper, I introduce a couple of challenges that this “Scene-Immediacy” of visual experience raises for traditional versions of Representationalism. I then identify a resource to which Representationalists can appeal in addressing these challenges: the low-detail fringe of visual experience. I argue that low-detail contents within visual experience provide the mind with a rich access to additional high-detail information, an access that is not found in conscious thought. This access, in turn, speaks to the challenges raised by the Scene-Immediacy of visual experience.
Representationalist accounts of perceptual consciousness identify the phenomenal character of a perceptual experience with the representational content carried by that experience.1 One general concern facing such accounts is that conscious thoughts carry representational content while not having the phenomenal character of perceptual experiences. As a result, Representationalists inherit the burden of explaining why some representational contents (those carried by experience) are phenomenal while others (those carried by conscious thought) are not.2 Although it’s controversial, I am going to assume that thoughts have an introspectible character—to borrow Nagel’s (1974) phrase, I assume that “there is something that it’s like” to have conscious thoughts.3 Given this assumption, the general challenge facing Representationalists is to explain the enormous difference between the introspectible characters of conscious thought and perceptual experience.
Following popular custom, I will focus on a particular kind of perceptual experience: visual experience. One important aspect of the difference between visual experience and conscious thought is that visual experience seems to present the objects/features that it represents before the mind (and give us access to them) in a way that conscious thought does not. In this paper, I focus on this “Scene-Immediacy” of visual experience. After a preliminary discussion of Scene-Immediacy (section 2), I explain how the phenomenon presents a basic problem for all versions of Representationalism and then zero in on two specific challenges that it presents to the traditional version of Representationalism that I favor (section 3). I then sketch how an appeal to the fringe of visual experience can help my preferred version of Representationalism respond to these challenges (section 4). This sketch is subsequently filled out and defended in sections 5-7: section 5 describes the fringe of visual experience in more detail. Section 6 uses this description of the fringe to fill out my responses to the challenges presented by Scene-Immediacy. Section 7 considers two objections to my account of Scene-Immediacy.
2. The Scene-Immediacy of Visual Experience
There are several ways in which a visual experience of an object introspectively differs from a conscious thought about that object. The way that I want to focus on is the topic of the following passage from John Searle.
If, for example, I see a yellow station wagon in front of me, the experience I have is directly of the object. It doesn't just ‘represent’ the object, it provides direct access to it. The experience has a kind of directness, immediacy and involuntariness which is not shared by a belief I might have about the object in its absence…The visual experience I will say does not just represent the state of affairs perceived; rather, when satisfied, it gives direct access to it, and in that sense it is a presentation of that state of affairs. (1983, p. 46)
Don’t let Searle’s use of the words “directness” and “immediacy” mislead you. The difference that he is describing is not about whether the visual experience or the thought makes us indirectly aware of the yellow station wagon in virtue of making us directly aware of something else.4 Both of these conscious states provide us with a “direct” awareness of the wagon in that sense of the word.
So what difference is Searle describing? Although both the visual experience and the conscious thought make you aware of the station wagon in an unmediated manner, in the former the station wagon seems present before the mind in a way that it doesn’t in the latter. As Martin (2002, p. 388) puts it, “sensory states involve a certain immediacy or apparent presence of an object which is simply not required in cases of pure thought.”
Others have pointed to the same aspect of the phenomenal character of visual experience. In discussing this phenomenon, which he calls “presence”, J.J. Valberg (1992) states:
Presence (in experience) connotes a kind of direct or immediate availability. An object which is present is right there, available to us. (p. 19, his emphasis)
Similarly, in his description the phenomenal character of visual experience, Scott Sturgeon (2000) says:
Its phenomenology will be as if a scene is made manifest to you. This is the most striking aspect of visual experience…Visual phenomenology makes it for a subject as if a scene is simply presented. Veridical perception, illusion and hallucination seem to place objects and their features directly before the mind. (p. 9)
I will follow the lead of Sturgeon (2000) and refer to this aspect of visual phenomenal character as “Scene-Immediacy”.
3. Scene-Immediacy and Representationalism
Scene-Immediacy presents a general challenge to all versions of Representationalism. Before introducing this challenge, however, a brief discussion of Representationalism is in order.
As a thesis about perceptual experience, Representationalism has evolved quite a bit over the last several decades. Following Chalmers (2004), we can unite all versions of Representationalism in virtue of their identification of a phenomenal property with the property of having a particular representational content in a certain manner. Early versions of Representationalism—which I’ll call “Good Old Fashioned Representationalism” (or “GOFR”, for short)—were reductive in that they analyzed the notion of having a particular representational content in a certain manner entirely in physical/functional terms.5 Michael Tye’s (1995, 2000) PANIC theory, for example, identifies a phenomenal feature with the property of having a representational content that is poised, abstract, and non-conceptual, where all these notions are analyzed in way that does not appeal to any phenomenal notions.6 In addition to being reductivist, early versions of Representationalism were also united in offering a similar account of what visual experience represents: namely, objects/properties in the surrounding environment (objects/properties that could also be represented in thought). This account of the content of visual experience was meant to accommodate the so-called “transparency” of visual experience; it was meant to accommodate the fact that the phenomenal features of visual experience seem like features of objects in the surrounding environment.7
Today, however, there is more under the Representationalism umbrella than just GOFR. To start with, there has been a movement towards non-reductive accounts of Representationalism.8 Advocates of this movement continue to maintain that a phenomenal property can be identified with the property of having a particular representational content in a certain manner, but they claim that the latter notion cannot be fully analyzed independently of phenomenal notions. They might, for instance, claim that the content of visual experience involves the attribution of phenomenal properties to objects and that it is impossible to analyze these properties in terms of physical properties or any other non-phenomenal notion. Or they might claim that perceptual experiences have representational content in a phenomenal way and that it is impossible to analyze the notion of having content in a phenomenal way using functional terms, physical terms, or any other non-phenomenal notion.
Another development within Representationalism involves enriching the representational content of visual experience beyond the classic GOFR conception. Shoemaker (1994), for instance, argues that in addition to representing external objects and their features, color experience represents relations obtaining between objects (or surfaces of objects) in the environment and its own intrinsic properties. Peter Carruthers (2000), in turn, maintains that perceptual experience carries a (partly) reflexive content that makes a representational claim about its own intrinsic features. And Thau (2002) goes so far as to argue that the properties represented by visual experience cannot, in fact, be represented by conscious thought.
With this quick review of Representationalism in place, let’s turn back to the Scene-Immediacy of visual experience. This aspect of visual experience presents a basic problem for all versions of Representationalism in virtue of its role in an argument for a competing theory: Naïve Realism. According to the Naïve Realist, our visual awareness of external objects is not representational; rather, it involves “acquaintance”—a brute cognitive relation that obtains between the mind and external objects and their features. Acquaintance, in effect, makes external objects/features constituents of veridical visual experiences, for according to the Naïve Realist one could not have the same visual experience if one was not acquainted with the same external object/features.9
Naïve Realists often act as though their theory is uniquely poised to capture the Scene-Immediacy of visual experience. Scott Sturgeon (2000) summarizes the argument thusly:
They say its capacity to explain Scene-Immediacy springs from its metaphysics of veridical phenomenology. According to that story, recall, such phenomenology consists in brute contact between percipient, public object and public feature. Scene-immediacy is said to result. The idea is that brute contact makes it for the subject as if a public object and its features are directly before the mind. (p. 12)
According to these Naïve Realists, Representationalists lack the resources necessary for explaining Scene-Immediacy. Sturgeon articulates the basic idea:
Consider the difference between seeing a loved one and thinking of her. On the present view, both states are intentionally directed upon the beloved. Yet only the former is Scene-Immediate. Only visual experience as of the loved one is as if she’s Immediately before consciousness. Mere thought, alas, is not like this. Scene-Immediacy looks to be a special kind of directedness, somehow more than intentional directedness upon a scene. (p. 27, his emphasis)
As a card-carrying Representationalist, I don’t think that the only possible way to explain Scene-Immediacy is via acquaintance.10 But saying that it’s possible for a Representationalist to give a plausible account of Scene-Immediacy is one thing; actually providing the account is another.
In a way, the task is relatively straightforward for a non-reductive Representationalist: she can claim that visual experiences represent in a phenomenal manner, that the notion of “representing in a phenomenal manner” cannot be further analyzed in non-phenomenal terms, and that this notion involves Scene-Immediacy (which is, itself, a phenomenal notion that cannot be reduced to non-phenomenal notions). When a Representationalist is prohibited from appealing to phenomenal notions—when she favors a reductive version of Representationalism, as I do—the task of explaining Scene-Immediacy becomes more challenging.
At this point, I want to zero in on a couple of specific challenges that Scene-Immediacy presents to the defender of GOFR, challenges that are inspired by the previous descriptions of Scene-Immediacy. (These are not the only challenges for GOFR that one could generate from the previous descriptions; they are simply the ones I am focusing on in this paper). For expositional purposes, I’ll call these challenges “the Challenges of Scene-Immediacy”.
One recurring idea from the previous descriptions of Scene-Immediacy is that visual experience gives us a kind of “direct access” to objects not found in conscious thought. To revisit Searle’s example, visual experience seems to give us a type of “direct access” to the yellow wagon that conscious thought does not. Under GOFR, however, it seems that a visual experience gives us “access” to the wagon in virtue of representing it and gives us “direct” access to the wagon in virtue of directly representing it (as opposed to indirectly representing it by, say, directly representing the effects the wagon has upon our visual system). This creates a problem: given the above, natural GOFR interpretation of “direct access”, it follows that a conscious thought can also give us “direct access” to the wagon. The “First Challenge of Scene-Immediacy”, then, is for the defender of GOFR to make sense of the claim that visual experience provides a “direct access” to the wagon not found in conscious thought, given that both representational states are capable of directly representing that wagon.
The “Second Challenge of Scene-Immediacy” is closely connected to the first challenge. To explain this challenge, I need to draw a distinction between the content of the information carried by a representational state (i.e. what the information is about) and, for lack of a better word, the “source” of that information. Information about a yellow station wagon can come a variety of sources; what’s particularly important for the purposes of this paper is the fact that this information can come from the wagon itself (when you perceive that wagon) or it can come from one’s memories of the wagon.11 I take it that there is something about the Scene-Immediacy of the visual experience of the wagon that suggests that the source of the wagon-information it carries is the wagon itself,and not one’s memories. This, in turn, presents a challenge to the defender of GOFR, for according to this theory the phenomenal character of this experience is determined by its informational content and, as we have seen, information with the same content can (arguably) come from different sources.
What can the defender of GOFR say in response to these challenges? To date, GOFR defenders have tended to focus on the difference between perceptual experiences and thoughts more generally; not much GOFR ink has been split on the specific topic of the Scene-Immediacy of visual experience. With regard to the former, more general issue, a number of proposals have been advanced, including: that experience carries a “nonconceptual” content as opposed to the “conceptual” content of belief12, that experience carries an “analog” content in contrast to the “digital” content carried by belief13, that the content of experience is “richer” than that of thought14, and that experience are “poised” to impact the belief/desire system in a unique manner15.
Some of these ideas could be of service in our attempt to answer the Challenges of Scene-Immediacy. For example, a defender of GOFR could argue that visual experience gives us a kind of “direct access” to the wagon not found in thought in virtue of giving us far more information about that wagon than thought; indeed, she could argue that the abundance of information provided by experience is a reason for thinking that the source of this information is the wagon itself, as opposed to memory (which might be limited in terms of the amount of information it can hold about wagons). That said, I’m not going to examine the merits of attempting to extend this or any of the other GOFR ideas mentioned above to the Challenges of Scene-Immediacy. My project is to examine and defend another GOFR-friendly resource, a resource that has been largely ignored by defenders GOFR. This resource is the “fringe” of visual consciousness.
4. A Sketch of an Alternative GOFR Approach to Scene-Immediacy
As a first step in introducing my GOFR responses to the Challenges of Scene-Immediacy, let’s examine a description of the phenomenal character visual recently given by Alva Noë. Noë claims that not only do visual experiences represent various objects and features of the surrounding environment, they also tell us that other objects and features are accessible and could be represented in high detail in subsequent visual experiences.
The content of a perceptual experience is not given all at once the way the content of a picture is given in the picture all at once…I have a sense of the visual presence of the detailed scene before me, even though it is not the case that I see all that detail (or that I think I can see it all). As a matter of phenomenology, the detail is present not as represented, but as accessible. Experience has content as potentiality. In this sense, the detail is present perceptually in my experience virtually. (2004, p. 215, his emphasis)
This holds true even in the case of seeing a single object. Consider, for example, a visual experience of a single tomato.
Notice, however, that you do not, as a matter of fact, have the whole of the facing side of the tomato in consciousness all at once. The facing side has extent and shape and color, and you can’t embrace all this detail in consciousness all at once… Take a tomato out. Look at it. Yes, you have a sense that the facing side of the tomato is all there, all at once. But if you are careful you will admit that you don’t actually experience every part even of its visible surface all at once. Your eyes scan the surface, and you direct your attention to this or that. (p. 217, his emphasis)
These passages are compelling as descriptions of the phenomenal character of visual experience. But how, exactly, does visual experience present various details “as accessible”? What’s the mechanism by which these details become present as accessible within visual experience?Noë’s (2004) own account is that detail is present as accessible within visual experience in virtue of the subject having knowledge of various “sensorimotor contingencies”—relationships of interdependence between movement and sensory stimulation. If, for example, I know that looking right will bring the objects located there into my visual consciousness, those objects will be present as accessible within my visual experience.
I want to develop an alternative (but related) account of how the details of the scene before the eyes can be present as accessible within visual experience. My account centers on what William James would call “the fringe” of visual consciousness—low-detail conscious representations that tend to be in the periphery of our visual consciousness. (To be fair, it’s not clear whether James himself would interpret the fringe of consciousness in representational terms. But given my commitment to GOFR, you should not be surprised that I will.) These representations make additional high-detail information about an object “accessible” to the subject by providing a low-detail preview of various features of that object to which the subject can rapidly allocate high-detail visual representational resources. The main difference between this account and Noë’s is that although we both think that an object’s being present as accessible within visual experience has to do with the subject’s ability to shift high-detail representational resources (like fixation and attention) to it, I maintain that there must be a conscious representation that gives us a low-detail preview of that object. Noë, in contrast, thinks that we only need to have “sensorimotor knowledge” of how various changes in visual behavior would result in new sensation of the object (and/or its features). In short, Noë thinks an object can be present as accessible within the content of experience even if there are no conscious low-detail representations of that object.
With this sketch of how the fringe of visual consciousness makes details “present as accessible” in hand, I can outline the responses I want to give to the Challenges of Scene-Immediacy. The First Challenge is that a visual experience of the wagon seems to provide a kind of “direct access” to the wagon not found in conscious thought. I will argue that this access is, at least in part, the result of various details of the wagon being present as accessible within our visual experience.16 Although conscious thoughts can also have a fringe that provides some access to additional wagon-information (information stored in memory), the access to additional wagon-information provided by the fringe of thought is nowhere as rich as the access to additional wagon-information provided by the fringe of visual experience.
The Second Challenge of Scene-Immediacy is that there is something about the visual experience of the wagon that indicates that the source of the wagon-information it carries is the wagon itself (as opposed to memory). In response to this challenge, I will argue that there are aspects of the process of using the fringe of visual experience to acquire additional wagon-information that (fallibly) indicates that one is accessing this information from the wagon itself and not from some other source, such as memory.
I will now fill in the details of these responses, starting with a description of the fringe of visual experience.
5. The Fringe of Visual Experience
Any discussion of the fringe of consciousness should start with William James, who famously argued that conscious states are experienced as being elements of a stream.
Every definite image in the mind is steeped and dyed in the free water that flows around it. With it goes the sense of its relations, near and remote, the dying echo of whence it came to us, the dawning sense of whither it is to lead. (1890, p. 255)
According to James, this stream contains both substantive resting places—high-detail contents that “can be held before the mind for an indefinite time and contemplated without changing” (p. 243)—as well as a fringe. Unlike its substantive parts, the fringe of consciousness tends to be elusive to the act of introspection.
The attempt at introspective analysis in these cases is in fact like seizing a spinning top to catch its motion, or trying to turn up the gas quickly enough to see how the darkness looks. (p. 244)
Extrapolating from these ideas, let’s tentatively define “the fringe” as being constituted by elements of consciousness that are lower detail and difficultto introspect (in that attempts to do so often result in their replacement with higher detail elements). There are elements of visual experience that satisfy this (rough) definition of being fringe elements. These elements are not free riders in the stream of visual consciousness; they play an important role in managing some of the representational limitations of the visual system.
One of these limitations involves the physiology of the human eye. Due to the uneven distribution of cones across the retina, only an area of about 2 degrees (approximately the size of a thumbnail at arm’s length) at the center of the visual field is represented in high detail at any given moment. More specifically, this small area of the retina—the “fovea”—yields the best color perception of the retina.17 Motion sensitivity, in contrast, improves as you move toward the periphery of the visual field. Due to a higher concentration of rods, the ability to distinguish a dim light from a dark surround also improves as you move towards the periphery of the visual field. To compensate for these disparities, the fovea is aimed at objects within the environment using a sequence of swift movements or “saccades”, each lasting between 5 to 80msec, and respites or “fixations”, each lasting around 250msec.18