James Gibson's Ecological Approach to Cognition

Edward S. Reed

INTRODUCTION

James J. Gibson was a highly controversial figure in modern psychology, who excited great passion among proponents and detractors of his theory alike. About the only matter on which Gibson's adherents and opponents have agreed is that his is an anti-cognitive theory of perception (compare e.g. Michaels and Carello, 1981, with Ullman, 1980). As it turns out, this sentiment -however widely promulgated- is simply and utterly false. Gibson thought of his as a cognitive psychology and of perception as a cognitive function. It is a fact that Gibson (rightly) dismissed modern 'cognitivism' (e.g. information-processing models, artificial intelligence, cognitive science), with its archaic yet incoherent theory of mental representations operating on stimulus inputs. But Gibson's dismissal of these facile, modish theories was not his position on cognition as such. Ever since the 1950s, when he self-consciously abandoned the response theory of cognition and behaviorism (Reed, 1986), Gibson considered his account of perception as a theory of how animals come to know their environments -a theory of cognition. Gibson's last two books (1966, referred to hereinafter as SCAPS; 1979, referred to hereinafter as EAVP) are replete with explicit discussions of the psychological issues concerning cognition, language, memory and imagination, discussions which have been ignored or worse throughout the entire secondary literature on Gibson's work.

In this chapter I attempt to reconstruct Gibson's cognitive [p. 143] psychology. My account is based primarily on SCAPS, EAVP and other published sources [including Gibson's (1982) selected essays called Reasons for Realism (cited by year followed by 'RR')]. Where necessary for context and clarification I have used unpublished material of two sorts. First, I have used passages from Gibson's 'purple perils', spirit-duplicated (hence the 'purple') memoranda Gibson used in his weekly seminar and which he distributed to interested colleagues... [cited as 'PP' followed by a date]. Second, I have used manuscript material housed in the Gibson archives at Cornell University...[cited by date .... and a file number].

When writing EAVP, Gibson (1975, 5-52) made the following note to himself:

Fields of Cognitive Psychology

I. The perception of the environment and its affordances for behaviour
Learning and the education of attention
Control of locomotion and manipulation
Orientation to the far environment

II. Perception and knowledge at second hand
Pictures, models, toys, (Images)
Experiments with graphic displays

III. Knowledge available in language
Speech, writing, concepts, words
Association, traces, memory, learning
Experiments with verbal items.

This review will cover all of these areas (though not in this order) except the control of locomotion and orientation to the far environment (cognitive mapping, so-called) which deserve separate discussions. I have also brought some of Gibson's ideas about knowledge at second-hand and about the role of language in knowing under the single rubric of 'representation systems', a term of my own invention, although the idea for it derives from SCAPS. Criticism of Gibson's ideas is here kept to a minimum. My goal in this chapter is to expound a set of profound and novel ideas that deserve serious discussion, not the kind of ignorant dismissal they have received. [p. 144]

THE COGNITIVE REVOLUTION

Modern cognitive psychology has its roots in behaviourist learning theory. Already in the 1930s and 1940s, Edward C. Tolman was developing his 'cognitive' interpretations of rat maze-learning experiments. In the 1950s, as the ideas of cybernetics and computation theory became widely known, there were the beginnings of a full-scale revolt against the stultifying assumptions of classical learning theory. Cognitive psychology's equivalent of storming the Bastille was Noam Chomsky's (1959) attack on B.F. Skinner's Verbal Behavior (1957). Chomsky showed that Skinner's view of language was missing an important aspect of language, namely its cognitive function. A language is more than a system of labels or associations, it is more than a complex lexicon of phrases called up at appropriate times by experience and circumstances. A language is also a system of predication, which requires a grammar as well as a phrasebook. Language is an evolved specialisation for sharing knowledge about the world.

Chomsky's attack on learning theory, and the contemporaneous developments in computation theory, led to a growing realisation that the psychology of stimulus, response and association had severe limitations. Human behaviour could not be cut off from human knowledge, which was more than a mere linking of stimuli and responses. Apparently human beings 'processed' stimulus inputs, creating grammar-like, rule-based structures for interpreting other stimuli, or for planning future actions (Miller, Galanter and Pribram, 1960). These structures, linked by grammar-like hierarchies of rules and not by experienced associations, came to be called mental representations, and cognition came to be viewed as the formation and use of representations according to rule (Chomsky, 1980; Johnson-Laird, 1983).

The so-called cognitive revolution in psychology has thus not challenged the truly basic assumptions of S-R psychology, so much as it has added a new set of assumptions. Cognitive psychologists who study perception do not deny that it is based on stimuli (as Gibson does), but they do deny that association of stimuli (or even Gestalt organisation of inputs) is sufficient to explain all of perception (Neisser, 1967; Pinker, 1985). [p. 145] Something else is needed: the rule-based processing of inputs into representations. Cognitivists who study action do not deny that it is based on responses, but they question whether the association of responses through innate linkages and learning is all there is to action. Isn't action also based on the rule-governed implementation of routines based upon stored knowledge, such as locomoting around the environment with the aid of a cognitive map (Gallistel, 1980; Requin, Semjen and Bonnet, 1984)? The cognitivists' idea is that a stimulus-response psychology can be made complete only by adding to it a psychology of mental representations or mental models that turn stimulus inputs into knowledge, and organise responses meaningfully (Johnson-Laird, 1983; Pylyshyn, 1984, Fodor, 1985).

Gibson's cognitive psychology also emerged from a critique of S-R psychology, but a very different critique. And, where cognitivists attempted to modify and supplement S-R principles, Gibson tried to overthrow them and replace them with an entirely different approach to psychology. The cognitivist critique of learning theory started with the problem of language and the kind of knowledge gained through language, whereas Gibson's critique of behaviourism started with perception and the kind of knowledge it facilitates. Gibson thought his perception-based critique was deeper than the other, and that is why he called for the replacement of S-R psychology, not its supplementation.

GIBSON'S CRITIQUE OF BEHAVIOURISM

The behaviourist or response theory of perception -which Gibson, following his teacher Holt (1915), espoused for years- involves the kind of associative principles which Chomsky and others have criticised so effectively. The basic idea is that one does not perceive an object until one has made a discriminative specific response to it. This may sound odd, but it is actually just a more sophisticated version of the classic and widespread 'touch teaches vision' theory. The child is said to learn to see that a tennis ball is soft by having experience in touching it and subsequently associating those tactile experiences [p. 145] with the visual experiences of round, white (or orange), and so on. Meaningful perception is thus learned perception, and learned perception requires associating sensory stimuli with previous experiences, an association based on behaviour (see Gibson and Gibson 1955/RR for a critique). Just as words are supposed to gain meanings through experienced association, so are sensory inputs.

There are many problems with such response theories of perception. First of all, as Gibson (1933; 1937) long ago showed, visual experiences of the same stimulus can change without any responses being made. Second, perceptual learning can be shown to occur without any reinforcement linking stimuli to responses (E. Gibson 1953). This raises the question (at least for behaviourists) of, third, what kind of reinforcement does occur in perceptual activity (Woodworth, 1947; Gibson, 1958)? In behaviour, animals and people supposedly repeat and learn those responses which succeed; in perception, observers tend to learn what is the case, regardless of the outcome. The exploratory behaviour involved in perception would seem to be of an entirely different order than the performatory responses studied by behaviourists (SCAPS, Ch. 2). Fourth, there is the vexed problem of what should count as a stimulus. Behaviourists tended to be glib about this, referring to objects and other animals as stimuli just as readily as they referred to physical energies as stimuli (Gibson, 1960/RR). Gibson's (1961/RR) discovery that complex energy patterns counted as information for perception, combined with his critique of the stimulus concept, and his novel idea of exploratory action, led to his complete rejection of S-R psychology (Reed, 1986).

For Gibson, perception now became not a response to stimuli, or the awareness of the mental representations caused by stimuli; rather perception was defined as an observer's awareness of the environment, based on information and the active, exploratory pick-up of information. Information is the basis for contact with the environment because it is specific to its sources in the environment. It is external to the animal, and various exploratory actions of perceptual systems are required for perception to occur. Observers look, listen, sniff, taste and feel by orienting and moving their heads, [p. 147] hands and other sensory organs. Neither the sensory inputs nor the motor responses of these organs count as the information, they are simply what enables an observer to detect external information. One can track a moving object by moving one's eyes with a stationary head, or by moving one's head and keeping one's eyes steady, or by combining these two procedures. In each case there are very different stimuli and responses, but the awareness of the moving object can be the same if the information picked up is the same (see Gibson et al., 1958 for experimental evidence).

On this information-based theory of perception the meaningfulness of perception does not derive from any form of association, only from the information actually detected by an observer. Whereas cognitivists explain perceptual knowledge of the world as a construction out of stimulus inputs Gibson explains it as the ability of the nervous system to home in on useful information: 'Instead of postulating that the brain constructs information from the input of a sensory nerve, we can suppose that the centers of the nervous system, including the brain, resonate to information' (SCAPS, p. 267). This resonance is a circular process that can achieve a state of equilibrium when the information being sought is detected. Perceptual learning is the process of becoming able to differentiate more and more kinds of information, and to increase the range and economy of the detection process (SCAPS, Ch. 13; E. Gibson, 1969). Perceptual learning is thus a change in both behaviour and awareness, and, according to the Gibsons, it is the fundamental cognitive process.

The modern proponents of cognitive psychology started with certain traditional principles: that stimuli and responses are the building-blocks of behaviour, that stimuli lead to sensations or sensory experiences, and that these stimulus inputs are the basis of our contact with the environment. These assumptions led them to treat cognition as the processing or interpretation of inputs into stored knowledge (representations) which can be accessed or re-called for use in language, memory and thinking. Gibson's rejection of these same assumptions resulted in a radically different definition of both perception and cognition. The novel and unusual definitions he proposed have led to the common contention that Gibson did [p. 148] not have a theory of the psychological processes underlying perception and cognition (see e.g. Heil, 1979; Ullman, 1980, and see the replies by Reed, 1980; Reed and Jones, 1981). The problem here is that what cognitive psychologists call psychological processes are processes Gibson treated as irrelevant or incidental to cognition, and cognitive psychologists have not yet even considered the processes of cognition discussed by Gibson:

Associating, organizing, remembering, recognizing, expecting, and naming -all these are familiar psychological processes, and all of them have been appealed to in the effort to explain the growth of knowledge. But all of these processes were first conceived as operations of the mind upon the deliverances of sense, and they still carry some of this implication. They have now been examined, one by one, and I have suggested that, as commonly understood, they are incidental, not essential to the developing process of information pickup. They need to be reinterpreted. The deeper, underlying kinds of perceptual development seem to involve exploration and attention (SCAPS, p. 283).

To understand Gibson's argument here requires first distinguishing and defining two kinds of cognition -direct and indirect; and second, analysing the psychological processes involved in each of these kinds of knowing.... [p. 149]

.... [p. 166]

....

CONCLUSION: THE ECOLOGY OF SYMBOLISM

The tendency to separate symbols from their symbolisers, to hypothesise a mental realm that guides our actions in the physical environment is an ancient and pervasive fallacy in [p. 167] western thought. However, once one gets stuck in this realm of the ideal, no one has ever figured how to get back out to the real world (Fodor, 1980). The problem of meaning and cognition cannot be solved by considering human behaviour physically, as responses to stimuli. But it is equally true that the hypothesis of an ideal realm of meaningful symbols does not further our understanding of cognition considered as knowledge of the environment. The correct question to be asked is how people come to know the meaningful environment, not how they respond to stimuli, nor how they become aware of inner ideas. No one has stated this better than Gibson (SCAPS, p. 26):

In the study of anthropology and ecology, the 'natural' environment is often distinguished from the 'cultural' environment. As described here, there is no sharp division between them. Culture evolved out of natural opportunities. The cultural environment, however, is often divided into two parts, 'material' culture and 'non-material' culture. This is a seriously misleading distinction, for it seems to imply that language, tradition, art, music, law, and religion are immaterial, insubstantial, or intangible, whereas tools, shelters, clothing, vehicles, and books are not. Symbols are taken to be profoundly different from things. But let us be clear about this. There have to be modes of stimulation, or ways of conveying information for any individual to perceive anything, however abstract. He must be sensitive to [stimulus information] no matter how universal or fine-spun the thing he apprehends. No symbol exists except as it is realised in sound, projected light, mechanical contact, or the like. All knowledge rests on sensitivity.

The process of direct perception has been differentiated and refined in a vast number of ways by human societies and civilisations. A tentative list of some of these peculiarly human modes of indirect cognition is given in Table 10.2. Note that each mode of indirect awareness has its roots in ecological information. Note also that each mode is standardised and embodied in culturally developed representation systems. These cultural-historical entities are preserved, modified and used in communication and teaching. They are the fundamental tools of cognition. It is about time that cognitive science broke out from the stale metaphors of computerese into the rich field of human history and comparative culture, so that we can begin to investigate these marvellous and diverse ways of apprehending reality, and of sharing our knowledge with others. True, [p. 168] cognition can and does proceed without materially produced writing, pictures, etc., but without these it is seriously curtailed, and education becomes exceedingly awkward and ritualised, instead of facile and available to individuals on their own. These aids to cognition, these tools of the intellect, have now become cultural institutions, in which all participate. The study of these institutions is the province of sociology, anthropology and history. But to understand the basic function of these modes of cognition is the job of psychologists. What enables these systems to be used for sharing awarenesses? How is the information coded, the tacit knowledge made explicit? What about historical and cultural variation and the [p. 169] cultural specificity of representation systems? These are important questions that cognitive psychology can ignore only at the peril of becoming irrelevant to human concerns.