F*cking affordances - how do they work?

Over on Bounds of Cognition, Ken has been doggedly pursuing what he thinks is a critical problem with the concept of affordances as described by Gibson (1979) and expanded on by Turvey, Shaw, Reed & Mace (1981; hence TSRM). I feel the need to spend some time consolidating my responses and some ideas in one place; every time I try to lay out why the problem is ill-posed or a potential route out, Ken just says I'm clouding or avoiding the issue or throwing out red herrings.

I actually think Ken has identified one very useful critique of the affordance concept: the lack of care with which we attach '-able' to words. But Ken isn't just making a methodological point; he thinks he's shown that affordances cannot structure light in a way that can specify the affordance, and that's the argument that needs to be tackled.

Saddle up: this is going to take some time, and I'm not going to solve everything. But to get a little ahead of myself, the answer to the problem is that it's complicated.

These quotes are two ways of thinking about what affordances are. Gibson suggests they might be constituted by the composition and layout of a surface. TSRM then formalised Gibson's original description to make affordances dispositions (like the solubility of salt in water). Dispositions are anchored to the world - salt is disposed to dissolve in water because of the chemical properties of ionic salts and the electrical charges of water molecules. The anchoring properties underwrite the disposition so that it is a non-accidental property. Then, for both Gibson and TSRM, the affordance must structure the light (for vision) in such a way as to specify the affordance; this optical structure comes from the surface of objects.

Ken believes that neither the "composition and layout of surfaces" or the anchoring properties of an affordance can, in fact, structure light so as to specify the affordance. The problem, as he states it, is that only surfaces can structure light, but most (if not all) affordances are actually anchored by properties beneath the surface. Therefore affordances cannot structure light and cannot be visually perceived.

The fun began with this post, in which the following basic thought experiment emerged (paraphrased):

Take two identical boxes, say 3" on each side. Now take one of these boxes and rig it so that if you come close to it, it explodes and kills you, but so that this rigging doesn't alter anything about the surface of the box.

You can now actually only pick up one of the boxes - the exploding box cannot be picked up because it explodes if you come too close. Visually, however, the two are identical; so whatever perception is detecting, it cannot be the affordance 'pick-up-ability' because only one of the boxes actually has that property and you cannot perceive a property that isn't actually there.

The exploding box example is actually fairly distracting, though, because it requires an Evil Philosopher who rigs boxes; it's directly analogous to the 'equivalent configurations' issue I've discussed with respect to the Ames Room, and Runeson provided what I think is a nicely robust rebuttal to these types of arguments. Ken's problem seems to hold holds, though, for a more straight-forward affordance such as walk-on-ability: 

If a terrestrial surface is nearly horizontal (instead of slanted), nearly flat (instead of convex or concave), and sufficiently extended (relative to the size of the animal) and if its substance is rigid (relative to the weight of the animal), then the surface affords support. It is a surface of support...it is stand-on-able...walk-on-able...run-over-able...

Gibson, 1979, p. 127

All of these anchoring properties are capable of structuring light, with the notable exception of rigidity (relative to the weight of the animal). It's not clear how the strength of the surface can structure light for visual perception. It is required to anchor walk-on-ability, however, and this is Ken's entry point.

(A side note: Ken restricts his objections to vision because a) it simplifies the problem and b) both Gibson (1979) and TSRM are primarily discussing visual perception. The first point makes this problem a straw man: shrinking the problem space to the point where the solution can't fit is cheating. That said, many of the affordances he's discussing are ones that those authors use as examples of of visually specified affordances, so the issue remains to be addressed. I'm going to insist on access to the rest of the perceptual systems, though, because it matters.)

The science of affordances
Ken's spent several posts showing pictures of things and asking whether they visibly afford anything. My answer has usually been "No idea: you should run the experiment and see". This hasn't counted as an answer, because Ken is really just trying to pile on examples of things he thinks he already has the answer for. However, there is a way to find the answer to his question, and it requires empirical research as laid out quite clearly  by Gibson. 

TSRM propose the following formal description of Gibson's empirical approach:

The analysis of an affordance as a natural kind for an animal is two stage. The first stage is very much a matter of ecological physics: to isolate occurrent physical properties that are invariant over the extension and the perspectives and which are non-accidentally related to the extension of the affordance.... The second stage in analyzing an affordance for an animal is very much a matter (for visually detected affordances) of mathematical optics: to describe the light patterned by an affordance-specific occurrent property and to characterize that patterning in terms of an optical variable, an optical property, that stands in strict correspondence to the occurrent property.

Turvey et al, 1981, p. 264-265

The two pillars of Gibsonian ecological psychology are affordances and information. For TSRM, you must first identify the relevant anchoring properties which are specific to the affordance, then identify how these properties lawfully structure (e.g) light to produce information. The lawfulness allows the information to specify the affordance.

Affordances exist; the real question for a theory of perception is whether there is information for them, and whether organisms can detect this information. The latter is demonstrated by showing that organisms organise their behaviour with respect to the affordance; the former is demonstrated by showing that this behaviour varies predictably in response to perturbations of the proposed information.

Can affordance properties structure energy arrays?
Ken isn't going to like this answer much, and to be honest it's incomplete at this point because I'm still trying to tie it all together. But the answer to the question is: it depends, and it's complicated.

1. Affordances as a subset of dispositions
Not all dispositions are affordances. In the exploding box scenario, one box is disposed to explode if someone comes too close. However, by the rules of the example, there cannot be information for this disposition - the manipulation is designed to be imperceptible. To create an imperceptible property and then claim a failure to perceive it kills a theory of perception is nonsense. I've come to think of the problem in terms of sets: there is the set of dispositions, and affordances are the subset which are about action and for which there can, in principle, be perceptual information. The disposition to explode is therefore a disposition, and not an affordance.

Ken's objection, if true, would simply mean that the set of affordances as defined here is empty because he claims there can't be information. This was just ruling out dispositions that don't impact on action, to narrow the field and rule out exploding boxes. Let's now focus on our problematic case, walk-on-ability, and think about how this can be specified.

2. Affordances are specified across both space and time, and perceived across perceptual systems

Ken tried showing images of surfaces (say, ice on a pond) and asking 'Does the surface of this pond structure light in such a way as to enable a human to visually perceive that this is walk-on-able?'. The sticking point is the issue of surface rigidity - can it bear your weight? There are several lines of attack here, which I actually think all come into play.

Vision and rigidity: The human visual system is actually surprisingly sensitive to rigidity. A surface is rigid if the distance between points is preserved over a transformation, either moving the surface or moving the point of observation. Genuinely rigid relative motion sticks out like a sore thumb. So actually, perhaps it is available to vision, although of course the key property is rigidity relative to the weight of the organism. 

What behaviour are we actually trying to explain? If you actually run the study to see what people might do, I think they'd be inclined to test the surface before committing their whole weight. So visually? Perhaps not; but there is, indeed, information available to a mobile exploring organism with multiple perceptual systems, and it's important to remember that this is what we are.

In addition, perception-action is not perfect. I talk a lot about how good the system is; I'm not interested in illusions and failures, I'm interested in our successes. But we do fail, and these failures can tell us something about how we were controlling our behaviour. We do fall into unexpected holes; we do occasionally stop paying enough attention and trip on that flat footpath; we do walk out on ice and fall over on it or through it. This is part of the actual behaviour we have to explain, and why Gibson insisted you have to do this empirically: an ecological psychology tries to explain what animals actually do.

The continuous nature of surfaces: When I am standing on a footpath and visually examining the footpath ahead for it's walk-on-ability, I can see two things: the surface ahead of me, and the fact that this surface is continuous with the surface I am currently happily walking on. I can now perceive the full affordance of this surface, but this perception occurs using multiple systems and spans current information about what I'm on as well as prospective information about future conditions.You could even, in principle, restrict this to just vision: you can visually perceive the future path, it's continuity with where you are, and the fact that you are standing and mobile. So again, perhaps vision can do the trick, but even if this doesn't quite work, we still have other information.

Continuity matters: for example, when I fell on ice and broke my wrist I was locomoting under the control of an affordance spanning a) my perception of the safe surface I was on and b) the perception of the icy patch ahead, which contains no information about friction, let alone the change in friction. To my perception-action system the surface was continuous and I acted in accordance with that perception (with disastrous consequences; but that's a separate issue). Surfaces can of course be perceptibly discontinuous, as in the case of the transition from the shore to the iced over pond. But this has consequences for behaviour - as I noted, I'm willing to bet people on the shore would alter their locomotion to explore the ice. 

Of course, I walk into work over numerous types of surface with numerous discontinuities and I don't explore every one - why not? This brings us to what I think is the often neglected third pillar of ecological psychology - perceptual learning. Neglected by modern researchers and philosophers, but not, of course, by Gibson.

I think we forget precisely how much experience we have in interacting with the world. Infants spend a huge proportion of their time learning simply to stand, then cruise, then locomote, and even once they've achieved these basics it takes years of practice, millions of steps, to become a truly expert walker. The perception-action basics (locomotion, prehension, postural control) are what we spend out entire waking lives doing and redoing. 

Add to this the modern, carpentered world. Gibson described how the natural world could be suitably reliable to support skilled, direct perception for action; I, however, live in a city designed by people for people, in which the regularities we depend on are manufactured into the system on purpose. I have extensive experience with the range of surface discontinuities I can expect in a city, and I have learned that none specify a functional discontinuity: the world may have a little break in it but to perception, that break effectively doesn't exist. Out in the more natural world, many surfaces are locally continuous for long periods, and when they aren't, it's either a discontinuity the observer has learned doesn't matter or it's a discontinuity that triggers a discontinuity in locomotion behaviour. 

The other part of perceptual learning is learning what it is the information specifies. All an organism has access to is information; the TSM laws account includes a symmetry principle which links the world to this information and (critically) the information back to the world. We learn to differentiate invariant features of energy arrays because things that persist can be learned over time; these features persist because they are lawfully generated by the ecological physics of the event or affordance; we then come to learn that this feature specifies that affordance. You can trick this system: a shark detecting a fake electrical signal might still come looking for the fish that signal specifies in it's niche. But that trick requires you to break the scope of the law the shark is depending on to underwrite the specification relation: the trick needs an Evil Biologist, and, like the exploding box case, there's no way for an organism to tell the difference. 

Organisms come to know what the information specifies by using that information to control action. That information can control action successfully because it is reliably (lawfully, in fact) present in specific circumstances: when the event or affordance is present. Laws underwrite a 100% correlation; in fact, laws are the only way to get a 100% correlation, in the long run. So organisms are sensitive to this reliability, and come to perceive what the world means over time, and all they ever have is their first person perspective: they never, ever get to peer behind the curtain.

Summary
I doubt Ken will find any of this useful. But that's not my problem. Gibson's ecological psychology has a lot of room to manoeuvre in  it, and this is not a distracting weakness. The room comes from the use of ecological laws, the recognition that perception-action is fundamentally extended over both space and time, and the recognition that we have multiple complementary perceptual systems with access to numerous energy arrays, and that a gap in one is often simply filled in by another. This is why Gibson was so insistent that laying out affordances is an empirical task: with this many degrees of freedom, there is more than one solution, and this flexibility is the reason perception and action can succeed to the degree that it does.

This is a huge post, and there's a lot in here where I've glossed over the details. All of this will come back up, so game on!

Tweet