Chemero (2009) Chapter 6: Information and Direct Perception

In the previous chapter, Chemero laid out his first idea as to how a radical embodied cognitive science could be a science; he suggested taking a 'dynamical stance' in which researchers use simple dynamical systems models such as the HKB to drive empirical work. In this chapter, he moves towards his more recent suggestion, namely using Gibson's ecological psychology as a theoretical basis from which to make predictions and run experiments. This is a much more robust idea; sciences need theories or else they wander from phenomenon to phenomenon, which I believe to be a real issue in modern psychology and cognitive science.

Chemero has his own flavour of ecological psychology, however. He believes it stays true to Gibson's essential framework but expands it to cope with new and important issues. The two pillars of ecological psychology, information and affordances, remain. Chapter 6 is Chemero's expanded notion of information, while Chapter 7 is his updated theory of affordances as relations (which I've covered briefly before).

Chemero briefly summarises the key points from Gibson (1979); perception is direct (no inference), perception is for action, perception is therefore of affordances. These have simple implications for the information for perception: it must be ubiquitous, it must be unambiguous, and it must be about affordances. How this might be possible was most rigorously laid out by Michael Turvey, Robert Shaw and William Mace, most notably in the 'ecological laws' paper in response to Foder & Pylyshyn (1981). Information can come to specify properties in the environment by virtue of the local, ecological laws that govern the structuring of energy (say, light) into patterns which the organism can detect.

Chemero worries that the Turvey-Shaw-Mace (TSM) reliance on natural law is too restrictive and means that there won't be enough information available for perception. He's concerned that there are things we need to be able to perceive and understand (and that we clearly do perceive and understand) that are not underpinned by any natural law. If this is the case, then either a) we do not perceive these directly, or b) there is information available which we use that is not covered by the 'laws' account. Only (b) is an option for RECS.

Within their scope, ecological laws provide direct and correct information about properties of the world. These properties will be shared by numerous objects: Chemero gives the example of the optical pattern O which specifies a particular type of tree, the silver maple, for a squirrel. He then notes that of the maples in the squirrel's environment, some will have particular relevance: that one is where the squirrel's nest is, for instance. Chemero claims that if information requires an ecological law, there is no information for these trees as individuals.

This point seems trivial to dismiss, which makes me worry about the rebuttal. Information gains its structure from the underlying dynamical structure of an object or event. For instance, there is a class of events, 'fly balls', which occur in baseball and are an example of a projectile motion dynamic. This physical category then has instances; fly balls that can be caught vs. fly balls that are heading out of the field for a home run, for instance. The dynamics are the same, and the information is therefore of the same type, but each is distinguished by different parameters (different velocity, perhaps, or a different initial angle of ascent). Each instance of the event type is different, the exact value of the information variable is different, and to the extent a perceiver can perceive the difference, behaviour can be different (a skilled fielder will not bother chasing a home run, for example). In the literature, this is referred to as the kinematic specification of dynamics, and like many good things comes from Sverker Runeson (e.g. Runeson & Frykholm, 1983). Another good although dense and technical exposition is Bingham's chapter in the book 'Mind as Motion' (Bingham, 1995). Some of these issues cropped up in the comments here, too.

There seems, therefore, to be information about individual instances of types which is still grounded in natural law. Even more usefully, a given object or event will produce more than one information variable. Each individual can be identified as a unique set of information with specific values; and if they cannot be identified as such, it's because the differences aren't perceptible. This process, of increasing differentiation of particulars from the flux of experience, is precisely what the Gibsons suggested perceptual learning was doing.

Turvey-Shaw-Mace rely on natural law for a very good reason: direct perception requires that there is a non-accidental relationship between the properties of the world and the information it produces. Conventions aren't sufficient, because they can be violated. Chemero's second concern is with events who carry meaning by virtue of convention, e.g. language. While the perception of the spoken word is presumably governed by lawful relations, the meaning of that word is merely conventional. For TSM, you would therefore have information about which word had been spoken, but no information about what that word meant. 

I actually think this is a fair description of the facts. I have yet to hear anything resembling a plausible ecological theory of language (speech perception and production, yes; language and the meaning of words, no). It does indeed seem to be the case that language is a system where the acts of speaking and listening can occur via Gibsonian information, but where the content is conventionally determined. I think language is a real challenge to ecological psychology that has yet to be met. I actually think Chemero's suggestion (situation semantics - see below) is not a crazy idea for this, although it has it's problems as well.

Chemero will take this challenge (of dealing non-representationally with conventional content) and use it to motivate a broader conception of information which spans both natural law and conventional cases. The motive is good: he wants a notion of information that allows direct perception of all these things, so that a RECS grounded in ecological psychology can remain non-representational at all levels. I remain unconvinced that the solution works in the way Chemero wants, however.

Barwise, Perry and situation semantics
Chemero suggests that the situation semantics of Barwise & Perry is both broad enough and Gibsonian enough to do the trick (these authors were, in fact, directly influenced by Gibson's work). I don't know a great deal about this work; what follows is derived from Chemero (p.116-122). I do know Barwise's name from my time at IU where he was universally respected and considered a very clever man. Perry also has a good reputation, as I understand it; so the source is good.

Barwise and Perry proposed that there is information for organisms within situations; a given situation will be an instance (token) of a type of situation, and situations can be connected by constraints. If two types of situation, S1 and S2, are connected via a constraint, then a token s1 is informative about a token s2 by virtue of that constraint. There are two features of this setup:

1. An organism has access to the information if it has access to one of the situations and the constraint.
2. The constraint can be a law (as before) or a custom or convention. Smoke can convey information about fire (via a law) or about a message (via the conventions of smoke signals).

So the suggestion is simple: situation semantics provides a framework for there to be information via both laws and conventions. The constraint must only be reliable; good enough for whatever you're using it for. Chemero then claims that this scheme is sufficient to underpin a radical cognitive science; perception is direct when there is an informational coupling between the world and organism that doesn't require a representation, and the coupling in situation semantics is of precisely this sort.

Why make this move? Chemero feels that there are things which we know that aren't underpinned by laws. His example is that the presence of a beer can is informative about the presence of beer, even though there is no lawful connection between the can and it's contents - through error or foul play the can may merely contain water, but this is sufficiently unlikely that it's a safe bet there's beer inside. It seems to be the case that the presence of the beer can is, in fact, informative about the (likely) presence of beer. If information requires laws this process must be indirect, which is of no use to RECS. If information merely requires a constraint, then the presence of a beer can is directly perceivable information about the presence of beer.

Consequences of expanding beyond laws
As I said above, the reason TSRM framed everything in terms of laws is that they wanted a way in which the structuring of light by affordances was non-accidental; this allows perception to be direct, because no inference is required to cope with the possibility of mere correlation. Specifically, this directness is underpinned by a specific consequence of lawfulness, the 'symmetry principle' (Shaw & McIntyre, 1974):

We can represent the symmetry principle as follows. Let E = ‘‘The environment is the way it is,’’ I = ‘‘The information is the way it is,’’ and P = ‘‘Perception is the way it is.’’ Also, let ‘‘>’’ stand for the logical relation of adjunction, a nontransitive conjunction that we can read as ‘‘specifies.’’ Then, the symmetry principle is

[(E > I) & (I > P)] & [(P > I) & (I > E)].

In English, this says: ‘‘That the environment is the way it is specifies that information is the way it is and that information is the way it is specifies that perception is the way it is, and that perception is the way it is specifies that the information is the way it is and that information is the way it is specifies that the environment is the way it is.’’ We can simplify this to say that the environment specifies the information, which specifies perception, and perception specifies the information, which specifies the environment. This principle is symmetrical in that the environment, information, and perception determine one another. This, on the Turvey-Shaw-Mace view, is what it is for perception to be direct.

Chemero, 2009, p. 111

This principle is critical, because it describes the route that information can take so as to specify the world in information, and it also describes how picking up the information takes an organism all the way back to the world. Detecting information created via the first part is equivalent to detecting the world, because of the symmetry between the first and second parts; no inference required.

Chemero correctly notes that the major objection to his account of information from ecological psychologists is losing this symmetry. Situation semantics is quite specifically not symmetrical: "information flows about tokens in virtue of constraints among types" (p. 122) and so the link from perceiver to the world (via tokens of the situation types) is not equivalent to to the link between the world and the perceiver (situation types; see the diagram on p. 121).

Ecological information processing
Chemero concludes the chapter with a discussion of two examples of how organisms use action to 'process' information (i.e. make it available for direct perception, etc). Weirdly, neither of these entail his situation semantics version; both are law-based. What they have in common is reliance on higher-order variables.

The first example is the variable tau (τ) which David Lee proposed could serve to specify 'time-to-contact' and thus allow the direct control of numerous interception tasks. τ is a higher order variable (mathematically defined as the ratio of image size and image expansion rate), but this ratio does not need to be computed  by the visual system because it actually describes a structure in optic flow that occurs when a surface is approaching a point of contact at a fixed velocity. Detecting that structure is to directly perceive the time-to-contact.

τ has a mixed history; Lee's original studies with gannets, etc are a little suspect, τ itself only specifies time-to-contact over an ecologically uninteresting scope (motion along the line of sight with constant velocity), and there is a long modern literature on interceptive action that suggests τ is not actually used anyway (James Tresilian from the University of Warwick has spent a lot of time on this problem; see Tresilian (1999) for an excellent review). This is an interesting topic in and of itself (including the historical footnote that Lee found τ in Fred Hoyle's book The Black Cloud but never tells anyone this), but for now let me just say that a) τ is indeed a good simple example of ecological information processing (visual information for time-to-contact does require action to become available) but that b) it turns out not to be an actual thing that gets used, so while it can serve as a useful, simple example, it's not anything I'd want to rely on too heavily.

The second example is analogical reasoning (making inferences about one situation on the basis of your knowledge of a similar situation). This is part of Chemero's offence, where he takes not only the straight-forward perception/action case (τ) but also a 'representation-hungry' task, and shows how it can be understood using ecological/radical methods. He describes (p. 129) experiments by Wasserman and colleagues that show pigeons and baboons can effectively do analogical reasoning by directly perceiving the higher order variable entropy. This variable is effectively a measure of disorder, and ranges from 0 (every element in an array is identical) to 'at least 1 element is different' and up to 'every element is different'. The exact values depend on the size of the array. Wasserman showed a pattern of behaviour in their animals that was best explained by the perception of entropy, and Chemero has produced a two-layer neural network that replicates this qualitative pattern. The lack of hidden layers in the neural network is a proof of concept that entropy can be discriminated without an intervening representational layer (i.e. directly). It's not clear what the information for entropy is, but this is certainly an interesting first cut at the problem and these assaults on 'representation-hungry' problems is a feature I admire of Chemero's approach.

Some thoughts
There are interesting elements in this chapter. The summary of the TSM formalism of Gibson's system is an excellent and readable resource that captures all the key points. Given how opaque Turvey can get when left to his own devices, this is no small thing. The discussion of situation semantics is also clear, and I assume a fair description, and the final section on what information processing might look like to an ecological psychologist is great (although τ is not as strong an example as he makes out, and the entropy story, while interesting, is in it's infancy).

However, Chemero's proposal for a theory of ecological information which includes meaning from conventions (via constraints) falls flat for me - I simply don't think the motivations for it work. I don't think his concerns about perceiving individuals matter, and it's not clear to me why I need to directly perceive beer presence from beer can presence. Squirrels famously don't remember where they bury nuts; they just look in places where, if they were a squirrel, they would have buried nuts. and use smell to hone in. I think beer-presence might not be what we perceive; perhaps the opportunity for beer hunting behaviour? I don't want to get too caught up on the specific example, but Chemero has not made it clear here what we do, in fact, know about beer-presence from beer-can presence, and these are, as ever, empirical questions.

I think the loss of the symmetry principle is a disaster for a theory of direct perception, which is why TSM make it such a central feature of the laws account. Directness is entirely underwritten by symmetry, and Chemero is simply too glib in his dismissal of this as a problem. Effectively, he defines 'direct perception' as 'not involving representation', which permits him to invoke the explicitly non-representational situation semantics of Barwise & Perry; but convention simply isn't enough to underwrite directness. I also don't share Chemero's pessimism about law based accounts: I think lawful information is going to be entirely ubiquitous enough.

Worse, though, I feel like there's another real problem for a theory of direct perception lurking in situation semantics. Chemero notes that situation semantics requires that the organism have access to the situation and the constraint in order to have any information: the constraint is what does the work (by acting between types of situations), but all you have access to are tokens of situations. In the laws scheme, the law does it's work 'behind the scenes' by restricting the set of possible situations. You don't have separate access to the law, but this is fine because you don't need it. The operation of the law makes the information the kind of thing that can be informative (see the first half of the symmetry principle). It's not clear how you  can have 'access' to the constraint without it being in the form of supplementary knowledge, because constraints are things which don't necessarily structure, say, light. Perhaps, like laws, mere constraints can 'work behind the scenes' to simply make it the case that the connection exists - but with no symmetry principle and no lawful necessity it is not clear that this will work well enough. While there may be ways of implementing this that aren't representational, it a) bangs the door wide open to a representational account and b) that implementation needs to be more carefully explicated than it is here. Gibson and TSM went to great lengths to discuss how properties of the world came to be specified by a process of projection into energy arrays; there is, as yet, no clear account for how this works for constraints.

Conclusion
In effect, I don't see that there is any need to give up on laws just yet. Turvey famously tells researchers that if your experiments haven't revealed the lawful relation yet, it's because you haven't looked hard enough so don't stop now. I tend to think this is a healthy attitude, and I don't see yet any need to abandon ship for the problematic land of situation semantics.

Next time
Next stop is Affordances, 2.0. As I've discussed before, Chemero has his own account of affordances as relations which he expands and develops in the next chapter.I think I'm going to break that chapter up, because the post I've prepared on Affordances 1.0 and 1.1 is already long.

Bingham, G.P. (1995). Dynamics and the problem of visual event recognition. In Port, R. & T. van Gelder (eds.), Mind as Motion: Dynamics, Behavior and Cognition, (pp403-448). Cambridge, MA: MIT Press. Download

Fodor, J., & Pylyshyn, Z. W (1981). How direct is visual perception?: Some reflections on Gibson's “ecological approach” Cognition, 9 (2), 139-196. DOI

Runeson, S., & Frykholm, G. (1983). Kinematic specification of dynamics as an informational basis for person-and-action perception: Expectation, gender recognition, and deceptive intention. Journal of Experimental Psychology: General, 112(4), 585-615. DOI 

Shaw, R., & M. McIntyre (1974). Algoristic foundations to cognitive psychology. In Cognition and Symbolic Processes, ed. W. Weimer and D. Palermo. Hillsdale, N.J.:Erlbaum.

Tresilian JR (1999) Visually timed action: time-out for ‘tau’? Trends in Cognitive Science, 3(8),  301-310. DOI

Turvey, M. T., Shaw, R. E., Reed, E. S., & Mace W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981) Cognition, 9 (3), 237-304. DOI 

Tweet