A Perception/Action Model of Coordination

Other coordination posts are here.

The role of perception in the dynamic leading to the HKB phenomena has been made clear by the work so far. But the exact form of this dynamic had yet to be modelled; the HKB model only consists of two cosine functions superimposed on each other to produce the two attractors at 0° and 180°, and the dynamic pattern hypothesis it embodies made predictions which did not held up empirically. It was now time to take an explicitly perception/action approach to modelling the task, which means it's finally time to turn to (my PhD advisor) Geoff Bingham's model (2001, 2004a, 2004b). This model is a fully perception/action model, and the modelling strategy Bingham lays out is, I think, a masterclass in how to go about building models of this kind.

The Action Bits
The first part to be modelled is a rhythmically moving human limb (two of which will be coupled later). Two earlier perturbation studies (Kay, Kelso, Saltzman & Schöner, 1987; Kay, Saltzman & Kelso, 1991) had demonstrated that such a limb exhibited five key properties, all of which a model has to produce:

• limit cycle stability: stability is a measure of the preferred state of a system, and you discover this by perturbing the system and seeing what it relaxes into doing. The attractors at 0° and 180° in the HKB model are examples of point attractors; a limit cycle is a periodic attractor, and so the system tends to relax into a periodic motion with some frequency and amplitude. Critically, this form of stability means that the underlying dynamic must be non-linear.
• phase resetting: phase is the measure of position within a cycle, and if you perturb a system you are effectively changing that position abruptly. If the system's position is a function of time, then the response to a perturbation at time t is for the system to go back to where the function specifies it must be at that time. If the phase resets itself (by returning to the limit cycle at a different point) then the dynamic is not being driven as a function of time. It is autonomous, i.e. being driven as a function of it's own behaviour.
• inverse frequency/amplitude relation: if you ask people to produce rhythmic movements and steadily increase their frequency, they will (if allowed) reduce the amplitude of their movements to achieve this.
• direct frequency/velocity relation: as frequency increased, so did the peak velocity. This unsurprising result is also a side effect of the mechanism for the frequency/amplitude relation (an increase in stiffness)
• rapid relaxation time, independent of frequency: the limb returned to the oscillation described by the limit cycle quickly and this time did not vary with frequency.

These properties mean that the model of the oscillating human limb must be nonlinear and autonomous, and also exhibit the last three properties. Kay et al (1987) modelled the limb by combining two well-known systems (the Rayleigh and the van der Pol oscillators) into a customised hybrid; but this hybrid oscillator bears no clear relation to the actual human movement system and thus this specific model was no use to Bingham.

Bingham started with the so-called λ-model, a model of actual human limbs in which movements are generated by moving the equilibrium point of a damped mass-spring (see Feldman, 2010 for a recent overview). A mass spring is literally a mass bouncing on the end of a spring; a damped one includes friction and so, unless driven, it will eventually come to rest at it's equilibrium point. Feldman's basic suggestion was that a) human limbs are organised into synergies (Bernstein, 1967) best described as damped mass springs, and b) you could control these synergies easily by simply controlling the equilibrium point.

The basic form of a damped mass-spring is
where x, x-dot and x-double dot are the state variables position and it's derivatives velocity and acceleration, and b and k are parameters (damping and stiffness, respectively). You keep such a spring moving by driving it (setting this equal to something other than 0). If the driver's value changes as a function of time, then you have a non-autonomous dynamic, which is no good. If, however, the driver's value is a function of the state variables (position and it's derivatives) only, then the spring drives itself - it is autonomous. With a suitable driver, this oscillator produces all the required characteristics from the list above.

We now simply take two oscillators of this basic form, and couple them together via perceptual information.

The Perception Bits
This action system now needs to be controlled, and this is what perception is for. The driver on this system must be autonomous, specify relative phase and reflect the known characteristics of the perceptual information for relative phase:

• it must be in principle detectable by both vision and proprioception
• it must be more readily detected at 0° and 180° than anywhere else
• it must be composed of state variables only (for the overall dynamic to be autonomous)

One unresolved question was whether this information was available throughout the trajectory, or whether it was only detected at the endpoints (peak amplitude). This latter point was plausible because information only available at the endpoints predicts that only 0° and 180° would be stable. An experiment reported in Bingham (2004a, 2004b) added phase variability everywhere along display trajectories except either a) at the endpoints, b) at peak velocity, or c) both. The results demonstrated that a) the variability was detected in all the conditions and b) that it was detected less clearly at 180° when present only at peak velocity. The conclusion was that the information is available throughout the trajectory but it's detection is a function of the relative speeds of the oscillators.

You can control a single oscillator autonomously by driving it with it's own phase, because phase is simply the arctangent of velocity over position (i.e. it is composed of state variables):
If you instead drive one oscillator with the phase of the other one, the two would now be coupled, but still autonomous. So far so good, but there's one final constraint: this coupling also varies as a function of relative phase (this is, after all, the key result from the coordination studies) and thus the drivers need to include a term for relative phase which is, itself, autonomous.

There is (indirect) evidence that this information term is the relative direction of motion; the HKB phenomena only emerge for parallel motion, i.e. when relative direction is uniquely defined (Bogaerts et al, 2003; Wilson et al, 2005; Wimmers et al, 1992). More recently we tested this more directly (Wilson & Bingham, 2008) but I will talk about that paper separately as it has a lot of important elements in it.

The current form of the model is therefore
Pay close attention to the subscripts; oscillator 1 is driven by the perceived phase of oscillator 2, modifed by the relative direction term Ρ (rho). Ρ(i,j) is simply +/-1 depending on whether the two oscillators are moving in the same or opposite directions. The value is affected by a noise term proportional to the velocity difference (i.e. the ability to resolve relative phase depends on the relative speed).

This model is the first (and currently the only) fully perception/action model of coordinated rhythmic movement. It explicitly models both perceptual and action components as having specific forms, and it places these components in a specific organisation with respect to each other. Critically, this model produces all the basic effects (0° and 180° the only stable states, 0° more stable than 180°, transition frequencies around 3-4Hz, plus limit cycle stability, autonomy and thus phase resetting* and the inverse frequency-amplitude relation). It instantiates a fully perception/action set of testable hypotheses about the identity of the information and the specific role it plays, and can be used to model both action and judgment experiments. It is, quite frankly, a master-class in how to go about building such a thing.

It is not over yet, however. The evidence for relative direction is compelling but indirect, and the model in this form cannot handle learning, say, 90°. My next post with discuss our perturbation study in which we identified the information for the visual perception of relative phase (Wilson & Bingham, 2008) and our recent empirical forays into learning which will constrain the future development of the model.

**************************
*Phase resetting has one interesting feature in human limbs movements. If you perturb an oscillator by slowing it down, the phase resets 'backwards' (i.e. it ends up behind where it would have been if left unperturbed); if you speed it up briefly, the phase resets 'forwards'. Human limbs, however, always rest 'forwards'; Kay et al hypothesised that this is due to the fact that actual limbs respond to perturbations by stiffening, a suggestion supported by the data and the modelling.

References
Bernstein, N. (1967). The coordination and regulation of movements. London: Pergamon Press.

Bingham, G.P. (2001). A perceptually driven dynamical model of rhythmic limb movement and bimanual coordination. Proceedings of the 23rd Annual Conference of the Cognitive Science Society, (pp. 75-79). Hillsdale, N.J., LEA Publishers. Download

Bingham, G. (2004). A Perceptually Driven Dynamical Model of Bimanual Rhythmic Movement (and Phase Perception) Ecological Psychology, 16 (1), 45-53 DOI: 10.1207/s15326969eco1601_6 (Download)

Bingham, G.P. (2004b). Another timing variable composed of state variables: Phase perception and phase driven oscillators. In H. Hecht & G.J.P. Savelsbergh (Eds.) Theories of Time-to-Contact. Boston: MIT Press. Download

Bogaerts, H., Buekers, M. J., Zaal, F. T., & Swinnen, S. P. (2003). When visuo-motor incongruence aids motor performance: the effect of perceiving motion structures during transformed visual feedback on bimanual coordination. Behavioural Brain Research, 138, 45-57. DOI

Feldman, A. G. (2010) Space and time in the context of equilibrium-point theory. Wiley Interdisciplinary Reviews: Cognitive Science. DOI

Kay, B. A., Kelso, J. A. S., Saltzman, E. L., & Schöner, G. (1987). Space-time behavior of single and bimanual rhythmical movements: Data and limit cycle model. Journal of Experimental Psychology: Human Perception and Performance, 13, 178–192. Download

Kay, B. A., Saltzman, E. L., & Kelso, J. A. S. (1991). Steady-state and perturbed rhythmical movements: A dynamical analysis. Journal of Experimental Psychology: Human Perception and Performance, 17(1), 183–197.DOI

Wilson, A. D., & Bingham, G. P. (2008). Identifying the information for the visual perception of relative phase. Perception & Psychophysics, 70(3), 465-476. Download

Wilson, A. D., Collins, D. R., & Bingham, G. P. (2005) Human movement coordination implicates relative direction as the information for relative phase. Experimental Brain Research, 165, 351-361. Download

Wimmers, R. H., Beek, P. J., & van Wieringen, P. C. W. (1992). Phase transitions in rhythmic tracking movements: A case of unilateral coupling. Human Movement Science, 11, 217-226. DOI

Tweet