Neuroscience
As Andrew has been tackling a new job description for the brain (part 1 and part 2), several comments have been made that suggest that his approach (and the ecological stance in general) might be fine for perception/action, but not for other types of tasks/behaviours. Later on in this post I also think about how we might be able to distinguish between association and direct access to meaning, which is another idea that has been coming up repeatedly in the comments (see here).
In this post I want to think about what makes perception different from conception (Andrew reviews William James' views on this distinction here). I will argue that both occur as networks of evolving neural activity (with perception, this network extends to the environment and the body), but these networks have different properties because they are driven by signals of differential stability. I do not think it is accurate to think of perception as something that happens in V1, for example, and conception as something that happens in SFG. If parts of the brain reverberate in a system along with parts of the brain that are directly interfacing with an energy array in the environment, it seems correct to refer to that whole system as perception. In contrast, if a network of activity appears to be relatively encapsulated with respect to external energy arrays, it seems correct to refer to that system as an act of conception. For now I will leave aside the problem of how conceptual networks develop, but I hope to return to this later.
By this characterisation, a conceptual network is identical to a perceptual network except that it is not being driven and constrained by the detection of perceptual information. This avoids a dualism between perception and conception while providing a basis for predicting different properties of the two systems.
In principle, it should be possible to distinguish perception and conception by analysing the properties of networks of neural activity. In practice, of course, we lack the tools to do this (although mathematical tools do exist that can distinguish between properties of different types of networks). Still, it is worth considering what would be the expected characteristics of perceptual vs. conceptual networks if we could peer inside the brain to observe them.
Perception: Perception involves directly interfacing with energy arrays produced by the environment. This perceptual information is structured and continuous. As the information hits some type of receptor (e.g., in the retina) the energy interacts with specialised cells to generate structured neural signals. I have no idea what the fidelity is between perceptual information and the neural signals it produces, but I think it's safe to say that there is a tight coupling between perceptual information generated by the environment and the resulting neural signal (if signal conjures too linear a picture, think of a reverberating neural system). The tight coupling between environment and neural activity must, to some extent, be communicated to the body to permit us produce flexible, appropriate actions in a dynamic environment.
Perceptual networks should be driven and constrained by the conversion of perceptual information in an energy array into a structured neural signal. In an action task, there will be additional information arising from the perception of our bodies as they perform. The structure provided by this interface with the external environment (and due to the fact that our bodies also possess relatively stable properties) should make perceptual networks relatively coherent and stable and they should reflect the dynamics embodied by the perceptual information driving them. The particular characteristics of a perceptual network will also be influenced by the history of activating the network (i.e., long-term changes in the brain that support a particular type of network behaviour), whether the network has been activated recently (i.e., medium-term changes in proximate network activity such as calibration), and other networks of activity (perceptual or conceptual) that might impinge.
The neural activity resulting from the interface between brain and structured energy arrays in the environment will continuously evolve in response to changes in perceptual information. In this sense there are not discrete networks that activate identically each time a given type of information is present. Instead, the present network will be pulled into a different state to reflect changing environmental conditions.
To the extent that the brain is not entirely single minded, the perceptual neural system can be impinged on by other networks of activity. There will be threshold effects such that the system is resistant to perturbations, except those of sufficient intensity. For example, if I am very pre-occupied with my growing to-do list at work, I might not pay as much attention to avoiding other pedestrians when walking down the street. But, if I am only a little concerned with my to-do list, then my walking performance will be unaffected. There will also be degree effects such that the system can either be influenced (but retain its essential characteristics) or interrupted (you are no longer a device that can perform that particular perception/action task). Thus, there are two predicted hallmarks of a perceptual system of neural activity: 1) some direct interface with an energy array and 2) stability against influence from other networks.
It is worth noting that this characterisation applies to a trained system - one that has already learned to effectively detect the relevant perceptual information (reflected in the long-term changes mentioned above). During learning, a source of instability in a perceptual network will be the organism's ability to reliably perceive the information variable.
Conception: Dreaming is one of the purest examples of conception. The perceptual world intrudes on dreams from time to time but in dreams we are more cut off from perceptual information than at any other period of state of consciousness. Compared to waking life, dreams are highly unstable. Locations appear to change in an instant, events do not necessarily follow any sensible causal sequence, impossible things occur.
Leaving aside these qualitative observations, what would be the expected properties of a network that was not directly interfacing with a stable external environment? At a guess, it seems like the stability of such a network would depend on whether or not a person has a history of activating this network (again reflected in long-term neural changes as a consequence of learning), whether the network has been activated recently (medium-term changes reflecting residual activation of the same or similar networks), and whether there are other competing networks of activity that might impinge. Without structured neural activity driven by the perception of perceptual information, conceptual networks loose an important source of stability. Even in the highly practiced domain of language, consider the differential stability of talking to oneself (without sub-articulating) and speaking aloud. Even so, language confers some important stability to conceptual networks, in that we can use words to maintain an idea in the absence of external perceptual information about that idea. Generally speaking though, conceptual networks will be relatively susceptible to perturbation. Thus, the predicted hallmarks of a conceptual system of neural activity are: 1) no direct interface with an external energy array and 2) instability to interference from other networks.
If it were possible to peer into an active brain and observe its activity in great detail, it would be possible to distinguish between perception and conception on the basis of the above descriptions. Although this is not possible, I think it is worth considering this distinction in light of another issue - how can we tell whether we have direct access to meaning (DAtM) via perception or whether the detection of perceptual information is associated with a meaningful representation? This issue has cropped up several times in comments to previous posts. Andrew and I have both expressed some concern about what type of evidence could distinguish one claim from another. Are they simply different ways of saying the same thing?
I will attempt to outline how these accounts might differ in the language of perception and conception described above. At this point I am not completely confident that I am not just re-describing the problem. But, I think there might be an opportunity here to use network metrics to disambiguate DAtM from association.
Association implies two or more things paired together (a representation of a tone becomes linked to a representation of food), whereas DAtM implies one thing with a higher order interpretation (tone means food). On the basis of these crude descriptions it seems like one could sensibly predict association and DAtM to be distinguished by their instantiation in networks of neural activity. Take a case where a rat has been extensively conditioned to expect a food to follow presentation of a tone:
For association: During training, the rat would experience two consecutive, independent networks of perceptual neural activity - one for the tone and one for the food (I'm obviously oversimplifying by referring to a single network of activity about food, but I want to avoid details like that for now). The activation of the tone network reliably predicts the activation of the food network. Overtime, basic associative neural learning mechanisms begin to reflect the statistical reliability of the tone / food pairing. In a trained rat one would observe a coherent system of perceptual neural activation (perception of a tone) causing the activation of a separate system of conceptual neural activation (thinking about food). These two systems should be relatively independent, except to the extent that the perceptual system causes the conceptual one to fire.
For DAtM: During training the rat would experience the continuous evolution of a perceptual network that is first driven by auditory tone information and is subsequently driven by visual and taste information about food. In the interim between tone and food the network would begin to loose coherence, but stability would return with the presentation of visual and taste information from the food. As I argued earlier, I do not think it makes sense to talk about discrete perceptual networks that fire the same way regardless of what else is happening. I think it is more appropriate to focus on transformation from one type of network into another based on perceptual demands. By this account, the rat is experiencing a single compound event (food -> tone) with a predictable dynamic. A key idea of event perception is that experiencing part of the event can give you direct access to the rest of that event. So, for example, hearing a bar of your favourite song gives you access to the rest of that song, not through association but by the activation of a single event structure with a coherent spatio-temporal dynamic. If this logic is extended to the current example, for a trained rat, experiencing the tone could give him direct access to the entire tone -> food compound event. In contrast to the association case, one would not observe two independent networks, but the evolution of a single network.
Still, one might argue that these two cases are very similar and might be indistinguishable. Both start with a network of activation reverberating with the perceptual information of a tone. But, the DAtM case involves the activation of a single (albeit complex) event structure. Therefore, the observed activity should exhibit the characteristics of a coherent perceptual network. On the other hand, the association case involves the activation of a perceptual network and an independent conceptual network. Thus, the observed activity will differ from the DAtM case because a conceptual network about food will be relatively unstable, susceptible to perturbation, and disconnected from an interface with the environment.
This is a very early first swing at the problem and I am admittedly neither a neuroscientist nor an expert in complex networks. I have tried to make minimal assumptions about both subjects in the hope that what I've suggested is at least somewhat plausible. To a large extent, I agree with Andrew saying that he's tired of having the argument and would like to move onto thinking about new positive ways to approach the problem. But, I'm also a little bothered by the idea that we can't figure out what type of evidence distinguishes an ecological account from a representational one, or a DAtM account from an associative one. So, in this post I've tried to start at the bottom, and hopefully later I can work my way up to making sensible predictions about what behavioural differences we'd expect.
- Nih Videocast: Network Neuroscience
From the 17th of May, 2010: Network Neuroscience - Connectivity and Dynamics of the Human Brain by Dr. Olaf Sporns View the presentation "Nervous systems are complex networks of interconnected neural elements that engage in spontaneous or evoked dynamics....
- A Taxonomy Of Information
Over the past several months I've been thinking about how perception falls within a hierarchy of types of information use. This was spurred by my ideas about an ecological approach to language, in which perceptual information and linguistic information...
- Language: A Task Analysis (kind Of)
In the last post, I discussed the similarities and differences between language and other types of information. From the first person perspective, spoken language is just another type of auditory event. The main distinction between the word "dog" and...
- Reading Group - The Problem Of Time (heft, 2000)
How big is the present? Classic theories of perception all essentially assume that 'now' is some instantaneous slice of time, and that the task of perception is to reconstruct the flow of experience from the snapshots provided by, for example,...
- Reading Group - Gibson (1979) Chapter 4
In this chapter Gibson really steps up the pace, so hold onto your hat. First, he shows that perception can’t be based on sensation. Then he shows that we can’t see light. Finally, he shows that we don’t see images. Here we go... Chapter 4: The...
Neuroscience
What's the difference between perception and conception?
As Andrew has been tackling a new job description for the brain (part 1 and part 2), several comments have been made that suggest that his approach (and the ecological stance in general) might be fine for perception/action, but not for other types of tasks/behaviours. Later on in this post I also think about how we might be able to distinguish between association and direct access to meaning, which is another idea that has been coming up repeatedly in the comments (see here).
In this post I want to think about what makes perception different from conception (Andrew reviews William James' views on this distinction here). I will argue that both occur as networks of evolving neural activity (with perception, this network extends to the environment and the body), but these networks have different properties because they are driven by signals of differential stability. I do not think it is accurate to think of perception as something that happens in V1, for example, and conception as something that happens in SFG. If parts of the brain reverberate in a system along with parts of the brain that are directly interfacing with an energy array in the environment, it seems correct to refer to that whole system as perception. In contrast, if a network of activity appears to be relatively encapsulated with respect to external energy arrays, it seems correct to refer to that system as an act of conception. For now I will leave aside the problem of how conceptual networks develop, but I hope to return to this later.
By this characterisation, a conceptual network is identical to a perceptual network except that it is not being driven and constrained by the detection of perceptual information. This avoids a dualism between perception and conception while providing a basis for predicting different properties of the two systems.
In principle, it should be possible to distinguish perception and conception by analysing the properties of networks of neural activity. In practice, of course, we lack the tools to do this (although mathematical tools do exist that can distinguish between properties of different types of networks). Still, it is worth considering what would be the expected characteristics of perceptual vs. conceptual networks if we could peer inside the brain to observe them.
Perception: Perception involves directly interfacing with energy arrays produced by the environment. This perceptual information is structured and continuous. As the information hits some type of receptor (e.g., in the retina) the energy interacts with specialised cells to generate structured neural signals. I have no idea what the fidelity is between perceptual information and the neural signals it produces, but I think it's safe to say that there is a tight coupling between perceptual information generated by the environment and the resulting neural signal (if signal conjures too linear a picture, think of a reverberating neural system). The tight coupling between environment and neural activity must, to some extent, be communicated to the body to permit us produce flexible, appropriate actions in a dynamic environment.
Perceptual networks should be driven and constrained by the conversion of perceptual information in an energy array into a structured neural signal. In an action task, there will be additional information arising from the perception of our bodies as they perform. The structure provided by this interface with the external environment (and due to the fact that our bodies also possess relatively stable properties) should make perceptual networks relatively coherent and stable and they should reflect the dynamics embodied by the perceptual information driving them. The particular characteristics of a perceptual network will also be influenced by the history of activating the network (i.e., long-term changes in the brain that support a particular type of network behaviour), whether the network has been activated recently (i.e., medium-term changes in proximate network activity such as calibration), and other networks of activity (perceptual or conceptual) that might impinge.
The neural activity resulting from the interface between brain and structured energy arrays in the environment will continuously evolve in response to changes in perceptual information. In this sense there are not discrete networks that activate identically each time a given type of information is present. Instead, the present network will be pulled into a different state to reflect changing environmental conditions.
To the extent that the brain is not entirely single minded, the perceptual neural system can be impinged on by other networks of activity. There will be threshold effects such that the system is resistant to perturbations, except those of sufficient intensity. For example, if I am very pre-occupied with my growing to-do list at work, I might not pay as much attention to avoiding other pedestrians when walking down the street. But, if I am only a little concerned with my to-do list, then my walking performance will be unaffected. There will also be degree effects such that the system can either be influenced (but retain its essential characteristics) or interrupted (you are no longer a device that can perform that particular perception/action task). Thus, there are two predicted hallmarks of a perceptual system of neural activity: 1) some direct interface with an energy array and 2) stability against influence from other networks.
It is worth noting that this characterisation applies to a trained system - one that has already learned to effectively detect the relevant perceptual information (reflected in the long-term changes mentioned above). During learning, a source of instability in a perceptual network will be the organism's ability to reliably perceive the information variable.
Conception: Dreaming is one of the purest examples of conception. The perceptual world intrudes on dreams from time to time but in dreams we are more cut off from perceptual information than at any other period of state of consciousness. Compared to waking life, dreams are highly unstable. Locations appear to change in an instant, events do not necessarily follow any sensible causal sequence, impossible things occur.
Leaving aside these qualitative observations, what would be the expected properties of a network that was not directly interfacing with a stable external environment? At a guess, it seems like the stability of such a network would depend on whether or not a person has a history of activating this network (again reflected in long-term neural changes as a consequence of learning), whether the network has been activated recently (medium-term changes reflecting residual activation of the same or similar networks), and whether there are other competing networks of activity that might impinge. Without structured neural activity driven by the perception of perceptual information, conceptual networks loose an important source of stability. Even in the highly practiced domain of language, consider the differential stability of talking to oneself (without sub-articulating) and speaking aloud. Even so, language confers some important stability to conceptual networks, in that we can use words to maintain an idea in the absence of external perceptual information about that idea. Generally speaking though, conceptual networks will be relatively susceptible to perturbation. Thus, the predicted hallmarks of a conceptual system of neural activity are: 1) no direct interface with an external energy array and 2) instability to interference from other networks.
If it were possible to peer into an active brain and observe its activity in great detail, it would be possible to distinguish between perception and conception on the basis of the above descriptions. Although this is not possible, I think it is worth considering this distinction in light of another issue - how can we tell whether we have direct access to meaning (DAtM) via perception or whether the detection of perceptual information is associated with a meaningful representation? This issue has cropped up several times in comments to previous posts. Andrew and I have both expressed some concern about what type of evidence could distinguish one claim from another. Are they simply different ways of saying the same thing?
I will attempt to outline how these accounts might differ in the language of perception and conception described above. At this point I am not completely confident that I am not just re-describing the problem. But, I think there might be an opportunity here to use network metrics to disambiguate DAtM from association.
Association implies two or more things paired together (a representation of a tone becomes linked to a representation of food), whereas DAtM implies one thing with a higher order interpretation (tone means food). On the basis of these crude descriptions it seems like one could sensibly predict association and DAtM to be distinguished by their instantiation in networks of neural activity. Take a case where a rat has been extensively conditioned to expect a food to follow presentation of a tone:
For association: During training, the rat would experience two consecutive, independent networks of perceptual neural activity - one for the tone and one for the food (I'm obviously oversimplifying by referring to a single network of activity about food, but I want to avoid details like that for now). The activation of the tone network reliably predicts the activation of the food network. Overtime, basic associative neural learning mechanisms begin to reflect the statistical reliability of the tone / food pairing. In a trained rat one would observe a coherent system of perceptual neural activation (perception of a tone) causing the activation of a separate system of conceptual neural activation (thinking about food). These two systems should be relatively independent, except to the extent that the perceptual system causes the conceptual one to fire.
For DAtM: During training the rat would experience the continuous evolution of a perceptual network that is first driven by auditory tone information and is subsequently driven by visual and taste information about food. In the interim between tone and food the network would begin to loose coherence, but stability would return with the presentation of visual and taste information from the food. As I argued earlier, I do not think it makes sense to talk about discrete perceptual networks that fire the same way regardless of what else is happening. I think it is more appropriate to focus on transformation from one type of network into another based on perceptual demands. By this account, the rat is experiencing a single compound event (food -> tone) with a predictable dynamic. A key idea of event perception is that experiencing part of the event can give you direct access to the rest of that event. So, for example, hearing a bar of your favourite song gives you access to the rest of that song, not through association but by the activation of a single event structure with a coherent spatio-temporal dynamic. If this logic is extended to the current example, for a trained rat, experiencing the tone could give him direct access to the entire tone -> food compound event. In contrast to the association case, one would not observe two independent networks, but the evolution of a single network.
Still, one might argue that these two cases are very similar and might be indistinguishable. Both start with a network of activation reverberating with the perceptual information of a tone. But, the DAtM case involves the activation of a single (albeit complex) event structure. Therefore, the observed activity should exhibit the characteristics of a coherent perceptual network. On the other hand, the association case involves the activation of a perceptual network and an independent conceptual network. Thus, the observed activity will differ from the DAtM case because a conceptual network about food will be relatively unstable, susceptible to perturbation, and disconnected from an interface with the environment.
This is a very early first swing at the problem and I am admittedly neither a neuroscientist nor an expert in complex networks. I have tried to make minimal assumptions about both subjects in the hope that what I've suggested is at least somewhat plausible. To a large extent, I agree with Andrew saying that he's tired of having the argument and would like to move onto thinking about new positive ways to approach the problem. But, I'm also a little bothered by the idea that we can't figure out what type of evidence distinguishes an ecological account from a representational one, or a DAtM account from an associative one. So, in this post I've tried to start at the bottom, and hopefully later I can work my way up to making sensible predictions about what behavioural differences we'd expect.
- Nih Videocast: Network Neuroscience
From the 17th of May, 2010: Network Neuroscience - Connectivity and Dynamics of the Human Brain by Dr. Olaf Sporns View the presentation "Nervous systems are complex networks of interconnected neural elements that engage in spontaneous or evoked dynamics....
- A Taxonomy Of Information
Over the past several months I've been thinking about how perception falls within a hierarchy of types of information use. This was spurred by my ideas about an ecological approach to language, in which perceptual information and linguistic information...
- Language: A Task Analysis (kind Of)
In the last post, I discussed the similarities and differences between language and other types of information. From the first person perspective, spoken language is just another type of auditory event. The main distinction between the word "dog" and...
- Reading Group - The Problem Of Time (heft, 2000)
How big is the present? Classic theories of perception all essentially assume that 'now' is some instantaneous slice of time, and that the task of perception is to reconstruct the flow of experience from the snapshots provided by, for example,...
- Reading Group - Gibson (1979) Chapter 4
In this chapter Gibson really steps up the pace, so hold onto your hat. First, he shows that perception can’t be based on sensation. Then he shows that we can’t see light. Finally, he shows that we don’t see images. Here we go... Chapter 4: The...