Understanding others actions via Simulation

Understanding others actions via Simulation

1. Start: Understand others with visual matching stored in memory. Match stored memory representation with visual input we receive at the time. Approach to summarise lit - dif view - alternative way in which we recognise others action is make use of own motor systems.

2. Central Qs

   • Might recognizing someone else’s action involve one’s brain simulating the motor movements involved in producing the viewed action? (This week). Might recognizing someone else’s emotional expression involve one’s brain simulating the motor movements involved in producing the viewed expression, or simulating the changes in body-state associated with the viewed emotion, or both? (next week) - recognising someones emotional expressions might involve simulation. When feel and express emotion you also undergo bodily changes represented in own brain. Recognising someone else - simulate online those changes?

3. Outline

   • 1. Behavioural and imaging evidence that the motor system is involved in the perception and understanding of other’s actions. 2. Is the action observation network (AON) a human ‘mirror neuron’ system (MNS)? 3. Theoretical accounts of action simulation in the AON.

4. Introduction

   • A range of evidence shows that the perception of another’s action can automatically activate one’s own representations for producing that action, and that that influences how one perceives and understands the observed action. E.g. evidence that the perception of actions is constrained by the observer’s implicit knowledge of the movements that he or she is capable to produce. Lots of behavioural evidence showing perception of anothers action can automatically activate own action. So discuss what studies could design to test idea that somehow own ablity to produce an action enables you to judge and percieve action in somebody else.Dancer - expert in particular movements. compare with those who arent experts. Expect experts to recognise it better.
     • 3 behavioural examples. 1. Have people view actions they have experience producing vs not experienced. Are they better at perceiving movement in other people than non experts? 2) How producing an action oneself influences perceiving someone else producing action. Compatibility of viewed vs undertaken action. 3) Related to expertise. Train someone to learn novel action. Show ppl images of ppl doing novel action. Test again, have they improved? Numerous studies show how the visual perception of an action engages compatible activity in the observer’s motor system (premotor and parietal cortices). For both intransitive actions (e.g., unconscious postural imitation) and transitive actions (e.g., grasping an object). Also can look at the neuroimaging studies of production vs perception

5. Behavioural and imaging evidence that the motor system is involved in the perception and understanding of other’s actions

   • Hamilton et al 2004. Your own action infuences how you perceive another person. Early demonstration of paradigm when asked to make visual judgement of action when viewing compatible/incompatible actions. Hand moving a box. Task: judge how heavy the box is, using only the kinematics or video they are viewing. Have to make judgements based on action of others. While doing it, either had to hold similar box or pick up boxes. Cond 1 = hold boxes - control. Cond 2 - picking uo boxes themselves - experimental. In experimental, varied weight of box. Hypothesis 1 = compatibility hypothesis. Picking heavy weight will increase judgement of weight of viewed box. Hypothesis 2 - contrast hypothesis. oicking heavy box will lead them to underestimate the weight of the viewed box.When picking up a heavy box, you tend to underestimate the weight of the viewed box, so supported contrast hypothesis.
     • Also looked at which brain regions correlated across ppts which underestimate box they were viewing. (2006). Which brain regions are related to this systematic bias, in which lifting a box influences participant’s perceptual judgments of the weight of a box lifted by another person? Five regions show correlations with this bias: left inferior frontal gyrus, left central sulcus (primary motor cortex) – also activated by box lifting, left extrastriate body area, left lingual gyrus, right intraparietal sulcus. Certain regions where activity correlates across ppts with the degree of bias the ppts show that picking up box affects perception
   • Casile and Giese 2006. Learning a novel motor behavior (without visual input) improves the ability to visually recognize that action in others. Participants had to discriminate between pairs of point-light walker stimuli (identical or not?), that differed in the arm/leg phase differences before and after nonvisual motor training. Motor training: Ps were blindfolded and learned a novel coordinated upper-body movement based only on verbal and haptic feedback. Arm movements that matched the arm movements of the gait pattern. Trained people to learn new action and looked at effects of that to percieve others doing action. Improved ability to recognise others doing action. Test - had to judge whether stimuli were identical or note. Changes in timing of arm/leg movements. Trained on particualr way of moving arm.
     • Results: There was a selective improvement of the visual discrimination performance for the learned movement, despite the absence of visual stimulation during the motor training phase. Visual discrimination performance after training correlated strongly with the accuracy of the execution of the learned motor pattern. Those who were more accruate at producing the action were better at discriminating the same action
   • The Classical action observation network
     • Bilateral set of structures in left and right hemisphere. Frontal gyrus region, inferios parietal cortex, inferiior frontal and parietal regions. More visual and multisensory regions too!
       • Inferior frontal & parietal & lateral occipitotemporal cortices & STS are (critically) involved in observation (& imitation) of face and body movements. Leslie et al 2004. Had ppt either imitate facial expression at same time of viewing or not viewing exps. Similar activity in parietal particularly for imitation. Observation and Execution (Kircher et al 2013 - activity when either observing or undertaking/executing an action). TMS pSTS (Grossman et al 2005). Oosterhof et al 2013. Saygin et al 2004, 2007- viewing biological motion - get activity. . Lesion overlap: impairment distinguishing biological from scrambled (Saygin 2007). These regions show up time and time again in studies with aactions, applies to facial, arms, legs, whole body actions. Observing and producing actions seemto involve same regions to some extent.
   • The AON: A fMRI meta-analysis. Caspers et al 2010. Significant meta analysis results for action observation - summarised over all effectors. Significant results for a) the conjunction and b) the contrast analysis between the main categories action observation and action imitation
     • The LOTC as part of the AON. Lingnau and Downing 2015. Region of superior temporal sulcus etc. Early on were considered as action observation network. They said defo part - called LOTC. Extrastriate body area, visual motion prcessing.
   • Action observation activates premotor and parietal areas in a somatotopic manner. Buccinso et al 2001. Early study to show it activates in somatopic manner - different subregions activated depending on the effect. Facial regions lower, more up through arm, legs and feet higher. This viewing seems to apply to viewing somebody elses actions. Observation of object seems to also activate areas of motor sensory cortex. Because it involves touch.
   • Calvo-Merino et al. (2006). Evidence that the ‘‘mirror system’’ uses specialized motor representations rather than general processes of visual inference and knowledge to allow us to understand observed actions. Activation associated with observation of moves for which the participant does compared to does not possess the motor schemata. Look at effects of expertise. Greater activation in AON when viewing dance movemtents if viewer is expert. Used expert ballet dancers: some part of ballet was female/male specific movements, Male dancers see both, but expertise in male and vice versa. Looked at the effects on brain of male/female dancers viewing male/female dance movements. Get perietal cortex activity, motor cortex activity. Some brain activity involved in experience of producing dance movemetns and viewing it.
     • Calvo-Merino et al. (2005) - Evidence that the brain processes of action observation are modulated by the expertise and motor repertoire of the observer. Effects of expertise on brain responses. Experts in classical ballet, experts in capoeira and inexpert control participants viewed videos of ballet or capoeira actions. Instead of single dance, had all different types of dance. Looking at video clips of these dances. Activity for classical ballet dancers looking at classical ballet vs capoiera and vice versa, mulitplied together. Same regions - inferior frontal and inferior parietal.
   • Learning of a novel movement sequence increases activity of AON regions when Ps observe others making those movements - e.g. Cross et al. (2006). People in brain scans. Learn novel movement sequence. This increased activity in network when obserrving other people do action. Viewing short movie clips of movements, LEarnt novel action. Scanned at the same time. Either video was same or not. Found that in subregion of network in left hemisphere correlated with their ability to do the movement. Better they thought they would be at movement, activity raised.
   • Hamilton & Grafton (2006) - Evidence that the goal of an observed action is encoded by regions of the left intraparietal sulcus (within the AON/MNS). Participants observed a series of movies depicting goal-directed actions. Some goals were novel and others repeated relative to the previous movie. Repeated presentation of the same goal caused a reduction of the BOLD response in 2 regions of the left intraparietal sulcus These regions were not sensitive to the trajectory taken by the actor's hand. Viewed several different sequences, sometimes repeated, soemtimes new action followed. Study relies on the fMRI adaptation - idea that if you repeat stimulus and look at activity, tend to find activity will decline over time. What that region is sensitive too. If dont see decrease then brain region not sensitive to stimulus. Therefore, these regions are not encoding the goal of the goal direction actions.
     • However - Brass et al. (2007). Evidence that action understanding in novel situations is primarily mediated not by the mirror system but by an ‘inferential interpretive system. Participants observed an unusual action in implausible vs. plausible contexts - Context is a cue to the intention of the action. No differential activation in mirror network regions. Brain areas that lack mirror properties but which are involved in inferential interpretive processes of rationalization and mentalization were more active in the implausible context. Certain circumstances where AON not egaged. Context of action influences whether or not AON is engaged. Did not find regions engaged. View 3 dif types of video clips - person turning on light switch with knee (either ibject in one hand, onject in both hands, or hands free). Regions where it wasnt normal activated less. Soemthing about understanding the context didnt activate AON regions.

6. Is the AON a human MNS?

   • Evidence that these areas in the frontal and parietal cortex show an increased response for the observation and execution (and imitation) of actions, has been interpreted as evidence for a human ‘mirror neuron system’. But the fact that a volume of cortex has an increased BOLD signal during observation and execution of an action does not necessarily mean that the same neurons are active in both conditions, i.e., observed and executed actions might be subserved by different but spatially overlapping neural populations. So it could be a MNS for many reasions, but it is not possible to measure individual neurons for ethical reasons so this is hard to ever know. Can do brain imaging - does activity of these regionsreflect properties we woud expect if they contained mirror neurons?
   • => A need to investigate the representations of actions. Hoe the brain represents actions. Is there any evidence for crossmodal visual/motor representation of actions? With fMRI, 2 methods for probing neural representations: Repetition suppression/adaptation: If the magnitude of the BOLD response in a brain region elicited by a specific stimulus or stimulus property reduces (adapts) with repeated presentations => that neuronal population ‘codes for’ that stimulus/stimulus property. Some sort of neural fatigue - more you repeat, more tired the neurons get. Multivoxel pattern analysis (MVPA): a classifier attempts to distinguish between experimental conditions (e.g. observation vs. execution of actions) on the basis of spatial patterns of fMRI activity (across voxels). Similar idea. Algorithm classifies between 2 actions from data, tested on dif data, an it distinguish based on production?
     • Oosterhof et al. (2010) - Experiment 1: Participants observed or performed 3 simple intransitive, meaningless hand movements. Patterns of activity across voxels distinguished among the 3 different actions, across the visual and motor domains, in the lateral occipitotemporal cortex (bilaterally) and in the left postcentral gyrus/anterior parietal cortex. Trained pattern classifier, could it distinguish between 3 actions? Yes. Regions of brain where activity did distinguish was in areas of AON.
       • Experiment 2: Participants observed or performed 4 object-directed actions. Very similar results to Experiment 1. In addition, postcentral gyrus carried more information about the effectors used to carry out the action (fingers vs. whole hand), whereas superior/posterior parietal regions carried more information about the goal of the action (lift vs. punch). Nice data to show that distinction of which regions encode which information
     • Kilner et al. (2009) - fMRI adaptation effects for action observation and execution: when the observed actions are repeated (e.g. Hamilton and Grafton, 2006; Dinstein et al., 2007) and when the executed actions are repeated (Dinstein et al., 2007). But areas of cortex that contain mirror neurons should show cross-modal adaptation/repetition suppression, i.e., when an action is executed and subsequently observed, and when an action is observed and subsequently executed. In previous studies, expect to find if fully representing MNS we should find repetition and suppression effects when action is observed then executed etc. Cross modally, observation followed by execution.
       • Participants either executed or observed hand actions (ring pull or precision grip). In the critical trials, execution was followed by observation or vice versa, for same or different actions. In bilateral IFG, responses were suppressed both when an executed action was followed by the same vs. a different observed action and when an observed action was followed by the same vs. a different executed action. Found repetition suppression effects in two regions.
         • But Kilner et al.’s (2009) and some other findings limited to the case of actions observed from a first-person perspective (viewing actions of one’s own, or as if they were one’s own). A key proposed feature of the “mirror” system is the ability to generalize to actions observed from third-person perspectives. In observation, way they see hand is as if it was their hand viewed by their own eyes. Orientation of hand is same for obervation and viewing condition! Key thing - third personOosterhof et al. (2012a) addressed this … Did first vs third.
           • Oosterhof et al. (2012a) - From first person - could be own hand. Third - someone elses hand. MVPA - highlighted area for first and third. Test differences between first and third - in cross modal, first person were above chance. Inferior frontal gyrus region seems to not reflect true mirror neuron as doesnt reflect cross modal information about first person perspective. Results of MVPA in left PMv: action-specific cross-modal visual-motor representations for the 1st-person but not for 3rd-person perspective
     • But in such studies … … when participants observe actions might they also be imagining executing the action? … when participants execute (unseen) actions might they be imagining themselves seeing their own actions? Such imagery effects may result in unimodal action-specific representations that are similar both during performed and executed actions, and therefore would appear (incorrectly) to reflect cross-modal coding. Can mental imagery of specific actions explain the apparent cross-modal coding of visuo-motor representations? Spontaneous, non deliberate effect of imagination.
       • Oosterhof et al. (2012b) - Are there neural representations of specific actions that are common across visuo-motor performance and imagery? Ps performed and observed themselves performing two different manual actions during some trials, and imagined performing and observing themselves performing the same actions (with eyes closed) during other trials. Whole-brain group analysis showing classification z-scores for (a) unimodal perform, (b) unimodal imagery, and (c) cross-modal perform–imagery discrimination between lift and slap actions. “in the crucial analysis comparing patterns of activity produced during performing vs. imagining actions, we found that the left anterior parietal cortex represents specific actions in a manner shared across both conditions. These findings are the first evidence that neural coding of specific imagined actions is similar to overtly performed actions” “the left anterior parietal cortex represented specific actions similarly across the perform and imagery
         • Note that this evidence for cross-modal coding in anterior parietal cortex but not in other areas e.g. premotor and occipito‐temporal cortices does not imply that those other areas are not representing specific actions during imagery. For … … (1) statistical power to detect imagery effects in these areas may have been too weak … (2) the absence of an overt task in this study (other than execute & observe) may have affected the strength of imagery representations.

7. 3. Theoretical accounts of action simulation in the AON

   • Understanding the movements as actions involves: A translation from visual processing of kinematics and changes in posture/ configuration of features to … … a description of the short-term goals or intentions behind the action, via … … a mapping of the visual information onto the observer’s own motor system (e.g. Hamilton & Grafton, 2007; Iacoboni, 2005; Kilner et al., 2007)
     • The face that there might be mirror neurons in that region doesnt get us very far in how we explain how we understand others actions. Need much more going on to reflect how mirror neurons are a part. What might these brain regions be doing? Idea goes: mapping. One have motor system engaged, you much better understand the action.
   • Iacoboni, M. (2005). Neural mechanisms of imitation. Current Opinion in Neurobiology, 15(6), 632-637. - “The inverse model is a controller that retrieves the motor plan necessary to achieve a desired sensory state (or goal), whereas the forward model is a predictor of the sensory consequences of a motor plan. Thus, the inverse model is updated on the basis of the forward model. In neural terms, the input of the inverse model would be the STS output that is sent to the frontoparietal mirror neuron system, and the output of the inverse model would be the output of the frontoparietal mirror neuron system downstream to motor areas. Efference copies of motor commands originating from the frontoparietal mirror neuron system would provide the input of the forward model, whereas its output would be the matching process occurring in the STS [32,33] (Figure 1).” (p.634)
     • Parietal - goals action. Frontal - motor plans. STS - visual info. Forward model and inverse model. Froward: idea that whenm motor command is produced in frontal, copy used to regions that produce movement and copy sent to other regions as a prediction of the sensry consequences of he action, so can update/correct action. As you pick up objecct, can adjust the force of grip etc last min, By matching of motor command and sensory. This is used in reverse to gain infor about motor commans and used to produce viewed kinemaics of acion.
   • A proposed role of the ‘mirror-neuron system’/ ’action observation system’ in understanding the intentions of others: e.g. Kilner et al. (2007). Might this system also be involved in understanding others’ emotions? Instead of connections typically used, we do it in reverse and udnerstand a viewed action. Diff approach - predicting coding model. Idea that the brain is a prediction engine engaged in predicting whats out there. Frontal region predict what action might be, sendint to parietal, predictions about what goal might be, STS, comapring predictions and sending predicion errors back forward. They are refined by feed foreard and feed abck.
   • A note on predictive-coding (the favoured interpretation of the interactions between the regions of the ‘mirror-neuron system’/ ‘action observation system’). Feedback (‘top-down’) connections send to ‘lower’ (e.g. sensory) regions predictions about the encountered stimulus. Feedforward (‘bottom-up’) connections send prediction error signals about sensory information (i.e., information about the difference between the encountered and predicted sensory signals) from ‘lower’ regions back up to the higher brain areas implementing the predictions. The result of a multilevel cascade of such bidirectional interactions is a minimization of sensory prediction errors (i.e., of uncertainty about the causes of the sensory inputs) See e.g. Clark (2013), Kilner (2011), Kilner et al. (2007)
     • Kilner’s (2011) proposal of an extension to the predictive-coding account of the AON/MNS - Proposal that The intention and goal of an observed action are generated through a ventral pathway (red) from middle temporal gyrus/LOTC to anterior IFG. The classical AON/MNS regions (green: more posterior and dorsal IFG regions & inferior parietal regions) encode the motor representations Action understanding involves an interaction between these 2 pathways

Images and files in the map