PSYC 318-Lecture#4(II)

Pascale Bockelmann
Mind Map by Pascale Bockelmann, updated more than 1 year ago
Pascale Bockelmann
Created by Pascale Bockelmann over 3 years ago
1
0

Description

Psychology Mind Map on PSYC 318-Lecture#4(II), created by Pascale Bockelmann on 01/19/2017.

Resource summary

PSYC 318-Lecture#4(II)
1 learning
1.1 Stimulus-stimulus learning
1.1.1 learn associations between diff. stimuli
1.1.1.1 not rely on dopamine reinforcement signals.
1.1.1.2 only requires one trial
1.1.1.3 observations + seeing patterns
1.1.2 could theoretically be used to control movements
1.1.2.1 but this is not how motor learning works: b/c the brain doesn't know how it produced a certain type of neural activity.
1.2 Procedural Learning
1.2.1 reinforcement mechanisms.
1.2.1.1 required for motor learning
1.2.1.1.1 Most of the neural activity that is produced by our brain trying to influence behaviour is thought to be blocked or b/c the neural activity is not strong enough to get through all the filters in the brain.
1.2.1.1.1.1 Anytime there is movement, some neural activity was successful in getting through the action selection filter.
1.2.1.2 every moment, the brain is assessing if things are getting better or worse for the animal.
1.2.1.2.1 this decision will result in increases or decreases in a reinforcement signal in the brain, which acts to strengthen or weaken recent neural activity, such that it becomes more or less likely that that neural activity will make it past the action selection filter next time a similar context arises
1.2.1.2.1.1 Most movement-related neural activity is probably randomly generated
1.2.2 Predicted Error Signal:
1.2.2.1 signifies: unexpected change in your current
1.2.2.1.1 signals brain that credit or blame should be appointed.
1.2.2.1.1.1 credit or blame should be used to adjust your value estimates of things (cost-benefit)
1.2.2.1.1.1.1 your probability of repeating that behaviour in the future.
1.2.3 Predicted Error Signal Formula:
1.2.3.1 Predicted Error Signal = actual – expected value of current situation
1.2.3.2 Fluctuations in the reinforcement signal
1.2.3.2.1 make it more or less likely that your brain will generate the same neural activity in similar situation
1.2.4 we learn through multiple trial and error
1.2.4.1 i.e. how to move our muscles
1.2.4.2 i.e. CPP
1.2.4.2.1 unclear if this is SS or PL
1.2.5 Process
1.2.5.1 Step 1:  animals envision what movement they want to make
1.2.5.2 Step 2: animals  reinforce any neural activity that gets them closer to achieving that goal
1.2.5.3 you don’t need to have a goal for reinforcement learning to improve your life Any randomly generated neural activity that seems to improve your situation in life in any manner, that neural activity would be reinforced
2 Where in the brain do actions get reinforced?
2.1 Dopaminergic Projections of the Rat Brain
2.1.1
2.2 The Dopamine System
2.2.1 Dopamine as a Prediction Error Signal
2.2.1.1 D. released
2.2.1.1.1 estimation of the current moment is better than you anticipated it to be
2.2.1.2 D. withheld
2.2.1.2.1 estimation of the current moment is worse  than you anticipated it to be
2.2.1.3 Dopamine signalling will strengthen recently active glutamatergic synapses
2.2.1.3.1 motor commands G.S. encode become more likely to win
2.2.1.4 As your expectations grow, the dopamine system becomes more and more selective
2.2.2 very few dopamine neurons
2.2.2.1 large
2.2.2.2 sends many projections
2.2.2.3 homogenous group of neurons; all fire at the same time
2.2.2.3.1 excluding PFC projecting neurones
2.2.2.4 unmyelinated & can’t fire v. fast (0-40 Hz)
2.2.2.5 dopamine
2.2.2.5.1 Cleared from extracellular space 100 times slower ( compared to glutamate or GABA )
2.3 All motor commands sent to the NAc (striatum)
2.3.1 this is the input nucleus of the basal ganglia
2.3.1.1 information is encoded in excitatory, glutamatergic inputs from the cortex
2.3.1.1.1 striatum also receives dopamine
2.3.1.1.1.1 Glutamate inputs
2.3.1.1.1.1.1 what gets reinforced
2.3.1.1.1.1.1.1 (actual information, movements, decision = motor commands)
2.3.1.1.1.2 Dopamine inputs
2.3.1.1.1.2.1 reinforcement signals
3 Three-Factor Rule
3.1 DEF
3.1.1 neurons that fire together become eligible for dopamine-induced synaptic plasticity.
3.1.2 changes in dopamine levels
3.1.2.1 if dopamine is at baseline (there is no learning that occurred)
3.1.2.2 strengthen or weaken recently active glutamatergic synapses in the striatum.
3.1.2.3 strength of glutamatergic inputs in the striatum can change when the synapses experience:
3.1.2.3.1 Postsynaptic activity
3.1.2.3.2 Pre-synaptic activity
3.1.2.3.3 Abrupt changes in local dopamine receptor activity
4 Dopamine Neurons Encode a Prediction Error
4.1 Full Experiment
4.1.1 guaranteed reward: 100% percent predictive by stimulus, dopamine neurones will fire when stimulus is present, but won’t fire when the reward is presente
4.1.2 unexpected reward: produces the same amount of dopamine neurone activation, as the amount of firing that occurs when they receive a stimulus that tells them they are guaranteed to get a reward.
4.1.3 dopamine neurone activity is reduced (response is muted)
4.1.3.1 dopamine response occurs when the animal first realizes the session will start
4.2 Test: Electrophysiological recordings of dopamine neurons in a monkey when it unexpectedly receives food.
4.2.1 raster plot graph
4.2.1.1 each row is a 2 second long interval
4.2.1.1.1 Halfway into each trial, the monkey unexpectedly gets a juice reward.
4.2.1.1.1.1 Each dopamine neuron fires about 5-10 times across each 1 second long interval
4.2.1.1.2 first trial = top row, last trial = bottom row
4.2.1.2 Black dot = when a midbrain dopamine neuron fired an action potential
4.2.2 summary histogram
4.2.2.1 summary of how many dopamine neurons fire at any point in time, but the data is summed across 50 trials
4.2.2.2 'collapsed raster plots'
4.2.3 50% prediction = you get half amount of dopamine activity that you would if it was a 100% unexpected reward
4.2.3.1 will fire to stimulus and again when they receive reward: these two events sums up the amount of dopamine activity that occurs when dopamine is guaranteed or unexpected
4.2.4 guaranteed reward
4.2.4.1 firing of the dopamine neurones won’t change
4.2.5 The amount that dopamine neurones fire upon stimulus presentation depends on how well the stimulus predicts reward
4.2.6 25% prediction
4.2.6.1 if the animal can sense that there is a 25% chance it will receive a reward, the value to the animal is 25% of the kool aid, and then the remaining 75% value of the kool aid is delivery once it receives the reward.
5 Expectation of Reward Affect Learning
5.1
6 Demonstration from Blocking
6.1 i.e. tone, light sugar water
7 Expectation of Reward Affect Dopamine neurone Activity
7.1 when a reward-predictive stimulus is only presented in conjunction with another stimulus, that has already been learned to be fully predictive of reward, then learning to the second stimulus is blocked
7.1.1 2nd stimulus is a redundant cue
8 How does dopamine discriminate between cues when it reinforces neural activity?
9 Second Order Conditioning
9.1 15th dopamine
9.1.1 only fire to the cue, not the reward
9.2 5th dopamine
9.2.1 only fire a little to the reward
9.3 you can see the dopamine signalling moves back further and further to earliest predictor, to the first time the animal has changes in expectations, the first time the animal can predict presentation of a reward
9.4 people think different types of learning have diff. time window involved
10 Pleasure Versus Prediction Error
11 Dopamine has Two Distinct Functions
12 Phasic versus tonic dopamine signalling
12.1 phasic dopamine doesn’t correlate well with perceived pleasure
12.1.1 b/c pleasure occurs in a situation dependent manner
12.1.2 unexpected dopamine events always increase phasic dopamine signals
12.2 dopamine, is a teaching signal that notifies the brain as soon as your anticipated value changes.
12.3 timing and amount of pleasure of rewarding events was largely anticipated, phasic dopamine signals will not change during the event itself and little to no learning will occur.
12.4 test: if we opto-genetically stimulate dopamine neurones while they are drinking the reward (sugar water) this should cause animals to learn about the light stimulus
12.5 stimulated dopamine neurones to make the animal think that something better than expected happened
12.5.1 similar to drugs
13 Messing with Tonic Dopamine Signalling
13.1 artificially inc. dopamine receptor
13.1.1 seem to be more engaged with their environment
13.1.2 are more willing to take risks and do hard things to get rewards
13.1.3 seem more motivated
13.2 if you artificially dec. dopamine receptor
13.2.1 seem less motivated
13.2.2 seem to be less engaged with their environment
13.2.3 seem less motivated
13.3 if you lose all dopamine receptor activity (severe Parkinson’s) you can’t initiate purposeful movement.
14 How does dopamine discriminate between cues when it reinforces neural activity?
14.1 dopamine neurones are needed to learn about antecedent cues
14.1.1 when the stimulus is presented, dopamine neurones fire, that is the moment when the value of the world changes
14.1.1.1 However, after this moment, there was no change in expectation, therefore, no change in learning needs to occur
15 Negative Prediction Error
15.1 current situation is worse than anticipated.
15.2 Dopamine neurons abruptly stop firing when an expected reward is not received
15.2.1 dopamine firing b/c they withhold the same amount of dopamine that was released previously
15.2.1.1 amount released previously to predictor was the same amount as what they believe the value of the reward be
16 Dopamine has 2 distinct forms
16.1 phasic dopamine signals regulate reinforcement learning by encoding a feedback signal ( a prediction error signal)
16.1.1 Abrupt changes in Phasic dopamine signals drive learning depends on how much base line activity is there
16.1.1.1 a) per neurone
16.1.1.2 c) ‘amount of phasic activity’
16.1.1.3 b) across all neurones
16.1.1.3.1 all three can causes changes in resting rate of dopamine
16.1.2 “high gain setting”
16.1.2.1 high background levels of dopamine
16.1.2.1.1 in extracellularl striatum
16.1.2.2 easy to get actions through the filers, willingness to exert effort
16.1.2.3 gain setting determines " how excitable neurones are to glutamate inputs"
16.1.2.3.1 a) how excitable neurons are within the striatum
16.1.2.3.2 b) how responsive they are to glutamate inputs
16.1.3 low gain setting
16.1.3.1 unmotivated to do anything
16.1.4 learning= phasic
16.1.5 tonic= motivational
16.1.5.1 general dopamine neurons fire at ~4Hz
16.1.5.2 slight changes in a) b) c) can cause large changes in the resting (tonic) amount of extracellular dopamine levels.
16.1.5.3 speed of this tonic activity
16.1.5.4 number of dopamine neurones that are participating (firing)
16.1.5.5 amount of phasic activity
16.1.5.6 influence the ‘gain’ settings in the system
16.1.5.7 willingness to exert effort: correspond to the animals estimate of overall value of current situation
16.2 tonic, baseline dopamine levels regulate motivational state
16.3 pre synaptic activity” glutamate input into the basal ganglia
16.3.1 when that occurs there is a potential to change the strength of that synaptic reaction, strength of synapse dec. when dopamine signals dec. (and vice versa)
16.4 dopamine neurones usually fire around 4 Hz,
16.4.1 firing rate corresponds to animals motivation and effort)
16.4.1.1 dopamine is cleared from the extracellular space slowly
16.4.1.1.1 there will be more dopamine in the system if a neurones ifs firing at 6Hz than if neurone is firing at 2 Hz
Show full summary Hide full summary

Similar

History of Psychology
mia.rigby
Biological Psychology - Stress
Gurdev Manchanda
Psychology A1
Ellie Hughes
Psychology subject map
Jake Pickup
Memory Key words
Sammy :P
Psychology | Unit 4 | Addiction - Explanations
showmestarlight
Bowlby's Theory of Attachment
Jessica Phillips
The Biological Approach to Psychology
Gabby Wood
Cognitive Psychology - Capacity and encoding
Tess W
Chapter 5: Short-term and Working Memory
krupa8711
Psychology and the MCAT
Sarah Egan