L13 Reinforcement learning

L13 Reinforcement learning

1. 1. THEORETICAL APPROACH

   • a. What actions lead to positive and negative outcomes
     • i. An animal is rewarded for a correct outcome - it knows something has been done correctly
       • BUT it does not instantly know which action led to this outcome
         • Unproblematic: set of potential actions carried out were small
         • Problematic: large action set
     • ii. When a mistake is made, no information is provided about the nature of the error - jus that reinforcement doesn't occur
   • b. The Law Effect
     • Of several responses made to the same situation, those which are accompanied or closely followed by satisfaction to the animal will, other things being equal, be more firmly connected with the situation, so that, when it recurs, they will be more likely to recur i.e. coming back for more
     • Rescoria-Wagner Rule
       • Associative learning occurs not because two events co-occur, but because that co-occurrence is unanticipated on the basis of current associative strength

2. 2. BRAIN CIRCUITS UNDERLYING RL

   • a. Hypothetical circuit
   • b. Nigrostriatal system
   • b. Cortical input to the striatum

3. 3. DOPAMINE CELLS: NEURAL BASIS OF RL

   • a. Dopaminergic projections in the brain
   • c. Circuits of the nucleus accumbens

4. Enables a species to acquire complex behaviours by learning to obtain rewards and avoid punishments

   • 4. RL AND PREDICTION ERRORS
     • b. Prediction in dopaminergic cells
     • a. Background