Chapter 6

Chapter 6

1. schedules of reinforcement

   • often focused on perfect contingency but this is rare
   • a rule determining if a response will be followed by a rein forcer
   • schedules influence how a response is learned and maintained
     • examples: rat in skinner box, slot machine, sports
   • simple schedules
     • a single factor determined the occurrence of the reinforcer
     • fixed ratio (FR)
       • a fixed ratio between the # of responses necessary to produce the reinforcer
         • Ex 1: FR10: 10 lever presses = 1 food pellet
           • Ex 2: FR6: 6 passcode #'s = phone opens
       • continuous reinforcement (FR1)--> steady moderate responding
       • partial reinforcement (FR50)--> produces vigorous responding
       • FR characteristics
         • post reinforcement pause: decreased responding after reinforcer
         • ratio run: a high steady rate of responding that completes the ratio
         • ratio strain: rapid increase in FR requirement results in loing pre-reinforcement pauses
     • variable ratio (VR)
       • in a FR schedule, the subject "knows" how many responses are needed and VR a different number of responses are required for reinforcement
         • average number of responses needed
           • ex: VR10: criteria order 9,6,11,14
             • average number needed= 10 responses
       • characteristics of VR vs FR
         • fewer reinforcement pauses
           • fewer ratio runs
             • more resistance to ratio strain
         • something about VR schedule that takes the bumps out of the responding characteristics
           • if you are looking to produce high steady states of responding once they learn what to do switch from FR to Vr schedule because they will perform more responses for less reinforcement
     • interval schedule
       • Responses are reinforced if they occur after a certain amount of time
         • Fixed interval schedule: the time between reinforcers in constant
           • Variable interval schedule: the time between reinforcers is variable
             • waiting for the dealership to fix your car
           • Ex: washing clothes in a washing machine
     • fixed interval
       • responses cluster around reinforcer delivery (FI scallop)
         • Silly first years might wait to study until just before the exam
       • depend upon the ability to percieve time
         • fester and skinner (1957): visual stimuli increased scalloping
     • variable interval
       • VI schedules support steady, stable rates
       • onces time has past, the response will be reinforced (2 min or 90 min)
       • limited hold: in some instances, a restriction can be placed on the length of time a reinforcer will be available
   • ratio schedules
     • reinforcement depends upon the number of responses performed
       • continuous reinforcement
         • every time the response occurs, so does the reinforcer
       • partial reinforcement
         • response is reinforced only some of the time
   • ratio vs interval schedules
     • Inter response time (IRT): interval between responses
       • if short IRTs are reinforced, increases responding
       • if long IRTs are reinforced, decreased responding
     • ratio schedules depend upon response accumulation, the fater the criterion the more reinforcement
     • interval schedules provide no advantage for increased responding until around the time of delivery
   • response rate schedules
     • requires a certain number of responses at a specific rate
     • example: assembly line work
       • work too slow --> slow down line--> fired
       • work too fast-->piss off others--> greased gloves
       • work just right--> team player--> paycheck
     • differential reinforcement of high rates (DRH)
       • responses are reinforced only if accumulated before a given time
         • encourages a high rate of responding
           • ex: at DRH12 a rat must press lever more or equal to 12 times/min
     • differential reinforcement of low rates (DRL)
       • responses are reinforced only if restricted in a given time
         • encourages a low rate of responding
           • ex: at DRL3 a pigeon can peck less than or equal to 3 times/min for reinforcement

2. choice behaviors

   • concurrent schedules
     • often we choose between response reinforcer pairs based on their consequences
     • common technique for studying choice
       • skinner box
       • concurrent schedules: 2 available reinforcement
         • mr. pigeon has a choice
           • key A V160
           • key B FR10
           • he may choose both
   • measuring choice
     • relative rate of responding
       • RRA for Key A = RA/(RA + RB)
         • RRB for key B = RB/(RA+RB)
           • RA= responses on key A
             • RB= responses to key B
               • RRA > 0.5 means RA is less than RB
               • RRB = 0.5 means that RA and RB are equal
     • choice is influenced by schedule of reinforcement
       • if 2 schedules = FR5, pigeons are predicted to spend equal time at each key given equal reinforcement
         • Relative rate of reinforcement (rr) can be calculated like RR
           • relative rate or reinforcement (rr)
             • rrA for Key A= rA/(rA+rB)
               • rA= responses on key A
             • rrB for Key B= rB/(rA+rB)
               • rB= responses on key B
   • Matching Law
     • the relative rate of responding (RR) on a given alternative is approximately equal ot the relative rate or reinforcement (rr) earned in that alternative
       • RRa/RRb=rra/rrb
         • the relative rates or responding Match relative rates or reinforcement
     • considerations
       • sensitivity: tendency to choose a particular schedule despite loss of reinforcment
         • for choice a: Ra/Rb=(ra/rb)^s
         • ex: monkeys have choice if cocaine on VI3 or VI6
           • undermatching: choice responding less than predicted, s<1.0 M. law predicted 2:1 ration, if choice is <2:1 (ex: 1:1)
           • overmatching: choice responding more than predicted, s>1.0M. law predicts 2:1 ration, if choice > 2:! (ex: 3:1)
       • bias: tendencies to certain responses and/or reinforcers
         • b>1.0 more preferred, b<1.0, a less preferred
           • Ra/Rb= b(ra/rb)^s
       • reinforcer value: reinforcer features influence rate of responding (R)
         • ex1: amount
           • ex 2: palatability
             • ex 3: immediacy
               • Ra/Rb=b(ra/rb)^s
     • applications
     • applications
       • Vollmer & Bourret (2000)
         • in basketball players choose
           • RRa: (3 pointer)= further but more points (rra)
             • RRb: (2 pointer = easier but less points (rrb)
         • shots (rr) were proportional to the shooting % (rr) of those shots
           • I.e. good long range shooting teams took more 3 pointers
         • similar in football (reed et al. 2006) with rushing vs. passing
     • mechanisms
       • study of choice suggests we maximize levels of reinforcement
         • levels or choice
           • molecular: individual responses
           • melioration: local rates or responding
           • molar: sum of responses
           • Molecular -->melioration--> molar
     • molecular mechanisms
       • molecular maximizing: choosing the response that is best at a single point
         • molar maximizing: choosing the response that will maximize reinforcement over the long run
           • Labratory examples :
             • what key light does a pigeon choose in a single instant: molecular maximizing
               • how many lever presses does a rat make on 2 levers over 3 days?: molar maximizing
     • melioration
       • we respond to improve local rates of reinforcement
       • local rate: time a subject responds to a particular alternative
         • ex: lever pressing 60 times in 60 min, but all occur in the first 30 mins
           • overall rate= 1/min but local rate (in 1st 30 mins)=2/min
         • ex: choosing pricier beer during reduced cost 7-8pm happy hour
           • overall beer tree super pacific=1/hr, local super pacific= 3/hr
   • choice commitment and self control
     • choices are often available ad lib, however, some decisions make other choices unavailable
       • concurrent chain schedule: testing choice and self control
         • choice link: A or B? not reinforced
         • participant is permitted to choose during the first link which of several simple reinforcement schedules will be in effect in the second link
           • once a choice has been made, the rejected alternatives become unavailable until the start of the next trial
         • self control: choosing a large/larger delayed reward over an immediate small reward
           • self control is easier if the tempter is not readily available
             • ex: marshmallow test
           • direct choice: pigeons prefer small immediate reward over large delayed reward
           • concurrent chain: with chain delays, pigeons choose the large delayed rewards
             • consistent across species
           • can we quantify self control?
             • value discounting function: the value of the reinforcer is reduced by how long you have to wait for it
               • mathematically V=M/(1+KD)
                 • V= value of reinforcer
                 • M= reward magnitude
                 • K= decay parameter
                 • D= reward delay
               • as delay increases, the value of reward decreases
                 • ex: if the delay= 0, V=M
               • consequences of VDF
                 • V=M/(1+KD)
                   • as a reward value decays over time (KD) choice is shifted
                     • T0 (onset): the reward value for "large" is greater, no decay
                       • T1 (early): immediate small reward preferred if large reward value decays with delay, like "direct choice":
                         • T2( late): in long delays, large reward retains value and is preferred like "concurrent chain"
                           • we display inherent or developed lack of self control
           • choice commitment in self control
             • in VDF (V=M/(1+KD)
               • K is the decay function
                 • small , shallow function= more self control
                   • large K, steep function= less self control
             • Madden et al. 1997: "stay off the horse"
               • subjects: controls and heroin users
                 • choice between: up to $1000 with delays (1 week to 25 years) or users choose between smaller amounts right away
                   • K large decay steep in heroin users ( low self control
           • delay discounting: decrease in the value of a reinforcer as a function of how long one had to wait to obtain it
       • green (1972): self control procedures
         • direct choice procedure pigeon chooses immediate, small reward
         • concurrent-chain procedure pigeon chooses the schedule with the delayed large reward
         • A=VR10, B=FR10 --> 10 min in terminal link
           • terminal link: reinforced
             • A leads to VR10
               • B leads to FR 10
           • subject is committed to a schedule in the terminal link
             • they would prefer A (VR) because there is a tendency to like variation