Validity

Validity

1. Internal Validity

   • Factors Affecting Internal Validity
     • Selection Bias: Non-random factors responsible for participants being in one group or condition and not another
       • e.g., self selection bias, assignment bias
         • REMEDY: Random sampling or allocation
     • Regression to the Mean: Tendency for extreme scores on one occasion to be closer to the mean on another occasion
       • e.g., top vs bottom quartile groups getting more 'average' over time
         • REMEDY: Control group (with same characteristics)
     • Maturation: Changes within participants over an (usually) extended period of time
       • e.g., fatigue, practice, therapy
         • REMEDY: Control group
     • History: Concurrent events happening between pretest and post test
       • e.g., price rise during a social campaign to conserve electricity
         • REMEDY: Control over test environment, identifying confounds, control group
     • Testing: Aspects of testing (not hypothesised) that affects the participant's response
       • e.g., sham surgery
         • REMEDY: Equivalent control group, counterbalancing order of treatment
     • Mortality: Loss (attrition) of participants that is related to the treatment
       • e.g., death, loss of interest, avoidance (weight loss programs)
         • REMEDY: Control group, matched controls on factors related to subject attrition
     • Instrument Change: Changes in the instrument (or observer) over repeated testing
       • e.g., observer bias, assessor getting more experienced, changing scoring criteria
         • REMEDY: double-blind (participant, observer)
     • Experimenter Bias: Contamination of participant's response by experimenter
       • e.g., standardised testing of groups (case of Clever Hans)
         • REMEDY: triple-blind (participant, observer, tester)
   • High Internal Validity
     • Maximising hypothesised systematic variance (e.g., stronger manipulation)
     • Eliminating other possible systematic variance or balancing these effects between groups
     • Minimising error variance
   • Ensuring High Internal Validity
     • Control Group and Random Assignment
       • Selection bias, regression to the mean, history, maturation
     • Special Control
       • Testing
     • Co-variate pretest then yoke
       • Mortality
     • Double-blind Testing
       • Instrument change, experimenter bias

2. External Validity

   • Factors Affecting Internal Validity
     • Representative Sample of Participants: Random Selection is rare due to ethics and logistics
       • REMEDY: Adequate description of the sample, replication with other samples
     • Representative Sample of X's and Y's: Accurate and complete operationalisation of a concept (X,Y) is rare due to artificial and multi-factorial nature of X and Y, ethics and logistics
       • REMEDY: Adequate description of X's and Y's, replication with other operationalisations
     • Representative Sample of Situations: Behaviour is often dependent on context
       • REMEDY: Adequate description of test situation, replication with other situations