Sampling Bias and Nonprobability

Research Bias

• MUST ELIMINATE BIAS
• Only without bias can researchers be confident in their findings
• Bias occurs when respondents are not selected through true random sampling

Online Panels - guidelines

• invitation preferred over "opt-in" - but it generates a low response rate
• professional respondents are undesirable
• verify your respondents are real
• Panel should represent US population
• Monitor Response Rates - low response rates means people are turning away
• Poor panelists removed (the people who press the same button or answer crazy)

Nonprobability Sampling

• Convenience
• Judgement
• Quota
• Snowball
• Not good for generalization, only for quick, preliminary results

Convenience Sampling

• respondents are selected because they are convenient and accessible.
• will most likely provide biased and unreliable information.
• should only be used when there is no need to generalize to the broader population
• appropriate for exploratory research or quick information such as questionnaire pretesting
• times and conditions can change a sample DRAMATICALLY

Judgement Sampling

• respondents selected based on an expert's decision on who is best to interview.
• the greater the expertise, the greater the confidence in the results of the research (directly related)
• only recommended when the is immense confidence in the expert opinion or only when preliminary, exploratory information is needed
• Approaches include:
• extreme cases
• typical cases
• intense or passionate cases

Quota Sampling

• Extension of judgement sampling
• attempts to represent demographics of the sample equal to that of the broader target population
• obtained in 5 steps:
• Determine the defining characteristics of the key subgroups
• Determine the percent of the total population represented by each defining charateristic
• Determine the percent of the total population represented by each quota cell.
• translate the percent into a sample size
• Sample population

Snowball Sampling

• Uses current participants to recruit new participants
• used for populations where access is through personal introductions
• subject to numerous biases
• can be caused by a reposting or forwarding of surveys by a current participant
• Can be inaccurate because you have no idea who is answering the survey