SURVEYS AND SAMPLING_ FINAL

SURVEYS AND SAMPLING_ FINAL

1. SURVEYS

   • WHAT ARE THEY?
     • A series of formatted questions delivered to a defined sample of people with the expectation of responses immediately or within a few days.
     • Very structured
     • Capture public opinion at a point in time
     • Use formatted (standardised) rather than open-ended questions
     • With appropriate sampling, we can generalize with a known statistical confidence from a sample to a wider population
   • PROS/CONS
     • PROS
       • Relatively low cost
       • Relatively fast delivery
       • Rapid data processing
       • Can research large populations
       • Multiple methods: phone, (e)mail, Internet, f2f
     • CONS
       • Generally limited to scaled or check-list questions
       • No control over response rate
       • Provides information more than understanding
       • Increasing public resistance
         • Not everyone is willing to do it
       • Difficult to explore issues in depth
         • with open ended questions you get a lot of info but with these types of questions, you don't
   • TYPES
     • Cross-sectional
       • Ask 1 group of people (respondents) the same questions at 1 point in time
       • “slice of time”
     • Longitudinal
       • Observations are taken more than once
       • examine differences/relationships over time
       • Not able to detect causal relationships
       • Takes into account influence of time
       • DIFFERENT TYPES
         • Trend
           • examines changes in a population across time
           • Each study collects data from different individuals
           • Not clear why change occurred because one is not collecting information from the same individuals over time
           • EXAMPLE:
             • In 2007, 1,000 people are asked for their support of same-sex marriage. In 2017, a different set of 1,000 people are asked for their support of same-sex marriage
               • The difference in opinions is the trend
         • Cohort
           • examines changes in a cohort across time
           • Each study collects data from different individuals within the same cohort
           • Provides more detail than trend study, but determining cause and effect is a problem because the same persons are not studied over time
         • Panel
           • examines changes in individuals across time
           • Each study collects data from the same individuals
           • EXAMPLE:
             • Study individuals who play violent video games to see how the increase of playing games increase the aggressiveness level
           • provide more in-depth analysis (because you study the same individuals)
           • But they are expensive
             • have to find a panel
           • Drop-outs can happen
           • more detail than cohort
         • Cross-lagged
           • best form of survey research design for ‘determining’ cause and effect
           • measure independent and dependent variable at several time points
           • need panel data
           • Need same exact survey
   • SUMMARY OF TYPES
     • Cross-sectional
     • Longitudinal
       • Trend
         • different people over time
       • Cohort
         • same type of people over time
       • Panel
         • same individuals over time
       • Cross-lagged panel
         • measures a dependent and an independent variable at two points in time
     • so... WHICH ONE?
       • cross-sectional: least expensive to implement, but cannot determine cause and effect
       • Longitudinal studies are more expensive to implement, but provide a better sense of cause and effect
         • *SIDE NOTE*: To determine causality you need a true experiment. However, panel + cross-lagged may provide you with some indication of causality.
   • RESEARCH VALIDITY
     • INTERNAL VALIDITY -
       • No alternative explanations for the causal conclusion
       • Is the researcher measuring the effect of the independent variable on the dependent variable?
     • External validity + (if we have a large and random sample)
       • Results are generalizable to the wider population
       • Units of analyses are representative of population
   • MODES
     • With interviewer
       • face-to-face
       • phone
     • Without interviewer
       • paper-and- pencil
       • e-mail/web
     • *check hand-written notes*
   • TYPES OF QUESTIONS
     • Dichotomous
       • 2 possible answers
     • Open ended
     • Multiple choice
       • check any
       • mutually exclusive
     • Rank order
     • Likert Scale
       • strongly agree - strongly disagree
     • Semantic differential scale
       • respondents mark their positions between word pairs
       • THE A BUILDING: UGLY - - - - - BEAUTIFUL
   • Problems with survey wording
     • Leading questions
       • How stupid is Donald Trump when it comes to immigration related issues?
     • Double-barreled questions
       • 2 questions
         • I felt welcomed by the lecturers and other staff on my first day at UvA
     • Double negative wording
       • Would you be for or against not allowing the sale of alcohol at campus?
   • PRE-TESTING
     • ask for feedback
     • Cognitive interviewing: use think-aloud method and ask respondents what they think when filling in questionnaire

2. SAMPLING

   • SAMPLING TERMS
     • Population
       • every person in the group you are studying
     • Sample
       • part of a population selected for study
     • Census
       • study of entire population
     • Parameters
       • describe a population
     • Statistics
       • describe a sample
     • Sampling frame
       • list from which a sample is drawn
   • 2 TYPES
     • Nonprobability sampling
       • Based on judgment by the researcher
         • Convenience
           • Based on convenience to researcher
         • Purposive / judgmental
           • Based on the idea that a specific person/media content will meet specific criteria
         • Quota
           • Specific characteristics are observed before so researcher can choose(=stratified)
         • Snowball/network
           • Rely on members of a network to introduce you to other members of that network
         • Volunteer
     • Probability sampling
       • Random selection. Every unit has an equal chance of being selected
         • Simple random
         • Stratified
           • groups-. randomly select
         • Systematic
         • Cluster
           • lots of data, different clusters (provinces and then people)
     • So... Which one?
       • Nonprobablity
         • + Convenience
         • - Sample does not represent the population being sampled
       • Probability
         • Sample represents the population being sampled
         • Time and resource consuming
   • Non-response
     • Unit non-response
       • Selected respondent does not participate in survey study (unreachable, speaks a different language....)
       • You need a larger sample
       • External validity lower
     • Item non-response
       • Item non-respons (Respondent does not understand a question, want to bc it's too intimate...)
       • External validity lower