GoConqr has a new look! Let us know what you think.

Sign up free

Flashcards

Research Planning and Design

Hannah Luke Psychology English 140 cards

Cards in this deck (140)

  1. 1Science is the use of ___ ___ to gain knowledgesystematic observation
  2. 2Observation results in ___ hypotheses and theoriestestable
  3. 3Science as a ___ __ ___ process.collaborative and democratic
  4. 4Science is ____.cumulative
  5. 5Define Inductive ReasoningDrawing general conclusions from specific observations
  6. 6Define Deductive ReasoningDrawing conclusions from general principles and rules to specific instances
  7. 7Dirk SmeestersData massaging and data collection inaccuracies
    Retraction of 7 papers publishes in high-ranking journals
  8. 8Diederik StapelData Fabrication
    19 Ph.D theses based on fabricated data
    Retraction of 30 papers published in high-ranking journals
  9. 9List some Questionable Research PratisesNot reporting all dependant measures
    Not reporting all study conditions
    Continuing/stopping data collection after peeking into results
    Rounding off a p-value (e.g., reporting p = 0.054 as less than .05)
    Selective reporting of studies with desired outcome
    Excluding data after checking the impact on results
    Reporting unexpected findings as having been predicted
    Claiming results are unaffected by demographic variables
    Falsifying data
  10. 10Define ReproducibilityTake the original data and analysis code to reproduce results
  11. 11Define ReplicabilityRe-run the study/experiment and it results in similar findings
  12. 12Define Close/Exact ReplicationsReplicate the study process in order to produce the same results
  13. 13Define Conceptual ReplicationsChange one or two variables to provide similar results
  14. 14List reasons studies don't replicateSubtle differences between studies such as boundary conditions
    False positives
    Publication bias
    File drawer problem
  15. 15List ways to incentivise best practisePublish results based on strong rationale and methods
    Pre-registration
    Open science
    Registered Reports
  16. 16Define VariableA condition or characteristic that can take on different categories, levels or values
  17. 17Define Psychological Construct"something that exists theoretically, but is not directly observable" (Vogt, 2011)
  18. 18Define OperationalisingTurning a construct into an operational variable. A description of the way researchers will manipulate or measure a variable.
  19. 19What are the variable measurement scales?Nominal
    Ordinal
    Interval
    Ration
  20. 20Define Nominal measurement scalesNominal scales are naming scales. Represent categories where there is no basis for ordering the categories.
    Gender
    Political party affiliation
  21. 21Define Ordinal measurement scalesCategories that can be ordered along a pre-established dimension. Simple ranking - without information of the size of difference between categories. We should not average numbers on ordinal scales.
    Olympic medals
    Sports rankings
  22. 22Define Interval measurement scalesAssigning scores using numerical scales. The difference between the successive numbers is equal. No true and meaningful zero point.
    Temperature
    IQ
  23. 23Define Ration measurement scalesHas a true and meaningful zero point. Represents a complete absence of quantity.
    Height
    Weight
    Number of siblings
  24. 24Define ReliabilityConsistence/stability of a measure.
  25. 25A measure is reliable if it...consistently yields the same results
  26. 26Sources of measurement errorPoor instrument
    User mistakes
    Random chance (variance due to all/any other variables)
  27. 27Ways to assess ReliabilityTest-Retest Reliability
    Parallel Forms of Reliability
    Internal Consistency
    Split-half Reliability (split-half correlation)
    Interrater Reliability
  28. 28Define Test-Retest ReliabilityGive someone the same test twice. If the test is reliable should get strong correlations
  29. 29Problems with Test-Retest ReliabilityAct of measuring can influence later measurement, Practise effects and Fatigue effects
  30. 30Define Parallel Forms ReliabilityGive someone two versions (forms) of the test
  31. 31Problems with Parallel Forms ReliabillityFatigue effects possible
  32. 32Define Internal ConsistencyConsistency of responses across items on a measure
  33. 33Assumptions of Internal ConsistencyItems represent the same underlying psychological construct
  34. 34Define Split-half ReliabilitySplit the items in the measure randomly into two halves. Calculate the scores for each half and check the relationship between the two halves.
  35. 35Define Interrater ReliabilityAgreement amongst raters observing a behaviour or coding data. Measured using Cronbach's alpha (quantitative) or Cohen's kappa (categorical)
  36. 36Examples of Interrater ReliabilityRate aggressiveness of play behaviour (on likert-scale)
    Categorise behaviour into pre-determined categories
    Code number of idea unites recalled in a test
  37. 37Define ValidityThe measure represents the variable it intends to measure
  38. 38List the types of result validityFace Validity
    Content Validity
    Criterion Validity
    Convergent Validity
    Discriminant Validity
    Predictive Validity
  39. 39Define Face ValidityDoes the measure make intuitive sense?
    Look at items of a questionnaire supposed to measure 'happiness', you would expect certain items
  40. 40Disadvantages of Face ValidityWeak type of evidence for the validity measurement. Trusting people's intuitions is not a good idea.
  41. 41Define Content ValidityDoes the measure reflect the conceptual definition of the construct?
    Content Validity is not quantitatively measured, but established by working with the conceptual definitions and theories in the literature.
  42. 42Define Criterion ValidityAre people's scores on a measure predictable related to other variables (to criteria)/to a relevant behavioural outcome?
  43. 43Define Concurrent ValidityCriterion and construct are measure simulatenously
  44. 44Define Predictive ValidityCriterion is measured in the future after construct has been measure (measure predicts future outcome)
  45. 45Define Convergent ValidityDifferent measure of the same constrict should be related to each other
  46. 46What are the main goals of experiments?To establish a casual relationship between variables. To demonstrate how changes in one variable leads to a change in another variable.
  47. 47List the Fundamental Features of an ExperimentSystematic manipulation of independant variables
    Control of extraneous variables
    Randomisation
  48. 48List the Four Validities in ExperimentsConstruct Validity
    Internal Validity
    External Validity
    Statistical Validity
  49. 49Define Internal ValidityAre casual conclusions between the IV and DV warranted? Can the difference between conditions in the dependant variable be completely attributed to the manipulation?
  50. 50Define External ValidityCan conclusions from experiment be generalised to a larger population and to other situations?
  51. 51Define Mundane realism in External ValidityRun experiment in 'real-world' situations or simulate close to 'real-world' setup and collect data from participants you want to generalise to.
  52. 52Define Psychological realism in External ValidityRun experiment that taps on the same psychological processes; that can be generalised to a broader population and other situations.
  53. 53Define Statistical ValidityDoes the data support the conclusions? Are we using the appropriate statistical procedures? Is there statistical significance? What is the size of the effect?
    Do we have a large enough sample to detect the size of effect we are expecting to find? Do we have enough power to detect an effect we are looking for given the sample size we are using?
  54. 54Define Confounding VariablesA variable that poses an alternative explanation for the measured difference in the dependant variable between different conditions.
  55. 55List the four main types of quantitative designs for gathering dataObservational
    Correlational
    Quasi-experimental
    Experimental
  56. 56Define MethodologyFrames how you will investigate psychological phenomena
  57. 57Examples of methodological approaches to data gatheringNeuroimaging methods
    Intervention studies
    Questionnaires
    Case Studies
  58. 58Define Intervention StudiesMake a change and measure the outcome
  59. 59Define QuestionnairesAsks a series of questions to measure psychological construct
  60. 60Define Case StudyExamines in detail a person or event
  61. 61List Relationships we can researchCorrelational and Observational Research
    Quasi-experimental Research
    Naturally observed variables
    Exploratory
    Leads to predictions
  62. 62List Casual Relationships we can researchCause and Effect relationships
    Confirmatory
    Experiments
  63. 63The research question needs to be...justified.
  64. 64Define ConstructsA concept or idea defined by psychologists
  65. 65We measure ___ NOT ___ .Variables
    Constructs
  66. 66List the types of Observational StudyNaturalistic Observation
    Observation with intervention
    Participant Observation
    Survey
  67. 67Define Naturalistic Observation StudiesStand back from the natural situation/behaviour and observe it as is.
    Keep as much distance between the researchers and observed as possible.
  68. 68Define Observation with Intervention studiesThe researcher is actively involved in the thing they are observing. A situation in a lab through a one-way mirror.
  69. 69Define Participant Observation StudiesResearcher actually joins in with a group or activity. Living in a society to study them.
  70. 70Define Survey StudiesDirect way to discover people's thoughts, opinions and feelings. Can be quantitative or qualitative.
  71. 71Define Association StudiesCan be observational or experimental. Can be quasi-experimental or full controlled experiment. Will have an IV and DV. Will be frequency data.
  72. 72Advantages of Association StudiesVery useful for categorical data which represents a lot of what we find in the world.
  73. 73Define the Alternative HypothesisStates an actual effect of the manipulated variable. Sometimes referred to as the Alternative or Research Hypothesis
  74. 74Define the Null HypothesisThe logical opposite of the alternative hypothesis. Always predicts no relationship between the variables in your study.
  75. 75Define a Two-tailed HypothesisNo prediction for the direction of the results
  76. 76Define a One-tailed HypothesisStates the direction of the effect expected.
  77. 77Disadvantages of One-tailed HypothesesOne-tailed hypotheses are very restrictive, it is best to avoid them unless you are making a confirmatory prediction and will not look at the second tail.
  78. 78Define a Simple Regression HypothesisModels the relationship between two variables by fitting a line to the data. The regression equation represents how much Y changes with any given change of X. Can be used to construct a regression line of a scatter disgram.
  79. 79Define a Multiple Regression HypothesisAttempt to determine the predictive power of more than one independent variable one dependent/outcome variable, and describe the direction and magnitude of the effect of each predictor on the outcome variable.
  80. 80Examples of Bias in researchConfirmation bias
    Cherry picking
    Biased thinking
    Survivorship bias
    HARKing
    Publication Bias
  81. 81Define Confirmation BiasThe tendency to interpret new evidence as confirmation of one's existing beliefs or theories. Biased information searches, biased interpretation or biased memory.
  82. 82Define Cherry PickingFallacy of incomplete evidence or evidence suppression
  83. 83Define Biased ThinkingDoing research involving many cognitive and social processes.
  84. 84Define Survivorship BiasSurvivorship bias or survival bias is the logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility.
  85. 85Define HARKingHypothesizing after the results are known, or HARKing, occurs when researchers check their research results and then add or remove hypotheses on the basis of those results without acknowledging this process in their research report
  86. 86Define Publication BiasIt occurs when the outcome of an experiment or research study influences the decision whether to publish or otherwise distribute it.
  87. 87What is Pre-registration?A time-stamped, read-only version of your research plan created before you begin data collection
  88. 88What does Pre-registration containHypothesis
    Data collection procedures
    Manipulated and Measured variables
    Statistical Model
    Inference criteria
  89. 89What problems does pre-registration fix?The file drawer effect
    P-Hacking
    HARKing
    Makes the distinction between confirmatory (hypothesis testing) and exploratory (hypothesis generating) research more clear.
  90. 90Define Exploratory ResearchGenerate a posteriori hypotheses
    Deals with unknowns
    Discovers new knowledge
    Less stringent methodological restrictions
    P-values meaningless
  91. 91Define Confirmatory ResearchTests a prior hypothesis
    Deals with knowns and unknowns
    Based on existing knowledge/research
    Very stringent methodological restrictions
    P-values interpretable
  92. 92What is the context for discovery in Exploratory research?Pushes knowledge into new areas/data-led discovery
    Finds unexpected relationships
    Goal is to minimise false negatives
  93. 93What is the context for confirmation in Confirmatory research?Traditional hypothesis testing
    Results held to the highest standards of rigour
    Goal is to minimise false positives
  94. 94What is the difference between population and sample?Population is the group you want to know about
    Sample is the individuals you actually study
  95. 95What is the aim of sampling techniques?To recruit a sample that is representative of target population
  96. 96Define Random SamplingRandoming choosing people from a population will create an unbiased sample
  97. 97Define Convenience SamplingTaken from some available subgroup in the population.
  98. 98Define Stratified random samplinga population which can be partitioned into subpopulations.
  99. 99Define Cluster Random SamplingRandomly selecting a cluster of participants i.e one class
  100. 100Define Quota Samplinga method for selecting survey participants that is a non-probabilistic version of stratified sampling
  101. 101What is the WEIRD Sampling (Henrich et al. 2010)Western
    Educated
    Industrialised
    Rich
    Democratic
  102. 102Disadvantages of the WEIRD sampleThis is a narrow database.
    Researchers often assume findings are universal
    WEIRD people are not representative of the population
  103. 103How to move beyond WEIRD samples?Global networks of labs to coordinate data collection for democratically selected studies.
  104. 104What are potential sources of error?Individual differences between people
    Environmental and Lab conditions
    Performance varies across time
    Measurement (unreliability)
  105. 105Types of Confounding VariablesPerson/Participant Centred
    Procedural Confounds
    Operational Confound
  106. 106Participant factors that affect Internal ValidityHistory
    Maturation
    Testing
    Instrumentation
    Biased selection of participants
    Experimental mortality
    Regression to the mean
  107. 107How to reduce error at the participant levelTreating all participants as similarly as possible
    Use participants with similar characteristics
  108. 108Procedural ConfoundsExperimenter Expectancy effects
    Practise Effects
  109. 109What are Reactive Effects?Reactive Testing
    Interactions between participant selection bias and IV
    Reactive effects of experimental arrangements
    Multiple treatment interference
  110. 110Define Reactive testingPre-test affects participants' reactions to a variable - responses are unrepresentative
  111. 111Define Multiple treatment interferenceExposure to multiple experimental treatments in which early treatments affects responses to later treatments
  112. 112Define Interactions between participant selection biases and IVEffects observed may apply only to the participants included in the study
  113. 113Define Reactive effects of experimental arrangementsHighly artificial experimental situations used in some research. The participant's knowledge that he or she is a research participant.
  114. 114Solutions to practise effectsStandardised Instructions
    Control Groups/Conditions
    Repeated Measures Design
    Random assignment to conditions and Random ordering of conditions/stimuli
    Counter balancing
    Avoiding Blocked Designs
  115. 115Define Avoiding Blocked DesignsNot always possible to avoid and sometimes a feature of the tool being used
  116. 116Define Counter BalancingAll conditions in the experiment have equal participants.
  117. 117Problems of One-item scalesDifficult to validate - Test- retest validation
    Psychometrics "rules" at least 3 items per construct
    Sometimes useful - job satisfaction
  118. 118Problems with adapting/changing a scaleChange wording
    Drop/add items
    Face vs construct validity - many scales are validated with students, so face validity can be problematic
  119. 119What characterises the 'good participant'?The participant attempts to discern the experimenter's hypotheses and tries to confirm them.
  120. 120What characterises the 'negative participant'?The participant tries to discern the experimenter's hypotheses in order to destroy the credibility of the study.
  121. 121What characterises the 'faithful participant'?The participant follows the instructions given by the experimenter to the letter.
  122. 122What characterises the 'apprehensive participant'?The participant is so concerned about evaluation by the experimenter that they suppress their immediate responses and behave in what they think is a socially desirable way.
  123. 123French & Raven (1959)Obedience to legitimate authority is a powerful social norm
  124. 124First part of the Nuremberg Code (1947) -> Declaration of Helsinki (2000)The voluntary consent of the human subject is absolutely essential
  125. 125Second part of the Nuremberg Code (1947) -> Declaration of Helsinki (2000)The experiment should yield fruitful results for the good society, that cannot be obtained by other means
  126. 126Third part of the Nuremberg Code (1947) -> Declaration of Helsinki (2000)The experiment should be based on previous research so that the anticipated results can justify research
  127. 127Fourth part of the Nuremberg Code (1947) -> Declaration of Helsinki (2000)All unnecessary physical and mental suffering should be avoided
  128. 128Fifth part of the Nuremberg Code (1947) -> Declaration of Helsinki (2000)No experiment should be conducted where there is a reason to believe that death or disabling injury may be the result
  129. 129Sixth part of the Nuremberg Code (1947) -> Declaration of Helsinki (2000)The degree of risk should also be less than the potential humanitarian importance of the research
  130. 130Explain the Nuremberg Code (1947)Research ethics principles for human experimentation. 10-point moral code for scientists.
  131. 131Explain the Declaration of Helsinki (7th revision, 2013)Developed by World Medical Association. "Morally binding" for investigators over and above local legislation
  132. 132BPS definition of Risk"the potential physical or psychological harm, discomfort or stress to human participants that a research project may generate"
  133. 133List what would be considered more then minimal risk in an investigationSensitive topics (sexuality, political opinion, health)
    Significant and Necessary Deception
    Psychological stress, anxiety or humiliation or cause more than minimal pain
    Access to records of personal or confidential information
    Vulnerable Groups - children, mental incapacity, patients
    Collection of biological samples/tissue
    'labelling' either by the researcher or by the participant (e.g. 'i am stupid')
    Access to information through a third party
    Hidden vulnerability - dyslexia, physical impairment, poverty
    Adverse impact on Employment or social standing
    Invasive interventions (drug administration, excessive exercise etc. more than daily life
  134. 134Define ConfidentialityParticipants have a right to expect that information they provide will be treated confidentially and, if published, will not be identifiable as theirs
  135. 135Define Valid ConsentParticipants are informed and consent freely
    Participants can withdraw their consent at any time
  136. 136Define DeceptionOmitting some details of a study different to deliberately falsely informing the participants of the purpose of the research.
  137. 137Deception necessary where...knowing the full details of the study may lead Participants to act differently
  138. 138What are the two kinds of personal data?Personal Date
    Sensitive Personal Data
  139. 139Define Personal DataAny information relating to a natural person who can be identified, directly or indirectly, in particular by reference to an identifier such as:
    Name
    Identification number
    Location Data
    Online identifier
    Pseudonymised data
    Factors specific to physical, physiological, genetic, mental, economic, cultural or social identity
    Include faces, voices, movement, MRU, EEG, fluids etc.
  140. 140Define Sensitive Personal DataRacial or ethnic origin
    Political opinions
    Religious or philosophical beliefs
    trade union membership
    genetic or biometric data processed for purpose of identification
    health
    sex life or sexual orientation

Information

Description
Psychology (2A) Flashcards on Research Planning and Design, created by Hannah Luke on 18/11/2020.
Created by
Hannah Luke
Views
1
Rating

More on Reliability, Validity and Research Evaluation

Feedback

What is it about?

What do you think of the new look?

How do you like it?