Flashcards
Research Meth

Cards in this deck (94)
- 1Problem: Productivity of factory employees is too low Research question:Which factors influence the productivity within the factory?
Theory:
- Perceived stress
- Many more
Practice:
- many deadlines - other cues - 2Conceptual definition Stress (IV) “Negative cognitive and emotional states elicited when persons perceive that their demands exceed their ability to cope”Operational definition
Negative cognitive and emotional state employees report as a result of the deadlines within the company - 3Indicators (Stress)Extent to which employees experience negative emotions.
- Extent to which employees have impression that deadlines ask too much of them.
- Extent to which employees experience work-related stress. - 4Questions (Stress)How sad/angry/frustrated/fatigued do you feel?
- To what extent do you have the impression that the deadlines within the company ask too much of you?
- To what extent do you perceive work-related stress? - 5Steps1. Problem
2. Research question
3. Literature study
4. Conceptual model
5. Propositions/Hypotheses
6. Indicators
7. Questions - 6Observed valuetrue value + measurement
error - 7Measurement errorsystematic error + random error
Systematic: always in the same direction
Random error: both directions, equals each other out (±) - 8Validity(Aussagekraft)
Do I measure what I want to measure? - 9Content validityExtent to which the measuring instrument provides adequate coverage of the concept you study.
Disadvantage: you have to have a very clear idea about what constitutes your concept. - 10Face validity (content v)“At first sight” a good translation
of the construct
Weak evidence
Better: select groups of experts to give their opinion - 11Criterion-related validity Predictive validityTheory: Construct A (now) causes construct B (future)
Indicator A is associated with
indicator B (= criterion)
Possible to predict future
(before B is measured)
Measured at different moments!
2016: Assessment test
2018: Job performance - 12Criterion-related validity Concurrent validityTheory: Construct A (now) causes construct B (now)
Indicator A is associated with indicator B (= criterion)
Possible to predict future (when A and B are measured at the
same time). Measured at same moment! - 13Construct validityExtent to which concepts relate to other concepts
- 14Convergent validitymeasures of constructs that theoretically should be related to each other are, in fact, observed
to be related to each other
Compare different measurement instruments that measure the same concept - 15Discriminant validitymeasures of constructs that
theoretically should not be related to each other are, in fact, observed to not be related to each other - 16What are correlations?Indicates the strength of a relationship between two concepts.
Not an indicator of causality
+1 = perfect positive relationship
0 = no relationship
-1 = perfect negative relationship
- 17MultiTrait MultiMethod matrix Three traits Two methodsThree “Traits”: Emotional, Informative, and Instrumental support at work
Two Methods: general evaluation vs. specific questions - 18What if Construct validity is not sufficient?Theoretical model incorrect
Procedure to test hypothesis not appropriate
Other methods lack construct validity/are not reliable - 19ReliabilityIf I would take the test again, would I get the same results?
Assess whether your measurement instruments provides similar measurements when used repeatedly - 20reliability Stability• Fluctuations in results because of personal and situational aspects (test-retest for the same respondent)
- 21reliability EquivalenceFluctuations in results because of differences between researchers (e.g. inter-rater reliability)
- 22reliability Internal consistencyDegree to which multiple items measure the same construct
- 23Split-half method:choose one half of the items randomly and compare those to the other half (should give the same results and high correlation)
- 24Statistic: Cronbach’s alpha= mean of all split-half; between 0 and 1
- 25Advantage of multi-item scaleLess influence of measurement error within one concept
- 26Observed valuetrue value valid measurement
- 27Random error0 (zero) reliable measurement
- 284 Data types Nominalnumbers indicate categories; order is not important
Examples:
gender, Religion: Catholic, Jewish, Muslim, Atheist... Special type: dichotomous - 29Ordinalnumbers;
order important,
but “2” is not two times “1”
Example: size
how do you feel today 1234 - 30Intervalnumbers,
order important,
“2” is two times “1”;
no fixed zero point
Example:
Temperature: Fahrenheit or Celsius (0oF ≠ 00C). Key: Arbitrary zero point. - 31Rationumbers,
order important,
“2” is two times “1”;
fixed zero point
Example: Length - 32Data types determine the analyzing technique from interval ratio to lower data type = NO PROBLEM not possible from lower to higher data types

- 33—


- 34Symmetric scaleTreat as interval (no reason to assume intervals are not equal)
- 35Asymmetric scaleTreat as ordinal (interval size cannot be estimated properly)
- 36Double-barrelled questionsDo you know the brand and what is your opinion about it?
Solution:
1. Do you know the brand?
2. If yes, what is your opinion about it? - 37Leading questions“Do you also think the current political conditions are bad?”
What do you think about the current
political conditions?”
verybad 1 2 3 4 5 6 7 verygood - 38Avoid ambiguitiesUse informal language
“In the context of fast-moving consumer goods, how often do you experience post-purchase cognitive dissonance ?”
Better:
“How often do you regret a purchase you made in the supermarket?Use informal language
“In the context of fast-moving consumer goods, how often do you experience post-purchase cognitive dissonance ?”
Better:
“How often do you regret a purchase you made in the supermarket? - 39 Precision “What is your income?”Better:
What is your net income per month?
How much holiday pay (if applicable) do you receive per year (net)? - 40Double negativeI don’t think that Albert Heijn has bad customer service
- 41Positive test strategyRespondents tend to answer questions in a positive way
Albert Heijn provides good customer service - 42Funnel approachQuestion order
Start with “warm-ups”
general vs. specific
How satisfied are you with Albert Heijn?
How satisfied are you with Albert Heijn’s chocolate assortment? - 43Closed questions Dichotomous scaleSimple category scale
two answers - 44Closed questions Categorical scaleMultiple choice single/multiple resp. scale
Where did you buy your last book? □ in a bookshop
□ in an antique store
□ at an online book store
□ on a market
Multiple choice single/multiple response scale - 45—


- 46Closed questions Likert scaleBuying books online is superior to buying them in stores
12345 Strongy Agree Neither agree Disagree Strongly agree nor disagree disagree - 47Semantic differential scaleIn my opinion, buying a book in an online book store is
easy ___ : ___ : ___ : ___ : ___ : ___: ___ expensive ___ : ___ : ___ : ___ : ___ : ___: ___ stupid ___ : ___ : ___ : ___ : ___ : ___: ___ negative ___ : ___ : ___ : ___ : ___ : ___: ___
difficult cheap smart positive - 48Numerical scaleWhat aspects are important in deciding to buy a book in a traditional book shop?
Not important at all Extremely important Highquality 1234567 Badservice 1234567 Goodassortment 1 2 3 4 5 6 7 Accessibility 1 2 3 4 5 6 7 Lowprice 1234567 - 49Graphic rating scale

- 50mutually exclusiveif two events cannot both occur
coin toss - 51collectively exhaustiveif at least one of the events must occur. For example, when rolling a six-sided die, the outcomes 1, 2, 3, 4, 5, and 6 are collectively exhaustive, because they encompass the entire range of possible outcomes
- 52Why do we want to sample instead of studying the whole population?Costs: sample is less expensive (17.110.608 citizens in the Netherlands as we speak)
• Accuracy: Research based on sampling can be more accurate than research based on census
• Speed: less information means faster processing
• Availability of population elements: e.g. destruction testing - 53PopulationThe group you want to say something about
- 54Sampling frameThe group you can say something about
- 55SampleYour selection from the sampling frame
- 56Characteristics of a correct sampleAccuracy: no systematic bias
(absence of systematic variance)
Precision: values in the sample (highly)
similar to values in the population (absensce of sampling errors) - 57Probability vs. non-probability sampling probability:Based on random selection: each unit has the same chance of being selected
Not easy and often impractical, effortful
simple random & comlex random sampling - 58Systematic Sampling• Selects every kth element of the population
• Random starting point
Advantages:
Simple to design
• Strictly speaking not random, but it approaches it
Disadvantages
Not possible without a meaningful system in the elements (e.g. regularity: days of the week) - 59Stratified sampleDivide population into groups (different sizes) based on self- selected variable
• Statistical advantage: If strata are large enough, you can test
differences between strata
• Ideally: large differences between strata, small differences within strata
Advantage: Possibility of different methods & conclusions for all strata. Also efficiency gains.
• Disadvantage: Sometimes hard to make a decision between criteria for stratification. - 60Cluster samplingPopulation consists of already existing primary units (e.g., classes, teams, cities)
• E.g.: random choice of cities (=clusters) and take a sample from supermarkets - 61Non-probability samplingnot based on random selection
different selection probailities for the elements
more subjective than probability sampling
selection probability is unkown
convienience, judgement quota snowball - 62Convenience samplingFreedom to choose whoever can be found or wants to participate
“Quick and dirty”
Can be useful to test an idea in limited time - 63Snowball samplingRespondents identify other potential respondents for the researcher
• Advantage:
• efficient way to find respondents in niche
situations (e.g croquet players) •
Disadvantage:
• good chance that respondents know similar respondentssample not representative for the population - 64—Experiments take you closer to causality than surveys alone

- 65Causality1) Correspondence (i.e., correlation)
2) Precedence
3) No other extraneous variables - 66Steps in experimenting1. Select relevant variables and determine treatment levels of IV(s)
2. Control experimental environment: Blind vs. double blind
3. Choose design and assignment process 3b. Pilottest and adjustments - 67Experiments are characterized by: Researcher’s control over experimental setting (manipulate independent variable)
Random assignment of subjects to conditions - 68True experimentFull experimental control
Experimental group vs. Control group
Groups are equal (randomization or matching)
Advantage
Study something which in real life is impossible
Disadvantages
› Evaluation apprehension: guinea pig effect › Manipulation makes situation less realistic:
generalizable? - 69Field experimentNatural setting(‘real’world)
› More heterogeneous participants than in lab
› Less evaluation apprehension
› Manipulate variables & randomly assign participants
disadvantage: less experimental control - 70Quasi-experimentsNatural setting, uses existing groups / differences
Diadvantage: less experimental control - 71Guinea pig effectoccurs when subjects knowledge that they are participating in an experiement
will affect their response to it - 72Test statisticnumber which gives you information about the test outcome
- 73p-valueprobability that the outcome is based on the chance
- 74Degree of freedomNumber of independent observations based on which test statistic is calculated
- 75Hypothesis/proposition vs. H0/H1Hypothesis/proposition:
• Motivation and satisfaction are positively related
• The more motivation, the more satisfaction, and vice versa. - 76testing group differencest-test (never perform multiple t-test, use ANOVA)
chi-square test
ANOVA
mann whitney u test - 77testing relationshipsregression
- 78Pearson correlationMeasure for strength of a linear relationship between two internal variables as r
-1 <= x >= 1 - 79spearman rank correlationMeasure how well the relationship between two ordinal variables can be described using a monotonic function
- 80ABCD / Grandma Methodfor report results
A what i the goal?
B how did you do that?
C What was the result?
D which conclusion - 81Testing hypothesis the big 31. How many variables are involved?
2. What is the data type of the involved variables?
3. Asymetric or symmetric? - 821. How many variables are involved?one = univariate analysis (descriptive statistics)
two = bivariable analysis (inferential statistics)
>2 = multivariate analysis - 832. What is the data type of the involved variables?Nominal
Ordinal
Interval / Ratio - 843. Asymmetric or symmetric?Asymmetric =
-when variable have diff data types
-Predicting DV from IV (“causal” relationship)
symmetric = same data type - 85—.

- 86Related vs unrelated sample

- 87cronbachs alphaone general measure for how well the items are related to each other
0 <= a => 1
a>0,6 = reliability sufficient, create a sum variable
a<0,6 = Reliability not sufficient, could some items be dropped? separate analyses - 88Regression vs CorrelationRegression:
When we want to predict one variable based on another variable
Correlation
when we want to establish a linear relationship - 89the research process1. The problem statement
2. Research question and Factors
3. The sub-research questions
4. The conceptual model
-made from sub research questions
-describes the proposed relationships
-independent, dependent and control variables
5. Hypothesis
-derive from step 3&4
6. Conceptual and Operational definitions
-clearly define each concept
7. Indicators
- one indicator for each variable - 90The data-analysis1.Preparing the dataset
-label all the variables
-check for outliers (boxplot)
2. Decriptive statistics
-distributions (control variables)
3. Correlation and reliability analysis
4. New variable
-if cronbachs alpha >0,6 = sum variables
5. Choice of technique
-the big 3
6. Control variable
-exclude any possible effects on the control variable
7. Testing the hypothesis
8. Conclusion and implications - 91Construct validityDo the indicators fit with existing theory about the concept?
- 92Content validityIf a combination of the indicators covers all aspects of a concept
- 93Operational Definitioneven more specific than conceptual def
making it measurable - 94Conceptual definitionclear definition by scientific theory
to whom related?
Information
- Description
- Flashcards on Research Meth, created by M.T. Niclassen on 04/06/2018.
- Created by
- M.T. Niclassen
- Views
- 17
- Rating
More on Reliability, Validity and Research Evaluation