VALIDITY & RELIABILITY

VALIDITY & RELIABILITY

1. Validity

   • validity:  warranted conclusions characteristics of the individuals studied. A reliable instrument is one that gives consistent results.
   • INSTRUMENTATION
     • process of collecting data 
     • Data
       •  information researchers obtain on the subjects 
     • OBJECTIVITY & USABILITY
       •  subjectivity from the judgments  achievement, performance, or characteristics of subjects  important consideration   choose or design an instrument 
     • CLASSIFY INSTRUMENTS
       • who provides the data the method of data collection who collects the data Examples:  researcher-completed instruments rating scales, interview schedules, observation forms, tally sheets,flowcharts, performance checklists, anecdotal records, and time-and-motion logs Instruments  completed by the subjects  questionnaires; selfchecklists; attitude scales; personality inventories; achievement, aptitude, and performance tests; and projective and sociometric devices Items or questions used in subject-completed instruments can take many forms classified as either selection or supply items True-false items, multiple-choice items, matching items, interpretive exercises Supply items: short-answer items and  essay questions.
       • Research data
         • Obtained directly or indirectly  Assessing the subjects
       • Self-report data
         • provided by the subjects of a study themselves
       • Informant data
         •  provided by other people about the subjects of a study.
   • appropriateness, meaningfulness, correctness, and usefulness
     • researcher's inferences  based on data obtained  through the use of an instrument.
     • Content-related evidence
       • Refers to judgments on the content and logical structure of an instrument 
     • Criterion-related evidence
       • information provided by an instrument agrees with information obtained on other, independent instruments.
     • Criterion
       • A Standard for judging With reference to validity A second instrument against which scores on an instrument can be checked.
     • Construct-related evidence
       •  The degree to which the totality of evidence obtained is consistent with theoretical expectations.
     • Coefficient
       • Numerical index representing the degree of correspondence between scores on an instrument and a criterion measure.
     • Expectancy table
       • A two-way chart used to evaluate criterion-related evidence of validity.

2. RELIABILITY

   • refers to the consistency of scores or answers provided by an instrument.
   • Errors of measurement
     • Refer to variations in scores obtained by the same individuals on the same instrument
   • The test-retest method
     • Involves administering the same instrument twice to the same group of individuals after a certain time interval has elapsed.
   • The internal-consistency method
     • Involves comparing responses to different sets of items that are part of an instrument.
   • The Score
     • Scoring agreement requires a demonstration that independent scorers can achieve satisfactory agreement in their scoring.
   • The standard error
     • A numerical index of measurement error. 

3. INTERNAL VALIDITY

   • When a study lacks internal validity, one or more alternative hypotheses exist to explain the outcomes.  These alternative hypotheses are referred to by researchers as threats to internal validity. When a study has internal validity, it means that any relationship observed between two or more variables is unambiguous, rather than being due to something else.
   • CONTROLLING THREATS
     • CONTROLLING THREATS TO INTERNAL VALIDITY  (1) standardizing the conditions under which the study occurs (2) obtaining and using more information on the subjects of the study (3) obtaining and using more information on the details of the study(4) choosing an appropriate design.