Validity

Validity

1. Reliability

   • Consistency
   • Can have reliability without validity

2. Usability

3. Assessments

   • Content: How well the sample of assessment tasks represents the domain of tasks to be measured and how it emphasizes the most important content.
     • Content Considerations
       • Defining the Content Domain
       • Test Selection
     • Construct Considerations
       • Construct Validation
         • Defining the Domain
         • Analyzing the Response
         • Comparing the Scores of known Groups
         • Correlating the scores with other measures
         • Comparing Scores before and after learning
   • Construct: How well performance on the assessment can be interpreted as a meaningful measure of some characteristic or quality.
   • Assessment- Criterion Relationship: How well performance on the assessment predicts future performance or estimates current performance on some valid measure other than the test itself.
     • Predictive Validation Study
       • Test Performance
         • Criterion Performance
     • Concurrent Validation Study
     • Predicting Future Performance
     • Correlation
   • Consequences: How well use of assessment results accomplishes intended purposes and avoids unintended effects.

4. Matter of Degree

5. Unitary Concept

6. Appropriateness

7. An evaluation of the adequacy and appropriateness of the interpretations and uses of assessment results

8. Factors influencing Validity.

   • Unclear Directions
   • Reading vocabulary and sentence structure too difficult (construct- irrelevant variance.
   • Ambiguity
   • Inadequate time limits
   • Overemphasis of easy-to-assess aspects of domain at the expense of important but difficult-to-assess aspects.
   • Test Items inappropriate for the outcomes being measured.
   • Poorly constructed test Items
   • Test too short
   • Improper arrangement of items
   • Identifiable pattern of answers