Measurement, Validity and Reliability

What this unit covers

The resources here deal with how researchers turn ideas into numbers, and how they judge whether those numbers can be trusted. The mind map Measurement of behavior sets out the scales of measurement (nominal, ordinal, interval and ratio) and the idea that an observed score is a true score plus measurement error. The mind map X = T + B + W + Er goes further, separating bias from validity error.

Where to start

Begin with the basics of measurement, then move to quality. A sensible order is:

1. Read Measurement of behavior and the mind map Four Scales of Measurment.
2. Work through the flashcard deck Chapter 8 Measurements, which follows the steps of the measurement process, from concepts to operational definitions.
3. Move on to reliability and validity with Types of Reliability and TYPES OF VALIDITY.
4. Finish with the mind map Threats to Validity and the one on internal and external validity threats.

Using quizzes and maps together

Read a mind map first, then test yourself. The quiz Validity Lecture 3 asks what constructs are and checks construct under-representation against construct-irrelevant variance. Reliability and Validity Quick Test is a short check once you have covered both ideas. Wherever a quiz catches you out, go back to the matching map and read that branch again.

Going deeper

Several maps look at test building and theory. Scale construction 2 covers cleaning data, including duplicate cases and missing responses. Classical Test Theory (CTT) and Generalizability Theory (G-Theory) suit students who want the measurement models, and the quiz Item Response Theory follows on from them. Teachers can pick out single maps or decks to set as pre-reading.

Where it fits

This unit sits in the Planning and Designing Research strand of Research methods and academic skills. Open a resource, try a quiz, and add your own notes or flashcards where the set has gaps.