Chapter 9
Measurement and Construct Validation
7 videos · about 48 minutes · watch as a YouTube playlist →
- 1Even counting the dead takes judgmentDeciding whether someone died with COVID-19 or from it is a judgment made by a certifier, and the proposed fix — excess mortality — brings challenges of its own. Construct, indicator, instrument, and measure: the vocabulary the rest of the chapter runs on.6 min
- 2From conceptual model to measurementA conceptual model you drew for another purpose will tell you what to measure. A DAG turns a causal inference problem into a data collection checklist, and a logic model does the parallel job for a program — ending in treatment fidelity and the difference between implementation failure and theory failure.6 min
- 3Finding good indicatorsDREAMY as a checklist for a good indicator, with two fully specified indicators of depression as the worked case. Includes what to do when the thing you care about cannot be put to a respondent as a question, such as measuring corruption by digging up a road.5 min
- 4Indexes and scalesIndexes and scales both combine several items into one score, and they do it for opposite reasons. In an index the items cause the score; in a scale the score causes the items, and the direction of that arrow changes what you are allowed to do with the number.7 min
- 5Validity is not a property of an instrumentA validated instrument is not a settled one. Validity is a judgment about whether scores mean what you think they mean — for these people, in this setting, for this purpose — worked through content validity, cognitive interviewing, and item analysis.8 min
- 6Factor analysis and reliabilityTwo questions about a set of items, asked in order. Factor analysis asks whether they measure one thing or several, and reliability asks whether they measure it consistently — test-retest, internal consistency, inter-rater agreement, and responsiveness to change.10 min
- 7Does it match reality, and does it travel?The last phase of construct validation turns outward: do the scores line up with something in the world? Diagnostic accuracy read off a confusion matrix, then cross-cultural validity and the response formats that fail when an instrument crosses a border.6 min