Finding good indicators
Chapter 9: Measurement and Construct Validation — Video 3
https://ghrbook.com/videos/finding-good-indicators/

[Slide 1] When you select and define indicators of outcomes and other key variables, this is called operationalizing your constructs, and it is a critical part of measurement planning.
[Slide 2] When you finish the study and present your findings, one of the first things colleagues will ask is, how did you define and measure your outcome?
[Slide 3] Hopefully you can say that your indicators are DREAMY. Defined, clearly specified. Relevant, related to the construct. Expedient, feasible to obtain. Accurate, a valid measure of the construct. Measurable, able to be quantified. And customary, a recognized standard. Accuracy is the subject of the whole second half of this chapter, so it gets its own treatment later.
[Slide 4] A good indicator is clearly specified: what exactly is being measured, how it will be measured, and when. This includes specifying the time frame. Are you measuring depression at baseline, at 3 months post-intervention, or at 12 months? A well-defined indicator enables readers to critically appraise your work, and it serves as a building block for future replication attempts.
[Slide 5] The authors defined two indicators of depression. The first was depression severity: the Beck Depression Inventory total score, measured at 3 months after the treatment arm completed the intervention. The second was depression prevalence: the proportion of participants scoring 10 or higher on the Patient Health Questionnaire total score, measured at 3 months post intervention. Notice that each definition specifies what, how, and when.
[Slide 6] An indicator must be quantifiable. You need to be able to assign a number to it. For some constructs this is straightforward: height, weight, blood pressure, test scores. For others, measurement requires creativity. Psychological constructs like depression are typically measured using questionnaires that translate subjective experiences into numerical scores. But what about constructs that people won't self-report honestly?
[Slide 7] Researchers faced exactly this challenge when trying to measure government corruption in Indonesia. Asking officials to report their own corrupt behavior wasn't going to work. So the researchers dug core samples of newly built roads to estimate true construction costs, then compared these estimates to the government's reported expenditures. The difference became their indicator of corruption.
[Slide 8] In general, it's good advice to use standard indicators, follow existing approaches, and adopt instruments that have already been established in a research field. There are several ways to do this. One way is to read the literature and find articles that measure your target constructs. If you're planning an impact evaluation of a microfinance program on poverty reduction and wish to publish the results in an economics journal, start by reading highly cited work by other economists to understand current best practices. How do these scholars define and measure outcomes like income, consumption, and wealth? Systematic reviews and methods papers are also good resources for learning about measurement.
[Slide 9] A third approach is to search for nationally or internationally recognized standards. If you're studying population health, a good source of customary indicators is the World Health Organization's Global Reference List of the 100 core health indicators. Another is the United Nations Sustainable Development Goals metadata repository, which includes 231 unique indicators to measure 169 targets for 17 goals.
[Slide 10] Some indicators are straightforward. A hemoglobin level below 7.0 g/dl indicates severe anemia. The number from the lab is the indicator, and you just need a clear definition and a reliable instrument. Other indicators require construction, combining multiple pieces of data, and the complexity of that construction varies.
[Slide 11] Consider a common indicator in malaria research. Does the household have any factory-treated mosquito nets, or nets that have been dipped in a liquid to kill or repel mosquitos in the past 12 months? You can't just ask this question directly. It's too long, too compound, and respondents may not know whether their net was factory-treated.
[Slide 12] Instead you break it into simpler questions and combine the answers. Does your household have any mosquito nets? If yes, how many months ago did your household get the net? If 12 months or fewer, was the net factory-treated with insecticide, which is often determined by observation or brand? If no, was it ever soaked or dipped in insecticide, and if yes, how many months ago? These questions combine through logical rules. A household has a valid net if they have a net and it was either factory-treated or dipped within the past 12 months. The final indicator, a single yes or no variable, is constructed from multiple survey items using predetermined logic.
[Slide 13] Rule-based composites are common in global health. Other examples include vaccination status, which combines records of multiple doses, and eligibility criteria for program enrollment. The other family of composites works differently: the items get combined by adding them up, and that is where measurement starts to get interesting.