External validity, generalizability, and transportability
Chapter 8: External Validity, Generalizability, and Transportability — Video 1
https://ghrbook.com/videos/external-validity-generalizability-transportability/

[Slide 1] This chapter is about external validity. We'll distinguish three related but distinct concepts — external validity, generalizability, and transportability — and develop a framework for thinking rigorously about when and why research findings travel.
[Slide 2] In 1997, Clinical Trial 320 delivered news that HIV and AIDS researchers and patients had been waiting for. A three-drug combination therapy cut the risk of AIDS or death roughly in half, from 11% to 6%. The findings helped launch the antiretroviral era that would transform HIV from a death sentence to a manageable chronic condition.
[Slide 3] Years later, researchers asked a different question. What would that treatment effect have looked like in the U.S. population of people newly infected with HIV a decade after the trial — a population that differed from trial participants in age, race, and disease severity?
[Slide 4] Re-analyzing the trial data, they put the reduction among trial participants at 49%. Standardized to that later population, the estimated benefit shrank to 43%. Same treatment. Same outcome. Different answer.
[Slide 5] The later analysis illustrates something important. Causal effects estimated in one population don't automatically apply to another. Understanding when they do, and when they don't, is the problem of external validity. That's what this chapter unpacks.
[Slide 6] In the causal inference chapter, we focused on internal validity. Is the causal effect we estimated correct for the people we actually studied? External validity asks the next question. Does that effect apply to the population or context we care about?
[Slide 7] While internal validity is specifically a causal inference concept, external validity concerns arise more broadly. A prevalence estimate, an association, a prediction model — each can be correct for the sample while failing to apply elsewhere, to the target population the sample was drawn from, or to new populations. Whenever we extend findings beyond the local study that produced them, we face an external validity question.
[Slide 8] External validity concerns take two forms, depending on the relationship between the study sample and the target population. Generalizability applies when the study sample is entirely contained within the target population — when the sample is a subset of the target. A clinical trial conducted at Kenyan hospitals, aiming to inform treatment policy for all Kenyans, faces a generalizability question: Do the trial participants adequately represent the broader target population they were drawn from? A national survey faces the same question: Does the sample reflect the population it was designed to represent? Concerns here arise from selection, nonresponse, eligibility criteria, and attrition.
[Slide 9] Transportability applies when the study sample is not contained within the target population — when you want to apply the results to a different population entirely. If we want to know whether the Kenyan trial's results apply in Uganda, or whether a prevalence estimate from urban clinics applies to rural areas that were never sampled, we face a transportability question. The study population is external to the target population.
[Slide 10] External validity bias arises when an estimate produced in a study differs from the corresponding quantity in a specified target population, because the conditions that generated the estimate do not hold in that population.
[Slide 11] For descriptive findings like prevalence, this is relatively straightforward. If your sample over-represents certain groups, your estimate likely won't match the target population. The solution is familiar: weight the sample to look like the population. For causal effects, the problem is more specific. What matters is not whether the populations differ on any characteristic, but whether they differ on characteristics that modify the effect — variables for which the treatment works differently across levels.
[Slide 12] A variable is an effect modifier if the causal effect of the treatment changes depending on that variable's value. Age might modify the effect of a blood pressure drug if the drug lowers blood pressure more in younger patients than older ones. Disease severity might modify the effect of a behavioral intervention if it works well for mild cases but poorly for severe ones.
[Slide 13] If a treatment works equally well in everyone, population differences don't threaten external validity. Your trial could over-sample young people, wealthy people, or urban residents — and as long as the treatment effect is the same across those groups, your estimate still applies to your target population. But if the treatment works differently across groups, then differences between your sample and target population become consequential. Effect modification is why some population differences matter for external validity and others do not.
[Slide 14] This logic applies to both generalizability and transportability. Whether you're asking if a trial represents its source population, or whether findings from one country apply in another, the key question is the same: Do the populations differ on characteristics that modify the estimate?