Slide 1This chapter introduces six quasi-experimental approaches to causal questions. Each one makes different assumptions about what serves as the counterfactual.

Slide 2London, 1854. Cholera was tearing through the city, and most physicians were sure they knew why: miasma, the foul air rising from the Thames and the city's open sewers. John Snow had a different idea. He suspected cholera spread through contaminated water, but he couldn't run an experiment. He couldn't randomize which Londoners drank from which source.

Slide 3Two water companies served overlapping neighborhoods in south London. One had recently moved its intake pipe upstream of the city's sewage outflows, and the other hadn't. Households on the same street, in the same conditions, were drinking water from different sources for reasons that had nothing to do with their health. Snow compared death rates and found that households served by the company still drawing from the contaminated section of the Thames died at roughly eight times the rate of those with cleaner water.

Slide 4There was no randomization and no experiment, but Snow made a powerful causal claim, built on the logic that an accident of infrastructure had created comparison groups almost as good as anything a trial could produce. His investigation is one of the earliest examples of what we now call a quasi-experimental design.

Slide 5We often find ourselves in Snow's shoes. We want to estimate the effect of conflict on child malnutrition, but we can't randomize wars. We ask if air pollution causes asthma in children, but we wouldn't randomly assign families to breathe dirty air. Sometimes randomization is technically possible but nobody with decision-making authority will agree to it. And sometimes the question is retrospective: the exposure already occurred, and we're working with the data history left behind.

Slide 6In our session on causal inference, we introduced two broad strategies for causal inference from non-experimental data. Confounder control closes backdoor paths through statistical adjustment. Design-based approaches isolate front-door variation that mimics randomization. This chapter is about the second.

Slide 7In 1983, the economist Edward Leamer published a critique of his own field. Empirical results, he argued, were fragile. Change the specification, the sample, or the set of control variables, and the conclusions changed too. In his words, “Hardly anyone takes data analysis seriously. Or perhaps more accurately, hardly anyone takes anyone else's data analysis seriously.”

Slide 8He was right. For most of the twentieth century, the standard approach to causal questions in the social sciences was to collect observational data, run a regression, control for confounders, and call it causal. Researchers had the statistical tools, but they lacked credible reasons to believe their estimates were causal and not confounded. You could always add another control variable, try another functional form, or argue about logs versus levels. The results often shifted with each choice, and there was no principled way to decide which specification was right.

Slide 9The tools for a different approach already existed. Economists had instrumental variables, others had cataloged threats to validity, and education researchers had formalized regression discontinuity. What changed, in what's been called the credibility revolution, was less about inventing new techniques than about demanding new standards. Researchers began asking a simple question: What is your identification strategy? What feature of your research design, and not your statistical model, gives you a credible claim to causality? The best contemporary design-based studies make both an institutional case, here is why this variation is plausibly exogenous, and an empirical one, here are the tests that support that claim.

Slide 10This chapter covers six designs, arranged roughly from simplest to most sophisticated: single group pre-post designs, multiple group post-test only designs, difference-in-differences, interrupted time series, regression discontinuity, and instrumental variables.

Slide 11We'll start with the simplest and weakest design: measure an outcome before an intervention, implement the intervention for everyone, then measure the outcome again. The pre-intervention outcome stands in for what would have happened without treatment. That's the counterfactual. The causal claim rests on a simple subtraction, post minus pre equals effect, and it holds only if nothing else changed between the two measurements. That's the key assumption, and it rarely holds.

Slide 12When one research team set out to study whether a parenting program could reduce child abuse among adolescents in low- and middle-income countries, there was no existing evidence. Designing a full randomized trial would have meant guessing at nearly every parameter, so they chose a pre-post design deliberately. Their program enrolled 115 adolescent-caregiver pairs in rural Eastern Cape, one of the country's poorest provinces, with no exclusion criteria. The program ran for 12 weeks and was delivered by local NGO childcare workers, not trained psychologists.

Slide 13Adolescent-reported abuse dropped from an average score of 4.33 to 1.33, and caregiver-reported abuse fell from 11.32 to 1.68. Positive parenting improved, depression decreased in both adolescents and caregivers, and no harmful effects were detected on any outcome.

Slide 14But the researchers didn't claim they'd proven the program works. They called their findings “indicative of potential programme results” and were explicit that a pre-post design “cannot determine causality.”

Slide 15The study served three purposes that a pre-post design handles well. It established that the program was safe, it generated the effect size estimates needed to power a future trial, and it revealed an unexpected finding. Families were spontaneously teaching program content to neighbors, in churches, and on shared taxi-buses. That meant an individually randomized trial would be contaminated, so the team pivoted to a cluster randomized design.

Slide 16This is a pre-post study used as intended, as the essential first step in a research program. The team invested in construct validity, with standardized measures from both adolescent and caregiver perspectives, and in external validity, with real-world delivery conditions and no exclusion criteria, rather than internal validity. That was a deliberate tradeoff, and they were transparent about it.