Video 6 of 6
Choosing and reading an observational design
Choosing an observational design starts with the question, then the outcome, the timeline, the budget and the exposure. Why three studies from the chapter fit their designs, constraints beyond the textbook factors, and what to check first when reading a case-control, cohort or cross-sectional study, including what the authors did not measure.
6:45 · 15 slides · printable slides · transcript
Slides
Printable deck →▶Transcript15 sections
Generated from the narration script. Plain text version.
1We have a research question, and we've determined that a randomized trial isn't feasible. How do we choose among the observational options? And when we read a published observational study, how do we judge whether it's credible?
2Start with the kind of question. Descriptive questions about how common something is, or what patterns exist, point toward case reports, surveillance, and cross-sectional designs. Questions about associations between exposures and outcomes can be tackled with cross-sectional, case-control, or cohort studies. Causal questions push us toward case-control or cohort designs with careful attention to confounding, and even then, the evidence is weaker than a randomized trial or quasi-experiment would provide.
3Then consider the outcome. Rare outcomes push us toward case-control studies, because we'd need enormous cohorts to capture enough cases. Common outcomes can be studied efficiently with cohort or cross-sectional designs. Timing matters next. If we need answers in weeks, case-control or cross-sectional designs are the realistic options. If we can wait years, a prospective cohort gives the cleanest temporal sequence. And if we want long-term outcomes without the wait, a retrospective cohort can deliver decades of follow-up in months of chart review, provided the records exist and capture what we need.
4Resources and exposure characteristics fill in the rest. Limited budgets favor cross-sectional and case-control designs over prospective cohorts that demand years of follow-up infrastructure. Past exposures with rare outcomes point to case-control. Current exposures we want to follow forward point to a prospective cohort. And exposures that vary over time call for longitudinal or panel designs that can capture the variation.
5We've already seen these factors at work. The cholera outbreak in Yemen needed answers within weeks during active conflict, and cases were already presenting to treatment centers, so a case-control study was the realistic choice. The question about long-term ART outcomes had clinical records going back years, so a retrospective cohort fit. And regional patterns in breastfeeding across Ethiopia were a descriptive question with survey infrastructure already in place, so a cross-sectional survey was the right tool.
6These factors don't always converge on a single answer, and they shouldn't. The point is to make the trade-offs explicit, so the design we end up with is a choice rather than a default.
7Beyond these textbook factors, real-world constraints often drive the choice. Sometimes it's constrained by what data already exist. If health facilities have been collecting electronic records for years, retrospective designs become attractive. Ethical constraints rule out some options entirely: we can't randomize mosquito net use, because withholding a life-saving intervention is unethical. Political and security realities can make prospective follow-up impossible, and building on a running surveillance system costs far less than launching something new. The team's skills matter as well. A cohort requires long-term tracking and retention; a case-control study requires carefully designed control selection. Lacking experience with one approach is a legitimate reason to choose another.
8Now to reading. The decisions that determine whether an observational study is credible mostly happen before any results appear: in how the comparison group was defined, how exposures were measured, how confounding was handled, and whether loss to follow-up was taken seriously. Reading a study well means looking for those decisions and not getting swept up in the headline finding.
9In case-control studies, start with the controls. Do the authors define a source population, and could a control have been identified as a case if they had developed the disease? Hospital controls for a community-acquired outcome are a warning sign, as are controls drawn for convenience. If the design is matched, look for conditional logistic regression or a matched analysis. A matched design analyzed with standard logistic regression is a red flag.
10In cohort studies, loss to follow-up is the first thing to check. What percentage of the cohort was lost, and do the authors compare baseline characteristics of those who completed follow-up against those who didn't? Substantial loss with no sensitivity analysis should put us on high alert. Differential loss can produce findings that look like effects but are really artifacts of who stayed in the study.
11In cross-sectional studies, the language tells us whether the authors understand their own design. "Associated with" is honest. "Caused" or "leads to" is not, because cross-sectional data can't establish temporal sequence. When causal language is attached to a snapshot, the authors are overreaching.
12Across all observational designs, the central question is confounding. What did the authors adjust for, and more importantly, what didn't they measure? Look for a discussion of unmeasured confounding and not silence about it. Be wary of long lists of adjustment variables without a conceptual framework, which often signals fishing. And ask whether the effect sizes make biological sense. Implausibly large effects from observational data more often reflect residual confounding than true causal mechanisms.
13The physicians who reported those first cases of Pneumocystis pneumonia and a rare cancer couldn't randomize. They couldn't wait. They couldn't control every confounder. But they could observe, document, and write down what they saw, and those case reports launched a chain of observational research that defined how a new disease behaved, who it affected, and how it spread, long before any treatment trial was possible.
14Every observational design in this chapter involves trade-offs. Cross-sectional studies sacrifice temporal clarity for speed. Case-control studies work backward from outcomes, efficient but vulnerable to recall bias. Cohort studies establish the temporal sequence that causal inference demands, at a cost in time and resources that isn't always available. None of these designs eliminates confounding, and all require assumptions. The difference between a credible observational study and a misleading one is whether the investigators made those assumptions explicit, tested them where possible, and reported the limitations honestly.
15In our session on causal inference, we introduced two strategies: confounder control and design-based approaches. The quasi-experimental designs exploit exogenous variation. The observational designs in this chapter measure and adjust for the things that confound the comparison. Both strategies have limits. The best evidence often comes from combining them: a cohort study with careful confounder adjustment, triangulated against a natural experiment that sidesteps the confounding entirely. When two imperfect approaches, with different assumptions and different vulnerabilities, converge on the same answer, that's when confidence in a causal claim grows.