Chapter 12 · Video 6

Choosing and reading an observational design

15 slides · Video page · All videos · Transcript
Print layout
Slide 1

Choosing and reading an observational design

Slide 2
Choosing a design

Start with the kind of question we are asking

Descriptive: case reports, surveillance, cross-sectional designs
Associations: cross-sectional, case-control, or cohort studies
Causal: case-control or cohort, with careful attention to confounding
Slide 3
Choosing a design

Rare outcomes, and how soon we need answers

Rare outcomes push toward case-control studies
Answers in weeks: case-control or cross-sectional
Years available: a prospective cohort
Long-term outcomes without the wait: a retrospective cohort
Slide 4
Choosing a design

Resources and the kind of exposure

Limited budgets favor cross-sectional and case-control designs
Past exposures with rare outcomes point to case-control
Exposures that vary over time call for longitudinal or panel designs
Slide 5

Three studies from this chapter, and why each design fit

Cholera in Yemen: case-control, because answers were needed in weeks
ART outcomes in South Africa: retrospective cohort, because records held years of data
Breastfeeding in Ethiopia: cross-sectional, because the question was descriptive
Slide 6

The design should be a choice rather than a default

The factors do not always converge on a single answer
The point is to make the trade-offs explicit
Slide 7

Constraints beyond the textbook decision factors

What data already exist, and what infrastructure is already running
Ethics: some options, like withholding bed nets, are ruled out
Political and security realities
The team's own skills
Slide 8
Reading critically

Credibility is decided before any results appear

How the comparison group was defined and exposures were measured
How confounding was handled, and whether loss to follow-up was taken seriously
Slide 9
Case-control

In a case-control study, start with the controls

Is there a defined source population for the controls?
Hospital controls for a community-acquired outcome are a warning sign
A matched design should have a matched analysis
Slide 10
Cohort

In a cohort study, check loss to follow-up first

What percentage was lost?
Do the authors compare those who completed follow-up with those who did not?
Differential loss can look like an effect
Slide 11
Cross-sectional

In a cross-sectional study, read the language

"Associated with" is honest
"Caused" or "leads to" is overreaching for a snapshot
Slide 12
Every design

Across all designs, what did the authors not measure?

Look for a discussion of unmeasured confounding
Long adjustment lists without a conceptual framework suggest fishing
Implausibly large effects more often reflect residual confounding
Slide 13
The physicians who reported the first cases in 1981
The order in which evidence about HIV arrived, from case reports to a randomized trial.
Slide 14

Every observational design involves trade-offs

Cross-sectional studies trade temporal clarity for speed
Case-control studies are efficient but vulnerable to recall bias
Cohort studies establish temporal sequence at a cost in time and resources
Slide 15
In Closing

When two imperfect approaches converge on the same answer

Confounder control and design-based approaches both have limits
Confidence in a causal claim grows when they agree

Choosing and reading an observational design

Slide 1We have a research question, and we've determined that a randomized trial isn't feasible. How do we choose among the observational options? And when we read a published observational study, how do we judge whether it's credible?
Choosing a design

Start with the kind of question we are asking

Descriptive: case reports, surveillance, cross-sectional designs
Associations: cross-sectional, case-control, or cohort studies
Causal: case-control or cohort, with careful attention to confounding
Slide 2Start with the kind of question. Descriptive questions about how common something is, or what patterns exist, point toward case reports, surveillance, and cross-sectional designs. Questions about associations between exposures and outcomes can be tackled with cross-sectional, case-control, or cohort studies. Causal questions push us toward case-control or cohort designs with careful attention to confounding, and even then, the evidence is weaker than a randomized trial or quasi-experiment would provide.
Choosing a design

Rare outcomes, and how soon we need answers

Rare outcomes push toward case-control studies
Answers in weeks: case-control or cross-sectional
Years available: a prospective cohort
Long-term outcomes without the wait: a retrospective cohort
Slide 3Then consider the outcome. Rare outcomes push us toward case-control studies, because we'd need enormous cohorts to capture enough cases. Common outcomes can be studied efficiently with cohort or cross-sectional designs. Timing matters next. If we need answers in weeks, case-control or cross-sectional designs are the realistic options. If we can wait years, a prospective cohort gives the cleanest temporal sequence. And if we want long-term outcomes without the wait, a retrospective cohort can deliver decades of follow-up in months of chart review, provided the records exist and capture what we need.
Choosing a design

Resources and the kind of exposure

Limited budgets favor cross-sectional and case-control designs
Past exposures with rare outcomes point to case-control
Exposures that vary over time call for longitudinal or panel designs
Slide 4Resources and exposure characteristics fill in the rest. Limited budgets favor cross-sectional and case-control designs over prospective cohorts that demand years of follow-up infrastructure. Past exposures with rare outcomes point to case-control. Current exposures we want to follow forward point to a prospective cohort. And exposures that vary over time call for longitudinal or panel designs that can capture the variation.

Three studies from this chapter, and why each design fit

Cholera in Yemen: case-control, because answers were needed in weeks
ART outcomes in South Africa: retrospective cohort, because records held years of data
Breastfeeding in Ethiopia: cross-sectional, because the question was descriptive
Slide 5We've already seen these factors at work. The cholera outbreak in Yemen needed answers within weeks during active conflict, and cases were already presenting to treatment centers, so a case-control study was the realistic choice. The question about long-term ART outcomes had clinical records going back years, so a retrospective cohort fit. And regional patterns in breastfeeding across Ethiopia were a descriptive question with survey infrastructure already in place, so a cross-sectional survey was the right tool.

The design should be a choice rather than a default

The factors do not always converge on a single answer
The point is to make the trade-offs explicit
Slide 6These factors don't always converge on a single answer, and they shouldn't. The point is to make the trade-offs explicit, so the design we end up with is a choice rather than a default.

Constraints beyond the textbook decision factors

What data already exist, and what infrastructure is already running
Ethics: some options, like withholding bed nets, are ruled out
Political and security realities
The team's own skills
Slide 7Beyond these textbook factors, real-world constraints often drive the choice. Sometimes it's constrained by what data already exist. If health facilities have been collecting electronic records for years, retrospective designs become attractive. Ethical constraints rule out some options entirely: we can't randomize mosquito net use, because withholding a life-saving intervention is unethical. Political and security realities can make prospective follow-up impossible, and building on a running surveillance system costs far less than launching something new. The team's skills matter as well. A cohort requires long-term tracking and retention; a case-control study requires carefully designed control selection. Lacking experience with one approach is a legitimate reason to choose another.
Reading critically

Credibility is decided before any results appear

How the comparison group was defined and exposures were measured
How confounding was handled, and whether loss to follow-up was taken seriously
Slide 8Now to reading. The decisions that determine whether an observational study is credible mostly happen before any results appear: in how the comparison group was defined, how exposures were measured, how confounding was handled, and whether loss to follow-up was taken seriously. Reading a study well means looking for those decisions and not getting swept up in the headline finding.
Case-control

In a case-control study, start with the controls

Is there a defined source population for the controls?
Hospital controls for a community-acquired outcome are a warning sign
A matched design should have a matched analysis
Slide 9In case-control studies, start with the controls. Do the authors define a source population, and could a control have been identified as a case if they had developed the disease? Hospital controls for a community-acquired outcome are a warning sign, as are controls drawn for convenience. If the design is matched, look for conditional logistic regression or a matched analysis. A matched design analyzed with standard logistic regression is a red flag.
Cohort

In a cohort study, check loss to follow-up first

What percentage was lost?
Do the authors compare those who completed follow-up with those who did not?
Differential loss can look like an effect
Slide 10In cohort studies, loss to follow-up is the first thing to check. What percentage of the cohort was lost, and do the authors compare baseline characteristics of those who completed follow-up against those who didn't? Substantial loss with no sensitivity analysis should put us on high alert. Differential loss can produce findings that look like effects but are really artifacts of who stayed in the study.
Cross-sectional

In a cross-sectional study, read the language

"Associated with" is honest
"Caused" or "leads to" is overreaching for a snapshot
Slide 11In cross-sectional studies, the language tells us whether the authors understand their own design. "Associated with" is honest. "Caused" or "leads to" is not, because cross-sectional data can't establish temporal sequence. When causal language is attached to a snapshot, the authors are overreaching.
Every design

Across all designs, what did the authors not measure?

Look for a discussion of unmeasured confounding
Long adjustment lists without a conceptual framework suggest fishing
Implausibly large effects more often reflect residual confounding
Slide 12Across all observational designs, the central question is confounding. What did the authors adjust for, and more importantly, what didn't they measure? Look for a discussion of unmeasured confounding and not silence about it. Be wary of long lists of adjustment variables without a conceptual framework, which often signals fishing. And ask whether the effect sizes make biological sense. Implausibly large effects from observational data more often reflect residual confounding than true causal mechanisms.
The physicians who reported the first cases in 1981
The order in which evidence about HIV arrived, from case reports to a randomized trial.
Slide 13The physicians who reported those first cases of Pneumocystis pneumonia and a rare cancer couldn't randomize. They couldn't wait. They couldn't control every confounder. But they could observe, document, and write down what they saw, and those case reports launched a chain of observational research that defined how a new disease behaved, who it affected, and how it spread, long before any treatment trial was possible.

Every observational design involves trade-offs

Cross-sectional studies trade temporal clarity for speed
Case-control studies are efficient but vulnerable to recall bias
Cohort studies establish temporal sequence at a cost in time and resources
Slide 14Every observational design in this chapter involves trade-offs. Cross-sectional studies sacrifice temporal clarity for speed. Case-control studies work backward from outcomes, efficient but vulnerable to recall bias. Cohort studies establish the temporal sequence that causal inference demands, at a cost in time and resources that isn't always available. None of these designs eliminates confounding, and all require assumptions. The difference between a credible observational study and a misleading one is whether the investigators made those assumptions explicit, tested them where possible, and reported the limitations honestly.
In Closing

When two imperfect approaches converge on the same answer

Confounder control and design-based approaches both have limits
Confidence in a causal claim grows when they agree
Slide 15In our session on causal inference, we introduced two strategies: confounder control and design-based approaches. The quasi-experimental designs exploit exogenous variation. The observational designs in this chapter measure and adjust for the things that confound the comparison. Both strategies have limits. The best evidence often comes from combining them: a cohort study with careful confounder adjustment, triangulated against a natural experiment that sidesteps the confounding entirely. When two imperfect approaches, with different assumptions and different vulnerabilities, converge on the same answer, that's when confidence in a causal claim grows.