Read the methods first, and find where bias enters
Chapter 5: How to Read Scientific Articles — Video 2
https://ghrbook.com/videos/read-the-methods-first/

[Slide 1] Critical appraisal means systematically assessing whether a study's methods, results, and conclusions are trustworthy, meaningful, and relevant. It is about understanding what a study can and cannot tell us. And the Methods section is where a lot of critical appraisal happens.
[Slide 2] I want you to try something counterintuitive. The next time you pick up a paper, skip the abstract and introduction entirely. Go straight to Methods. Read it carefully. Only then go back and read the rest of the paper. You'll be amazed how differently everything else reads when you already understand what the researchers actually did. There are six foundational questions to ask of every Methods section, and here they are.
[Slide 3] First. What is the research question? Can you articulate the study's central question in one sentence? If you can't, that's a problem, either with the paper's clarity or with your understanding. Don't move forward until you can state what the study is trying to learn. Look for the question in the final paragraph of the introduction, the first paragraph of methods, or sometimes labeled as objective. Many papers state objectives, but you can convert them: the objective was to evaluate the effect of X on Y becomes, does X affect Y? And practice articulating them in plain language. Do poorer patients have worse adherence when HIV care is delivered differently?
[Slide 4] Second. Does the study design match the question? At this stage you're not evaluating design details, just checking basic alignment. If the research question asks whether an intervention causes an outcome, is the design capable of addressing causation? If the question explores how patients experience a phenomenon, is the methodology suited to capturing subjective experience? Watch for mismatches. A cross-sectional survey can't tell you whether an intervention caused outcomes, no matter how sophisticated the statistical adjustment. A qualitative study with three participants can't tell you about prevalence.
[Slide 5] Third and fourth. Who are the participants, and what comparisons are being made? There is the study sample, who participated, and then there is the larger group the authors are trying to say something about. Ask about the sampling strategy, the inclusion and exclusion criteria, where and how participants were found, and how similar they are to the population you care about. Then the comparison. Treatment against control. Exposed against unexposed. Before against after. Different doses. The comparison structure determines the causal logic, and a study comparing patients who received an intervention to patients who didn't can only support causal claims if the groups are otherwise comparable. Pay attention to what makes groups different, beyond the thing being studied.
[Slide 6] Fifth and sixth. How were outcomes measured, and are the methods transparent? How are key outcomes defined? Is treatment success defined the same way you would define it? Are measurements valid, meaning they measure what they claim to, and reliable, meaning consistent across time and raters? Are outcomes self-reported or objectively measured? How was missing data handled? Then look for signs the researchers made their decision-making visible. Was the study pre-registered? Is there a flow diagram showing recruitment and retention? Is attrition reported? Are ethics approvals and conflicts of interest disclosed? Are the data and the analysis code available? Transparency doesn't guarantee quality, but its absence is a warning sign.
[Slide 7] Now the question that sits underneath all six. Where could bias enter? Bias sounds like an accusation, but in research it's a technical term for systematic error that threatens validity. There are three to watch for. Confounding, when a third variable affects both the exposure and the outcome, creating a spurious association. Imagine a study in which smartphone ownership predicts child survival, but wealth causes both. Selection bias, when participation in the study is related to both the exposure and the outcome. A study of exercise and depression recruits from gyms, but people who go to gyms are already less depressed on average. And measurement bias, when the measurement of exposure or outcome is systematically inaccurate.
[Slide 8] The first two trip people up, because different disciplines use them differently and they produce similar-looking problems. Confounding is fundamentally about what you didn't account for in your analysis. A third variable causes both the exposure and the outcome. The classic example: coffee drinking is associated with lung cancer, but both are caused by smoking. Coffee doesn't cause cancer; smoking confounds the relationship. And confounding can often be addressed through statistical adjustment, if you measured the confounder. Selection bias is fundamentally about who ended up in your study. If the process that determines study participation is related to both the exposure and outcome, your sample won't represent the population you're trying to study. The classic example: studying the relationship between occupation and health using hospital patients. Certain occupations might make people more likely to seek hospital care regardless of their actual health, which distorts the relationship. Selection bias can't be fixed by statistical adjustment. The problem is baked into who you're studying.
[Slide 9] Here's a practical distinction. If you're worried you didn't measure or adjust for something important, that's confounding. If you're worried the wrong people ended up in the study, or completed the study, that's selection bias. One caution when you read across disciplines: economists sometimes use selection bias more broadly, to include what epidemiologists call confounding. Pay attention to how authors define their terms.
[Slide 10] You don't need to identify every possible bias. You're developing pattern recognition, and the more papers you read, the more easily you'll spot where bias could creep in. So far these are six questions on a slide. In the next video I take a real paper, a malaria vaccine study running across three countries, and ask all six of them out loud, including the parts I didn't understand on the first pass.