Video 2 of 5

Start with what someone already synthesized

Cochrane, Campbell, and WHO guidelines are places the work has already been done. The evidence hierarchy, and the qualification that matters: the best evidence depends on the question, and a poorly conducted trial does not beat a rigorous case-control study.

4:37 · 12 slides · printable slides · transcript

Slides

Printable deck →

Start with what someone already synthesized

1 / 12
Transcript12 sections

Generated from the narration script. Plain text version.

1This video is about the work you should do before you build a search: finding out whether someone has already synthesized the evidence, and then exploring quickly enough to know what you are dealing with.

2Before you begin building complex search strings, check whether someone has already done the work. If a high-quality systematic review exists on your topic, start there. Why wade through hundreds of primary studies when someone has already identified, appraised, and synthesized them for you?

3The Cochrane Library is the gold standard for systematic reviews of healthcare interventions. Cochrane reviews follow rigorous methods and are regularly updated, and if a Cochrane review exists on your topic, it should be your first stop. That rigorous methodology is what makes these reviews trustworthy. But be aware that Cochrane traditionally focuses on randomized controlled trials, which may not capture the implementation and contextual evidence you need.

4The Campbell Collaboration is like Cochrane, focused on social interventions in areas including education, crime and justice, and social welfare. It's particularly useful when your research question spans health and social domains. And the World Health Organization synthesizes evidence to create guidelines that shape practice globally.

5One classic text on evidence-based medicine offers this wisdom. Always start at the top of the evidence pyramid. If someone has already synthesized the evidence, begin there. You've probably seen versions of the pyramid: systematic reviews and meta-analyses at the top, followed by randomized controlled trials, cohort studies, case-control studies, case series, and expert opinion at the bottom.

6The idea is that higher levels represent more reliable evidence because they're less susceptible to bias. This hierarchy is useful as a starting point. But don't take it too literally. The best evidence for your specific question depends on your specific question. If you're asking about the average effect of an intervention across populations, a systematic review of randomized controlled trials might be ideal. If you're asking how to implement that intervention in your specific context, a qualitative study from a similar setting might be more informative than a meta-analysis of trials from high-income countries.

7When you find a relevant systematic review, you have in your hands a roadmap to the literature. The reference list is a curated bibliography of relevant primary studies. Check when the review was conducted, and search for newer studies that might have been published since. Google Scholar makes this easy with the Cited by feature. Keep up this approach with every high quality study you find, and you'll have a good sense of the literature on your topic.

8Sometimes you need to get oriented quickly. You're brainstorming a new project, checking whether an idea is feasible, or trying to understand what literature exists before committing to a formal review. This is where quick and dirty searching comes in. I use that term deliberately. These searches are exploratory. They're designed to give you a sense of the landscape, and a complete map of it is a different job.

9The goal of exploratory searching is to answer questions like: has anyone studied this topic before? What terminology do researchers use, who are the key authors, what journals publish on this topic, and what are the main debates or controversies? Where are the obvious gaps? You're trying to learn enough to decide whether to proceed and, if so, how to structure a more rigorous search.

10Google Scholar is remarkably useful for exploration. It indexes broadly, includes grey literature and books that PubMed misses, and its Cited by and Related articles features are powerful tools for discovering relevant literature. Start broad and then narrow. Once you find a seminal paper, the Cited by feature is gold for finding recent work that builds on foundational studies. A foundational paper with 500 citations essentially provides a curated reading list of the field's development.

11But beware. Google Scholar doesn't tell you what it's searching, doesn't use controlled vocabulary, and may include predatory journals in the results alongside legitimate ones. It's a discovery tool, not a comprehensive search tool. Never rely on it alone for a formal review.

12Exploratory searching can become a rabbit hole. You find one interesting paper, which leads to five more, which lead to twenty more, and suddenly you've spent three days reading and haven't written a word. There are two signs you've searched enough for exploratory purposes. Redundancy: you keep finding the same papers cited repeatedly, and new searches return mostly papers you've already seen. And conceptual clarity: you can articulate the main debates, gaps, and approaches in the field. It's important to accept that science is iterative. You'll search, read, and think on repeat.

Evidence SynthesisCochraneEvidence HierarchyGoogle Scholar