Chapter 4 · Video 2

Start with what someone already synthesized

12 slides · Video page · All videos · Transcript
Print layout
Slide 1

Start with what someone already synthesized

Slide 2

Check whether someone has already done the work

If a high-quality systematic review exists, start there
It has already identified, appraised, and synthesized the studies
Hundreds of primary papers you do not have to wade through
Slide 3
Pre-appraised evidence

Cochrane, for reviews of healthcare interventions

Rigorous methods, regularly updated, and your first stop
That methodology is what makes the reviews trustworthy
Traditionally focused on trials, so contextual evidence may be thin
Slide 4
Pre-appraised evidence

Campbell, and the WHO guidelines

Campbell: social interventions in education, justice, and welfare
Useful when a question spans health and social domains
WHO synthesizes evidence into guidelines that shape global practice
Slide 5
Greenhalgh
Always start at the top of the evidence pyramid
Evidence pyramid. Source: CFCF (2015). ghr.link/evi. Systematic reviews and meta-analyses at the top; case series and expert opinion at the bottom.
Slide 6

The hierarchy is useful as a starting point

Higher levels are held to be less susceptible to bias
But the best evidence depends on your specific question
For implementation, a qualitative study may be more informative
Slide 7

A relevant review is a roadmap to the literature

Its reference list is a curated bibliography of primary studies
Check when the review was conducted, then search for newer studies
Google Scholar's Cited by feature makes that easy
Slide 8

Quick and dirty searches, for early exploration

Brainstorming a project, or checking whether an idea is feasible
Before committing to a formal review
Designed to give you a sense of the landscape
Slide 9

What exploratory searching is for

Has anyone studied this before? What terminology do they use?
Who are the key authors, and what are the main debates?
Where are the obvious gaps?
Slide 10

Google Scholar is remarkably useful for exploration

It indexes broadly, including grey literature and books
Cited by turns a foundational paper into a reading list
Related articles surfaces work you would never have searched for
Slide 11

Google Scholar is a discovery tool

You cannot see what it is searching
It uses no controlled vocabulary
And it may return predatory journals alongside legitimate ones
Slide 12
In Closing

Two signs you have searched enough

Redundancy: new searches return papers you have already seen
Conceptual clarity: you can articulate the debates and the gaps
Science is iterative — you will search, read, and think on repeat

Start with what someone already synthesized

Slide 1This video is about the work you should do before you build a search: finding out whether someone has already synthesized the evidence, and then exploring quickly enough to know what you are dealing with.

Check whether someone has already done the work

If a high-quality systematic review exists, start there
It has already identified, appraised, and synthesized the studies
Hundreds of primary papers you do not have to wade through
Slide 2Before you begin building complex search strings, check whether someone has already done the work. If a high-quality systematic review exists on your topic, start there. Why wade through hundreds of primary studies when someone has already identified, appraised, and synthesized them for you?
Pre-appraised evidence

Cochrane, for reviews of healthcare interventions

Rigorous methods, regularly updated, and your first stop
That methodology is what makes the reviews trustworthy
Traditionally focused on trials, so contextual evidence may be thin
Slide 3The Cochrane Library is the gold standard for systematic reviews of healthcare interventions. Cochrane reviews follow rigorous methods and are regularly updated, and if a Cochrane review exists on your topic, it should be your first stop. That rigorous methodology is what makes these reviews trustworthy. But be aware that Cochrane traditionally focuses on randomized controlled trials, which may not capture the implementation and contextual evidence you need.
Pre-appraised evidence

Campbell, and the WHO guidelines

Campbell: social interventions in education, justice, and welfare
Useful when a question spans health and social domains
WHO synthesizes evidence into guidelines that shape global practice
Slide 4The Campbell Collaboration is like Cochrane, focused on social interventions in areas including education, crime and justice, and social welfare. It's particularly useful when your research question spans health and social domains. And the World Health Organization synthesizes evidence to create guidelines that shape practice globally.
Greenhalgh
Always start at the top of the evidence pyramid
Evidence pyramid. Source: CFCF (2015). ghr.link/evi. Systematic reviews and meta-analyses at the top; case series and expert opinion at the bottom.
Slide 5One classic text on evidence-based medicine offers this wisdom. Always start at the top of the evidence pyramid. If someone has already synthesized the evidence, begin there. You've probably seen versions of the pyramid: systematic reviews and meta-analyses at the top, followed by randomized controlled trials, cohort studies, case-control studies, case series, and expert opinion at the bottom.

The hierarchy is useful as a starting point

Higher levels are held to be less susceptible to bias
But the best evidence depends on your specific question
For implementation, a qualitative study may be more informative
Slide 6The idea is that higher levels represent more reliable evidence because they're less susceptible to bias. This hierarchy is useful as a starting point. But don't take it too literally. The best evidence for your specific question depends on your specific question. If you're asking about the average effect of an intervention across populations, a systematic review of randomized controlled trials might be ideal. If you're asking how to implement that intervention in your specific context, a qualitative study from a similar setting might be more informative than a meta-analysis of trials from high-income countries.

A relevant review is a roadmap to the literature

Its reference list is a curated bibliography of primary studies
Check when the review was conducted, then search for newer studies
Google Scholar's Cited by feature makes that easy
Slide 7When you find a relevant systematic review, you have in your hands a roadmap to the literature. The reference list is a curated bibliography of relevant primary studies. Check when the review was conducted, and search for newer studies that might have been published since. Google Scholar makes this easy with the Cited by feature. Keep up this approach with every high quality study you find, and you'll have a good sense of the literature on your topic.

Quick and dirty searches, for early exploration

Brainstorming a project, or checking whether an idea is feasible
Before committing to a formal review
Designed to give you a sense of the landscape
Slide 8Sometimes you need to get oriented quickly. You're brainstorming a new project, checking whether an idea is feasible, or trying to understand what literature exists before committing to a formal review. This is where quick and dirty searching comes in. I use that term deliberately. These searches are exploratory. They're designed to give you a sense of the landscape, and a complete map of it is a different job.

What exploratory searching is for

Has anyone studied this before? What terminology do they use?
Who are the key authors, and what are the main debates?
Where are the obvious gaps?
Slide 9The goal of exploratory searching is to answer questions like: has anyone studied this topic before? What terminology do researchers use, who are the key authors, what journals publish on this topic, and what are the main debates or controversies? Where are the obvious gaps? You're trying to learn enough to decide whether to proceed and, if so, how to structure a more rigorous search.

Google Scholar is remarkably useful for exploration

It indexes broadly, including grey literature and books
Cited by turns a foundational paper into a reading list
Related articles surfaces work you would never have searched for
Slide 10Google Scholar is remarkably useful for exploration. It indexes broadly, includes grey literature and books that PubMed misses, and its Cited by and Related articles features are powerful tools for discovering relevant literature. Start broad and then narrow. Once you find a seminal paper, the Cited by feature is gold for finding recent work that builds on foundational studies. A foundational paper with 500 citations essentially provides a curated reading list of the field's development.

Google Scholar is a discovery tool

You cannot see what it is searching
It uses no controlled vocabulary
And it may return predatory journals alongside legitimate ones
Slide 11But beware. Google Scholar doesn't tell you what it's searching, doesn't use controlled vocabulary, and may include predatory journals in the results alongside legitimate ones. It's a discovery tool, not a comprehensive search tool. Never rely on it alone for a formal review.
In Closing

Two signs you have searched enough

Redundancy: new searches return papers you have already seen
Conceptual clarity: you can articulate the debates and the gaps
Science is iterative — you will search, read, and think on repeat
Slide 12Exploratory searching can become a rabbit hole. You find one interesting paper, which leads to five more, which lead to twenty more, and suddenly you've spent three days reading and haven't written a word. There are two signs you've searched enough for exploratory purposes. Redundancy: you keep finding the same papers cited repeatedly, and new searches return mostly papers you've already seen. And conceptual clarity: you can articulate the main debates, gaps, and approaches in the field. It's important to accept that science is iterative. You'll search, read, and think on repeat.