What kind of review are you doing?
Slide 1This video is about the decision you make before you type anything into a search box: what kind of review you are actually doing, and what each kind commits you to.
The volume of published research is staggering
PubMed alone contains over 36 million citations
Google Scholar indexes hundreds of millions of documents
And that is only what is published in English, in indexed journals
Slide 2If you've ever felt overwhelmed by the prospect of searching the literature, you're in good company. The sheer volume of published research is staggering. PubMed alone contains over 36 million citations. Google Scholar indexes hundreds of millions of documents. And that's just the stuff that's been published in English in indexed journals. We haven't even talked about grey literature, reports from ministries of health, or the dissertation gathering dust on a shelf in Nairobi that contains exactly the information you need.
Searching the literature is a thinking process
The way you search reflects what you know
And what you don't yet realize you need to know
Slide 3Here's what I want you to understand from the outset. Searching the literature is not a mechanical task you complete and check off a list. It's a thinking process. The way you search reflects what you know, what you think you know, and what you don't yet realize you need to know.
Before you type anything into a search box, answer this
What kind of review are you actually doing?
The expectations for rigor and the time required vary dramatically
So do the methods you will use
Slide 4So before you type anything into a search box, you need to answer a fundamental question. What kind of review are you actually doing? The expectations for rigor, the time required, and the methods you'll use vary dramatically depending on whether you're writing a literature review for a class paper or leading a Cochrane systematic review.
Review type 1 of 4
A systematic review is the gold standard of evidence synthesis
A pre-specified protocol, and reproducible searches across databases
Predetermined inclusion and exclusion criteria
Critical appraisal, then synthesis using rigorous methods
Slide 5A systematic review is the gold standard of evidence synthesis. It follows a pre-specified protocol. It uses explicit and reproducible search strategies across multiple databases. It applies predetermined inclusion and exclusion criteria, critically appraises the quality of the studies it includes, and synthesizes findings using rigorous methods, sometimes including meta-analysis.
When do you need a systematic review?
They take months, a team, and documentation of every decision
Cochrane and Campbell set the methodological standard
Use one when you make claims about what the evidence shows
Slide 6Systematic reviews are hard. They typically take months, require a team, and demand documentation of every decision. The Cochrane Collaboration and the Campbell Collaboration produce systematic reviews that follow strict methodological standards and are recognized worldwide as authoritative sources of evidence. When do you need a systematic review? When you're making claims about what the evidence shows on a clinical or policy question. When you're informing guidelines. When you need to be comprehensive and transparent about how you identified, selected, and synthesized evidence.
Review type 2 of 4
A scoping review maps the literature that exists
Types of evidence available, concepts, and knowledge gaps
No quality assessment, and no effectiveness answer
Useful for a new area, or before designing something focused
Slide 7A scoping review maps the existing literature on a topic to identify the types of evidence available, clarify concepts, and identify knowledge gaps. Scoping reviews don't typically assess the quality of the studies they include, and they don't synthesize findings to answer a specific effectiveness question. They're useful when you're exploring a new area, when the literature is diverse and heterogeneous, or when you want to understand the lay of the land before designing a more focused systematic review.
Review type 3 of 4
A narrative review selects on relevance and expertise
The synthesis is often interpretive and thematic
Right for background and context in a thesis introduction
The danger is cherry-picking
Slide 8A narrative review, sometimes called a literature review or a traditional review, synthesizes and summarizes literature on a topic without following the strict protocols of a systematic review. The author selects sources based on relevance and expertise, and the synthesis is often more interpretive and thematic. That makes narrative reviews appropriate for providing background and context in a thesis introduction, for identifying themes and debates in a field, and for educational purposes. The danger is cherry-picking: selecting only evidence that supports your argument while ignoring contradictory findings.
Review type 4 of 4
A bibliometric analysis asks what the literature looks like
How many papers, in which journals, by which authors
In which countries, citing which work, over what time
The unit of analysis is the publication itself
Slide 9A bibliometric analysis asks a different question from the reviews above. Instead of synthesizing what the literature says about a topic, it analyzes what the literature looks like. How many papers, in which journals, by which authors, in which countries, citing which work, growing or shrinking over time. The unit of analysis is the publication itself, not the finding.
Questions a systematic review cannot easily answer
Who writes about Ebola, and where are they based?
How has interest in climate and health shifted since 2010?
The answer is in authorship and publication patterns
Slide 10Bibliometrics are useful for mapping the structure of a field, identifying influential papers and research clusters, surfacing geographic or institutional concentration in who produces evidence, and tracking how an area is evolving. In global health they're particularly good for asking questions like: who writes about Ebola, and where are they based? Or, how has interest in climate and health shifted since 2010? These are questions a systematic review can't easily answer, because the answer isn't in the studies' findings. It's in their authorship and publication patterns.
The analysis is only as good as the corpus
A structured search, then counts, networks, and keyword maps
VOSviewer and Bibliometrix make the visualizations approachable
The danger is treating publication counts as evidence quality
Slide 11The method typically involves a structured literature search, much like a systematic review, and then quantitative analysis. Counts by year, journal, country, and institution. Co-authorship and citation networks. Keyword co-occurrence maps. The danger is treating publication counts as evidence quality. A topic with many papers might be well-studied, or just fashionable. A country with few papers might be neglected, or just publishing in venues your search missed.
In Closing
What the rest of this chapter is for
When to use which approach, and how to build searches that work
Where to find evidence that does not show up in PubMed
And the traps, from confirmation bias to predatory journals
Slide 12Whichever of these you're doing, the rest of the chapter is about doing it well. When to use which approach. How to build search strategies that actually work. Where to find the evidence that doesn't show up in PubMed. And how to avoid the traps that catch unwary researchers, from confirmation bias to predatory journals.