Video 3 of 5
Emerging AI tools, and being wary of fake science
AI discovery tools and predatory journals are the same failure wearing different clothes: something that looks like a paper and is not. What Elicit, Consensus, and Scite are good for, what hallucinated citations cost, and how to spot a predatory journal.
4:55 · 12 slides · printable slides · transcript
Slides
Printable deck →▶Transcript12 sections
Generated from the narration script. Plain text version.
1This video covers two things: the emerging AI tools that are changing how researchers discover literature, and how to be wary of fake science, including the journals that will publish anything for a fee.
2AI tools are changing how researchers discover literature, and creating new ways to get into trouble. Used wisely, they can accelerate discovery. Used carelessly, they'll lead you to cite papers that don't exist.
3What's available. Tools like Elicit, Consensus, and Scite are designed specifically for research discovery. They let you ask questions in natural language and return relevant papers with AI-generated summaries. General-purpose tools like ChatGPT and Perplexity can also help you explore topics, though they weren't built for systematic literature work.
4What they're good for. Rapid orientation: ask what the main approaches to treating severe acute malnutrition are, and get a structured overview faster than hours of reading. Surfacing papers that use different terminology than you'd think to search for. Summarizing findings across multiple studies to reveal patterns. And generating search terms when you're stuck on keywords or subject headings.
5Why they're dangerous. No AI tool searches all databases systematically. They're discovery tools, and they don't replace proper database searching. AI tools can and do hallucinate citations, inventing plausible-sounding papers that don't exist. Never cite a paper you found through AI without verifying it exists and says what the AI claimed. Training data has cutoffs, so the newest literature may not be indexed. And unlike database searches, you can't fully document what an AI searched or why it returned certain results, which is a problem for reproducible reviews.
6It's best practice to verify everything. If an AI suggests a paper, find it in PubMed or Google Scholar and read it yourself; if you can't find it, it may not exist. Use AI for discovery, but databases for rigor. Document your process by noting which tools you used and how, even if you can't fully reproduce the outputs. And don't trust AI summaries, especially if you have not provided the article PDF. They can miss nuance, misstate findings, or reflect training biases rather than what the paper actually says.
7The researchers who will use these tools most effectively are those who understand both their power and their limitations. AI probably won't replace careful scholarship, but it can make careful scholars more efficient. Be quick, not dirty. And stay current: these tools improve rapidly, so what's unreliable today may be better tomorrow, and vice versa.
8Humans can mislead you too. That's because not everything that looks like science is good science. As you search the literature, you'll encounter predatory journals, poorly conducted studies, and sometimes outright fraud. Predatory journals mimic legitimate academic journals but exist primarily to collect publication fees without providing genuine peer review or editorial services. They look like real journals, but behind the facade there's no real peer review, no editorial oversight, and no quality control.
9If you have an academic email address, you've probably received solicitations from predatory journals. The emails often flatter: Dear Distinguished Professor. They claim to have seen your recent work: we were impressed by your publication. And they urgently invite submission: special discount if you submit within 48 hours.
10These emails can be persuasive, especially for early-career researchers eager to build their publication records. The journals often have names that sound similar to legitimate journals, the Journal of Medical Sciences against the Journal of Medical Science, making it easy to confuse them with reputable venues.
11Some predatory publishers have become sophisticated. They organize fake conferences, create fake impact metrics, and establish fake indexing services to lend credibility to their operations. The ecosystem of predatory publishing is now self-reinforcing. Predatory journals cite each other to inflate citation counts, predatory conferences invite authors to present papers later published in predatory journals, and predatory indexing services include predatory journals to make them appear legitimate.
12A few cross-checks usually reveal whether a journal is legitimate. The Directory of Open Access Journals only indexes journals meeting quality criteria, so if an open access journal isn't in it, investigate further. Check whether the journal is indexed in PubMed, Scopus, or Web of Science. Indexing isn't a guarantee of quality, but lack of indexing in any major database is a warning sign. Look up the publisher by searching the name plus predatory or scam. Examine the editorial board: are the editors real people with verifiable academic affiliations, and can you find their work in legitimate venues? Finally, read a few published articles: do they look professionally edited, and is the science plausible?