Summary
If you want to ascertain the effects of a diet or supplement, looking at one study won’t suffice. Studies can differ in their findings because they had different designs, dosages, populations, or methods of data analysis or interpretation, among other reasons. Review articles aim to summarize the state of the evidence on a given topic by covering all the relevant studies.
But what makes a study “relevant”? How will you gather all the relevant studies? And how will you combine their findings into a single conclusion? Creating a review requires making a lot of choices, some of which are legitimate while others introduce bias — for example, a company selling a given supplement could review only the studies that showed an effect.
A systematic review aims to minimize bias by laying out a clear procedure for finding relevant papers and reporting all the relevant results with all the relevant details. Ideally, it presents its procedure so clearly that other researchers can reproduce it and obtain the same results. It also lays out any assumptions made during the process of review and interpretation so as to allow for productive scientific debate about its results. All this makes a systematic review generally more reliable than other types of reviews.
Reviews aim to summarize the state of the evidence on a given topic. Systematic reviews try to be comprehensive, rigorous, clear, and free of bias; they detail their procedures and so are generally more reliable than other types of reviews.
Many factors can make a scientific review more or less reliable. In this short entry, we’ll mention just two: preregistration and PRISMA standards.
Preregistration allows the authors of a systematic review to present their procedure publicly before they start the review process. It forces them to really think about their procedure ahead of time and creates a public record of what they intend to do, which encourages them to stick to their plan or justify any departure from it. There are many places where you can preregister a systematic review, but the most common database is PROSPERO.
Just as preregistration makes a systematic review more reliable, so does its following PRISMA standards. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is widely recognized as representing the minimum set of items a systematic review needs in order to be considered unbiased, transparent, and reproducible.
A systematic review can be considered more reliable if it was preregistered (i.e., if its methods were publicly laid out ahead of time) and if it followed the PRISMA standards.