I Want To Do a Systematic Review

May 1, 2013 at 10:36 am

“I need to do a systematic review.”

“I want a systematic review.

The refrain is increasingly common, but it’s not always clear what someone means by a systematic review. Before our reference librarians will start on a “systematic review” literature search, they will work with you to make sure that what you really need–and want–is a systematic review.

So, what is a systematic review?

In basic terms, a systematic review is a protocol-driven, comprehensive literature review, usually designed to answer a specific clinical question.

What’s different about a systematic review than a regular literature review?

  • Systematic reviews generally answer very focused, PICO-based questions.
  • Systematic reviews have a protocol in place prior to the literature review beginning, including:
    • The clinical question
    • Specific inclusion and exclusion criteria
    • Methods for assessing bias
    • Methods for combining the data (e.g., via a meta-analysis)
  • A systematic review literature search is pre-specified and designed to find all relevant materials; a literature review does not follow a pre-specified protocol, nor does it need to be truly “comprehensive”.
  • A systematic literature search attempts to reduce bias by searching in all languages; searching across multiple databases, including subject-based and regional databases; and searching “grey literature” sources like clinical trial repositories, the web, and unpublished trials.
  • Systematic review protocols generally dictate that two or more individuals independently review each retrieved article separately to determine whether it meets inclusion criteria. Conflicts are solved by a third party.  A literature review, on the other hand, could be done by a single person.
  • Systematic reviews commonly produce large amounts of search results, sometimes even in the tens of thousands, because they are designed for high sensitivity. A typical literature review would aim primarily for specificity–that is, all of the results would be focused on the topic.
  • Systematic reviews publish, as part of their methods sections, the details of the search strategy. Systematic review literature searches are ultimately replicable; the search strategies (including database names and platforms, dates of the search, all search terms, and any limits used) are published so that others who want to redo the searches can find the same information.
  • Regular literature review searches do not need to be replicable or even reported, though on occasion, you may see a published search strategy or a list of keywords the author used to search as part of a regular review’s methodology. Just because a literature review does not publish a list of search terms (which is not particularly helpful without the logic behind it) or a full strategy does not make it a bad or “loosey-goosey” search–it just means the authors did not need to make the search replicable for a systematic review.
  • Systematic reviews are often the basis for a meta-analysis, where the data from the materials fitting the pre-specified criteria are pooled and statistically analyzed. Traditional literature reviews do not apply additional statistical methods to the materials found.
  • Systematic reviews take exponentially more time to do, from the search strategy creation itself, to going through each retrieved citation in duplicate or triplicate, to analyzing the data from the included articles.

Before you begin a systematic review, ask yourself:

  • Do I have a clearly defined clinical question with established inclusion and exclusion criteria?
  • Do I have a team of at least three people assembled?
  • Do I have time to go through as many search results as we might find?
  • Do I have resources to get foreign-language articles appropriately translated?
  • Do I have the statistical resources to analyze and pool data?

If you answered “No” to any of the first four questions, a traditional literature review will be more appropriate to do. If you answered “No” to the last question, a meta-analysis will not be an appropriate methodology for your review.

Systematic Review Standards: Include a Librarian on Your Team

The Institute of Medicine recently issued standards for systematic review teams, including a set of standards specifically for conducting searches. The first standard (3.1.1) for searching states that systematic review teams should work with a librarian to plan the search strategy. The third searching standard (3.1.3) also states that a second librarian should peer review the strategy.

This peer review step is a relatively new concept for systematic review searches, but is one that many of the Mayo Clinic librarians have been using amongst themselves when working on systematic review searches. A peer reviewer might be able to easily spot a mistake in the Boolean logic or a typo that the original searcher might have missed.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement, which has been adopted by many major medical journals, including BMJ, JAMA, and many others, includes two reporting items relevant to the search:

  • Information sources: Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched
  • Search: Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.

Librarians can help you understand which information sources are appropriate to search, conduct the search, and write the methods section pertaining to the search.

Because the heart of any systematic review is the literature search, librarians should be offered authorship or at minimum, and with permission of the librarian, acknowledged for their efforts in the manuscript. A systematic review literature search is a major intellectual undertaking, and can take several weeks to produce, depending on the topic.

The Mayo Clinic Libraries and Systematic Reviews

The reference librarians at the Mayo Clinic libraries have experience with systematic review searches, and would be happy to help any Mayo Clinic staff member or student contemplating a systematic review.

Want some examples? Here are 10 of the most recent systematic reviews with Mayo Clinic librarian authors:

  1. Deo SV, Shah IK, Dunlay SM, Erwin PJ, Dillon JM, Park SJ. Myocardial Revascularisation in Renal Dysfunction: A Systematic Review and Meta-Analysis. Heart Lung Circ. 2013 Mar 30. doi:pii: S1443-9506(13)00098-X. 10.1016/j.hlc.2013.03.005. [Epub ahead of print] PubMed PMID: 23548338.
  2. Deo SV, Dunlay SM, Shah IK, Altarabsheh SE, Erwin PJ, Boilson BA, Park SJ, Joyce LD. Dual Anti-platelet Therapy After Coronary Artery Bypass Grafting: Is There Any Benefit? A Systematic Review and Meta-Analysis. J Card Surg. 2013 Mar;28(2):109-16. doi: 10.1111/jocs.12074. PubMed PMID: 23488578.
  3. Tleyjeh IM, Abdulhak AA, Riaz M, Garbati MA, Al-Tannir M, Alasmari FA, Alghamdi M, Khan AR, Erwin PJ, Sutton AJ, Baddour LM. The Association between Histamine 2 Receptor Antagonist Use and Clostridium difficile Infection: A Systematic Review and Meta-analysis. PLoS One. 2013;8(3):e56498. doi: 10.1371/journal.pone.0056498. Epub 2013 Mar 4. PubMed PMID: 23469173; PubMed Central PMCID: PMC3587620.
  4. Rubin MN, Wellik KE, Channer DD, Demaerschalk BM. A systematic review of telestroke. Postgrad Med. 2013 Jan;125(1):45-50. doi:  10.3810/pgm.2013.01.2623. Review. PubMed PMID: 23391670.
  5. Deo SV, Shah IK, Dunlay SM, Erwin PJ, Locker C, Altarabsheh SE, Boilson BA, Park SJ, Joyce LD. Bilateral internal thoracic artery harvest and deep sternal wound infection in diabetic patients. Ann Thorac Surg. 2013 Mar;95(3):862-9. doi: 10.1016/j.athoracsur.2012.11.068. Epub 2013 Jan 24. PubMed PMID: 23352296.
  6. Rank MA, Hagan JB, Park MA, Podjasek JC, Samant SA, Volcheck GW, Erwin PJ, West CP. The risk of asthma exacerbation after stopping low-dose inhaled corticosteroids: a systematic review and meta-analysis of randomized  controlled trials. J Allergy Clin Immunol. 2013 Mar;131(3):724-9. doi: 10.1016/j.jaci.2012.11.038. Epub 2013 Jan 12. PubMed PMID: 23321206.
  7. Prutsky GJ, Domecq JP, Elraiyah T, Wang Z, Grohskopf LA, Prokop LJ, Montori VM, Murad MH. Influenza vaccines licensed in the United States in healthy children: a systematic review and network meta-analysis (Protocol). Syst Rev. 2012 Dec 29;1(1):65. doi: 10.1186/2046-4053-1-65. PubMed PMID: 23272706; PubMed Central PMCID: PMC3537683.
  8. Singh B, Murad MH, Prokop LJ, Erwin PJ, Wang Z, Mommer SK, Mascarenhas SS, Parsaik AK. Meta-analysis of Glasgow coma scale and simplified motor score in predicting traumatic brain injury outcomes. Brain Inj. 2013;27(3):293-300. doi: 10.3109/02699052.2012.743182. Epub 2012 Dec 19. PubMed PMID:  23252405.
  9. Tleyjeh IM, Bin Abdulhak AA, Riaz M, Alasmari FA, Garbati MA, AlGhamdi M,  Khan AR, Al Tannir M, Erwin PJ, Ibrahim T, Allehibi A, Baddour LM, Sutton AJ. Association between proton pump inhibitor therapy and clostridium difficile infection: a contemporary systematic review and meta-analysis. PLoS One. 2012;7(12):e50836. doi: 10.1371/journal.pone.0050836. Epub 2012 Dec 7.  PubMed PMID: 23236397; PubMed Central PMCID: PMC3517572.
  10. Rubin MN, Wellik KE, Channer DD, Demaerschalk BM. Systematic review of teleneurology: methodology. Front Neurol. 2012;3:156. doi: 10.3389/fneur.2012.00156. Epub 2012 Nov 8. PubMed PMID: 23162527;  PubMed Central PMCID: PMC3497715.

For more, see a complete list in Mayo Authors (Mayo Clinic only).

More Resources:


Melissa L. Rethlefsen, MLS, AHIP
Learning Resource Center

Entry filed under: What's Your Reference Question?. Tags: , .

Mayo Clinic Family Health Book NLM Updates

%d bloggers like this: