When should systematic reviews be replicated, and when is it wasteful?

Workshop category: 

  • Priority setting
Date and Location


Tuesday 22 October 2019 - 16:00 to 17:30


Contact persons and facilitators

Contact person:


Welch V1, Grimshaw J2, Rada G3, Smith M4, Pardo Pardo J5, Soares-Weiser K6, Tugwell P7
1 Bruyere Research Institute, Canada
2 Ottawa Hospital Research Institute, Canada
3 Cochrane Chile, Chile
4 Cochrane Consumer, Canada
5 Cochrane Musculoskeletal, Canada
6 Cochrane, UK
7 University of Ottawa, Canada


Karunananthan S1
1 Ottawa Hospital Research Institute, Canada
Target audience

Target audience: 

Priority-setting groups, review authors, funders, journal editors, consumers

Level of difficulty: 

Type of workshop

Type of workshop : 



Background: replication is a corner stone of the scientific method yet replication of systematic reviews is too often overlooked, done unnecessarily or done poorly. Failure to replicate may lead to continued uncertainty about the implications of a body of evidence. Too frequent replication can represent an inefficient use of scarce systematic review resources and may generate noise to signal. There is a lack of guidance for when and how to replicate systematic reviews.

Objectives: to present evidence-informed, consensus-based guidance on when to, and when not to replicate systematic reviews, and to discuss strategies for disseminating and implementing these recommendations to key stakeholder groups.

Description: we propose that systematic review replication is conducted with the objective of testing whether the results of the index review can be repeated or extended, and may or may not include new data, new methods or new analyses. We recently held a consensus meeting of 36 participants representing key stakeholder groups: patients, clinicians, journal editors, researchers, systematic review organizations, and guideline developers. We discussed an extensive list of criteria for systematic review replication drawn from empirical studies, reviews, key informant interviews, and an online survey. In this workshop, we will present:
1) a framework to determine how issues in the conduct of an index review represent threats to validity sufficient to justify formal replication and what are the appropriate review methods to address the specific threat of validity within the replicated systematic review;
2) a checklist applying the value of information (VOI) concept to determine whether the benefits of replicating an existing systematic review outweigh alternative uses of resources.
We will discuss examples of how the replication framework and checklist can be used to guide decisions on when and how to replicate systematic reviews.