Issues affecting the public health relevance of Cochrane Reviews




Oral session: Global health and equity (2)


Wednesday 23 October 2019 - 11:00 to 12:30


All authors in correct order:

Chong L1, Opiyo N2, Ryan R3, Hartmann-Boyce J4, Thomson H5, Verbeek J6, Hill S3, Wolfenden L7, Jani Ruotsalainen J6, Doyle J7, Lowe D3, Naude C8, Hilton-Boon M5, Bero LA9
1 Cochrane Public Health and Health Systems Network, University of Oxford, UK
2 Cochrane Editorial and Methods Department (EMD), UK
3 Cochrane Consumers and Communication, La Trobe University, Australia
4 Cochrane Tobacco Addiction, University of Oxford, UK
5 Cochrane Public Health, University of Glasgow, UK
6 Cochrane Work Review, Finnish Institute of Occupational Health, Finland
7 Cochrane Public Health, University of Newcastle, Australia
8 Cochrane Nutrition, Stellenbosch University, South Africa
9 Cochrane Public Health and Health Systems Network, University of Sydney, Australia
Presenting author and contact person

Presenting author:

Lee-Yee Chong

Contact person:

Abstract text
Background: Cochrane Reviews are widely seen as trustworthy and informed by rigorous, evidence-based methods. Although these methods are expanding, they were historically developed with a focus on clinical treatments. Cochrane authors and review groups may experience significant challenges with public health and health systems reviews due to the scope of reviews, complexity of interventions, sources of evidence and types of relevant outcomes. Therefore, it is important to identify and document the issues faced whilst conducting these reviews, to anticipate them and address them adequately.

Objectives: to identify key issues impacting the public health relevance of Cochrane Reviews evaluating prevention measures, harms, exposures, policies/programmes and/or their implementation.

Methods: we collated issues from various relevant sources:
1) reviews identified as high priority, potentially controversial or referred by an editor from the Public Health and Health Systems (PHHS) Network;
2) feedback/examples from editors within Cochrane and the PHHS Network;
3) external commentaries on PHHS reviews published in 2018;
4) screening reports by the Cochrane Editorial and Methods Department of PHHS Network reviews.
We categorized the identified issues into themes. We reviewed and discussed these iteratively with the editors within the PHHS Network until we reached agreement on categories.

Results: the common issues affecting the relevance of reviews were grouped into the following topics:
1) review aims/framing of questions;
2) choice of included study designs, especially when to include non-randomized studies (NRS);
3) population(s) and the corresponding perspectives/outcomes;
4) choice of outcomes, including non-standard outcomes;
5) use and interpretation of surrogate outcomes;
6) analysis issues, such as synthesis without meta-analysis;
7) interpretation of findings, such as effect sizes and applying GRADE in public health reviews.
There is often an underlying tension between taking a broader and more inclusive perspective in reviews, versus pragmatic considerations of the timely delivery of those reviews within the resources available. This often contributes to decisions which could eventually affect the public health relevance of reviews produced.

Conclusions: some of the issues faced in these reviews could be at least partially anticipated through careful consideration of the review’s context, perspective and the potential application of evidence when questions are framed to ensure that inclusion criteria, outcomes, analysis methods, and interpretation are closely aligned. Other issues are more 'methodological' in nature and would benefit from evidence-based guidance. Authors of public health and health systems reviews need specifically tailored guidance, as the issues faced and methods needed are different from clinical treatment reviews.

Patient or healthcare consumer involvement: none