Do Cochrane and non-Cochrane systematic reviews report results integrating certainty of evidence and effect size?




Poster session 3


Thursday 24 October 2019 - 12:30 to 14:00

All authors in correct order:

Ciapponi A1, Glujovsky D1, Comandé D1, Calderón M1, Bardach A1
1 Instituto de Efectividad Clínica y Sanitaria (IECS-CONICET), Argentina
Presenting author and contact person

Presenting author:

Agustín Ciapponi

Contact person:

Abstract text
Background: the certainty (quality) of the evidence should be presented together with effect estimates for each outcome using consistent words and expressions in the results section of a systematic review (SR). Some Cochrane Review Groups recommend certain wording to improve reporting consistency (See Table 1) and the Cochrane Handbook for Systematic Reviews of Interventions, version 6, reinforces this approach.

Objectives: to analyze if Cochrane and non-Cochrane SRs’ abstracts use a wording expressing the certainty of evidence and the effect size, and to describe alternative reporting approaches.

Methods: we analyzed all Cochrane SRs of interventions published in November to December 2018. We excluded updates. We also analyzed the top five Impact Factors Journal List (1 New England Journal of Medicine, 2 Lancet, 3 BMJ, 4 JAMA, 5 Annals of Internal Medicine), based on 2017 Thomson Reuters Journal Citation in medicine general and internal category, where Cochrane ranked 13th. The PubMed search strategy for non-Cochrane systematic reviews is shown in Table 2.

Results: we have analyzed eight Cochrane and nine non-Cochrane SRs so far and we will present the analysis of all the reviews at the Colloquium. All Cochrane and 56% of non-Cochrane SRs reported the level of certainty of evidence for primary outcomes.

Half of Cochrane and 33% of non-Cochrane SRs followed the suggested wording in Table 1 for certainty of evidence. Regarding the effect magnitude, 63% of Cochrane and 56% of non-Cochrane SRs followed the suggested wording. Examples of alternative wordings are described in Table 2.

Conclusions: a high proportion of recent Cochrane and non-Cochrane SRs used inconsistent wording when reporting the results and did not follow a wording scheme that integrates certainty of evidence with effect sizes. Even among the prestigious journals analyzed there is a big opportunity of reporting improvement. A larger sample size will allow us to compare the performance of Cochrane Reviews with those of top-five journals’ reviews.

Patient or healthcare consumer involvement: none, since it was a methodological study about published systematic reviews.