Systematic reviews of prognosis studies 4: the use of GRADE for assessing the certainty of evidence for questions of overall prognosis and prognostic factors

Workshop category: 

  • Other review formats (e.g. overviews of reviews, network meta-analysis)
Date and Location


Thursday 24 October 2019 - 16:00 to 17:30


Contact persons and facilitators

Contact person:


Foroutan F1, Guyatt G1, Alba AC2, Trivella M3, Aldin A4, Kreuzberger N5, Skoetz N6, Iorio A2
1 , Canada
2 McMaster University, Canada
3 University of Oxford, UK
4 Cochrane Haematological Malignancies, Germany
5 University of Cologne, Faculty of Medicine and University Hospital Cologne, Department I of Internal Medicine, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Cochrane Haematological Malignancies, Cologne, Germany
6 University of Cologne, Faculty of Medicine and University Hospital Cologne, Department I of Internal Medicine, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Cochrane Cancer, Cologne, Germany
Target audience

Target audience: 

Review authors with an interest in systematic reviews of prognosis studies

Level of difficulty: 

Type of workshop

Type of workshop : 



Background: the GRADE approach to rating certainty in the results considers study design, risk of bias, inconsistency, imprecision, indirectness, publication bias, magnitude of effect, and dose-response.
We developed the guidance for applying the GRADE approach to determine certainty in estimates of future events in broad categories of patients (overall prognosis), and in estimates of the association between patient characteristics and undesirable or desirable outcomes (prognostic factors).

Objective: in this workshop, we will provide an interactive session designed to share and engage participants with our guidance for application of GRADE to evidence on overall prognosis and prognostic factors.

Description: during our workshop, we will engage participants on 10 topics related to the application of the GRADE criteria for determining the certainty of evidence on questions of prognosis. For most topics, we will provide participants with examples from real systematic reviews and meta-analyses. The participants will review the pertinent segments of the reviews, and apply our GRADE guidance to determine the certainty of the evidence.

- We will engage the participants in discussions over the ideal study design (randomized controlled trials vs observational studies) when assessing evidence related to prognosis.
- Risk of bias: we will introduce the first GRADE domain (high vs low risk of bias studies) with focus on considerations necessary for evidence related to prognostic factors (adjusted vs unadjusted associations).
- Inconsistency: in this segment we review the concept of inconsistency (and its relation to statistical heterogeneity). Through reviewing systematic reviews, we will assess the impact of aberrant studies and their impact on the overall conclusion of the meta-analysis.
- Imprecision: the participants will review the different frameworks for assessment of imprecision (non, partially, and fully contextualized), along with the difference between relative compared to absolute effect estimates. The workshop will provide an online calculator for determining absolute risks, necessary for the assessment of imprecision under the partially contextualized framework (most ideal for systematic reviews).
- Indirectness: in this segment, the attendees will compare the research question of a systematic review to the characteristics of studies addressing the research question. We will provide guidance for circumstances when it is necessary to rate down our certainty for indirectness.
- Publication bias: we will provide an example of a systematic review in which the assessment of publication bias based on visual inspection (funnel plot) is different compared to the statistical test.