Assessing risk of bias in non-randomized studies of interventions (1): introduction to the ROBINS-I tool

Core Methods Workshop

Workshop category: 

  • Investigating bias
Date and Location


Wednesday 23 October 2019 - 11:00 to 12:30


Contact persons and facilitators

Contact person:


Sterne J1, Higgins J1, Savovic J1
1 University of Bristol, UK


Target audience

Target audience: 

Systematic reviewers and people from other disciplines with an interest in how bias can arise in non-randomized studies of interventions

Level of difficulty: 

Type of workshop

Type of workshop : 



Non-randomized studies of interventions (NRSI) can provide information about effects of interventions that is not available from randomized trials, but their results may be affected by confounding, selection and information bias. Assessing the risk of bias in NRSIs included in systematic reviews is essential to ensure the robustness of review findings.

Objectives: 1) to describe ROBINS-I (Risk Of Bias In Non-randomized Studies – of Interventions), a tool to assess risk of bias in NRSI; 2) to guide participants through completion of an example ROBINS-I assessment; and 3) to describe recent updates and improvements to ROBINS-I.

Description: this workshop will describe key features of the ROBINS-I tool.
- Specification of a hypothetical pragmatic randomized trial that is free of bias, as a basis for risk of bias assessments.
- Preliminary considerations at review protocol stage.
- Specification of the effect of interest (either the effect of assignment to intervention, or the effect of adhering to intervention).
- Assessments of risk of bias within seven domains (confounding; selection of participants into the study; classification of interventions; deviations from intended intervention; missing data; measurement of outcomes; selection of the reported result).
- Signaling questions to inform judgements on risk of bias.
- Overall risk of bias in the result.

The session will also introduce recent improvements.
- Introduction of signalling questions that address biases such as immortal time bias that were not well-covered by the original tool.
- Development of algorithms to map answers to the signalling questions to suggested risk of bias judgements.

The workshop will include a guided practical session. Working in small groups, participants will complete parts of a 'Risk of bias' assessment for a selected non-randomized study. There will be opportunities for discussion of the tool and its role in systematic reviews. We are also offering a workshop describing an online implementation of the ROBINS-I tool.

Participants are encouraged to bring their own portable computing device to access detailed guidance from the internet ( The facilitators are unable to provide printed copies of the guidance document.