Assessing risk of bias in non-randomized studies of interventions 2: an online tool to facilitate risk of bias assessments using the ROBINS-I tool

Core Methods Workshop

Workshop category: 

  • Investigating bias
Date and Location


Thursday 24 October 2019 - 11:00 to 12:30


Contact persons and facilitators

Contact person:


Higgins J1, Savovic J1, Sterne J1
1 University of Bristol, UK


Hutton D1
1 University of Bristol, UK
Target audience

Target audience: 

Systematic review authors and people from other disciplines with an interest in how bias can arise in non-randomized studies of interventions

Level of difficulty: 

Type of workshop

Type of workshop : 



Background: non-randomized studies of interventions (NRSI) can provide useful data on the effects of healthcare interventions but are likely to be subject to confounding and other biases. We present a new online implementation of the ROBINS-I (Risk of Bias In Non-randomized Studies - of Interventions) tool that allows for a comprehensive assessment of risk of bias in NRSI through a user-friendly interface.

- describe the features of an online implementation of the ROBINS-I tool;
- give participants the opportunity to try out the online tool;
- obtain feedback from participants on the usability of the new online platform.

Description: this workshop will describe the key functionality of the ROBINS-I online implementation:
• user-friendly layout of the tool’s elements, including signalling questions, study documents and information on confounding domains/co-interventions;
• advanced review management, facilitating collaboration;
• conditional signalling questions (asked only when relevant based on answers to previous questions);
• on-screen guidance, placing relevant information at users’ fingertips;
• comparison of independent assessments to resolve disagreements;
• export of assessment results.

We will demonstrate these features through a facilitated practical session during which participants will perform parts of a ROBINS-I assessment using the online tool. We will provide a completed ROBINS-I assessment for participants to work with, but participants are welcome to use their own completed assessment (e.g. from an ongoing or completed review).

The workshop will end with a discussion session, eliciting participants’ suggestions for improvements to the online tool.

Workshop participants will need their own laptop. Participants must be familiar with ROBINS-I. We are also offering a workshop providing an introduction to the ROBINS-I tool: participants new to the tool must attend that workshop first.