Attrition bias, the use of intention-to-treat and magnitude of treatment effects in randomized controlled trials: a meta-epidemiological study

Session: 

Oral session: Investigating bias (6)

Date: 

Thursday 24 October 2019 - 16:00 to 17:30

Location: 

All authors in correct order:

Armijo-Olivo S1, Da Costa B2, Arienti C3, Negrini S3
1 University of Alberta; Institute of Health Economics, Canada
2 University of Toronto, Canada
3 Cochrane Rehabilitation, Italy
Presenting author and contact person

Presenting author:

Susan Armijo-Olivo

Contact person:

Abstract text
Background: intention-to-treat (ITT) is a gold standard strategy used to analyze the results of randomized controlled trials (RCTs). ITT analysis has been considered a indicator of methodological quality, since has been used to determine trials' quality. The extent to which the use of ITT or not is related to treatment effects in RCTs has been poorly explored. In addition, the influence of biases related to attrition and missing data and their association with treatment effects in RCTs have not been extensively explored.

Objective: therefore, the main objective of this study was to determine the association between biases related to attrition and missing data and the use of the ITT principle, and changes in effect size estimates in RCTs.

Methods: this was a meta-epidemiological study. We identified a random sample of RCTs included in meta-analyses in the discipline of physical therapy by searching the Cochrane Database of Systematic Reviews. Independently, two reviewers conducted data extraction - including assessments of the use of the ITT principle, missing data and dropouts. To determine the association between biases related to attrition, missing data, and the use of ITT and effect sizes, we conducted a two-level analysis using a meta-meta-analytic approach.

Results: a total of 393 trials included in 43 meta-analyses, analyzing 44,622 patients contributed to this study. We considered 134 trials (34.1%) from the 393 trials included to have used ITT, while 218 (55.5%) did not use ITT. The remaining trials (n = 41; 10.4%) were unclear about whether they used ITT or not. We found 14 trials (10.4%) from the 134 trials evaluated as having ITT to be at high risk of bias. By contrast, 79 trials (38.7%) from 204 trials that we considered to be at low risk of bias, did not use the ITT principle. Trials that did not use the ITT principle (ES = -0.13; 95%CI -0.26 to 0.01), or which were assessed as having an inappropriate control of incomplete outcome data tended to underestimate the treatment effect when compared with trials with adequate use of ITT and control of incomplete outcome data (ES = -0.18; 95%CI -0.29 to -0.08).

Conclusion: our results suggest that when evaluating risk of bias of primary RCTs, systematic reviewers and clinicians implementing research into practice should pay attention to these biases since they could underestimate treatment effects. Systematic reviewers should perform sensitivity analysis including trials with low risk of bias in these domains, as primary analysis and/or in combination with less restrictive analyses. Authors and editors should make sure that ITT and missing data are properly reported in trial reports.