Generalizability of findings from systematic reviews and meta-analyses in the leading general medical journals

Presentation video:




Oral session: Investigating bias (5)


Thursday 24 October 2019 - 14:00 to 15:30


All authors in correct order:

Malmivaara A1
1 National Institute for Health and Welfare, Finland
Presenting author and contact person

Presenting author:

Antti Malmivaara

Contact person:

Abstract text
Background: the PRISMA statement for reporting of systematic reviews and meta-analyses, endorsed by the leading general medical journals, necessitates adequate reporting of the essential elements of randomized controlled trials (RCTs).

Objectives: to document reporting of characteristics of RCTs in systematic reviews and meta-analyses in the leading general medical journals; and to examine whether reporting allows interpretations on clinical homogeneity, justification for meta-analysis or generalizability of the overall findings.

Methods: we searched all systematic reviews and meta-analyses in the Annals of Internal Medicine, BMJ, JAMA and Lancet from 1 January 2016 to 28 February 2019 using PubMed, and we extracted data of important RCT characteristics using the Benchmarking method.

Results: none of the 115 systematic reviews reported participants’ path prior to randomization, reasons for exclusion, or numbers declining participation in the RCTs. Disorder-specific clinical features were reported in 35%, and co-morbid conditions in 25% of the systematic reviews. Participants’ functioning, behavioural factors, environmental factors and inequity-related factors in the RCTs were reported in 3%, 21%, 0% and 9%, respectively. Adherence to interventions was reported in 7%, cross-over to other treatment arms in 0%, and co-interventions in 2% of the systematic reviews. Follow-up percentages and adequacy of the statistical analyses in the RCTs were reported in 8% and 3% of the systematic reviews.

Conclusions: reporting of participant selection, baseline characteristics, and adherence to interventions is lacking or insufficient in all the systematic reviews; and consequently, we can make no assessment of clinical homogeneity of RCTs, justification for meta-analysis, or generalizability of findings from these systematic reviews. All the shortcomings can easily be avoided by using the Benchmarking method.

Patient or healthcare consumer involvement: comprehensive description of participant-relevant characteristics of RCTs