Background: research funders increasingly request that new research is underpinned by high-quality systematic reviews. But how do we measure high quality and does ‘quality’ in itself influence uptake or impact, or both? Are systematic reviews that are methodologically robust more likely to be cited and potentially therefore more likely to influence policy and/or practice?
Objectives: to use the Rx for Change database to explore the relationship between the methodological quality of systematic reviews of the effectiveness of strategies and programmes designed to alter behaviours in healthcare and their subsequent citation rates.
Methods: the data set was from Rx for Change, a publicly accessible database for healthcare professionals and policy makers, offering quick access to quality-rated evidence about the effectiveness of strategies and programmes designed to alter behaviours in healthcare. The database was set up in 2007 and regularly updated/maintained until 2013.
For all reviews with at least 10 years of citations data available, we fitted a Negative Binomial regression model to the total number of citations within the first 10 years of publication; all reviews included in the analysis contributed 10 years of citations data. We included AMSTAR score, grouped into low (0 to 3), moderate (4 to 7) or high (8 to 11) quality, as a predictor and we adjusted the model for year of publication.
Results: we included 413 reviews, published between 1977 and 2009 inclusive, in the analysis. Descriptive statistics by publication decade for the included reviews are shown in Table 1. There was evidence of an effect of AMSTAR score on citation rate in the first 10 years of publication. Relative to low scoring reviews, the citation rate for moderate- and high-scoring reviews was 74% and 60% higher, respectively (Table 2).
Conclusions: further exploration of the data is ongoing but this preliminary analysis provides some evidence that reviews reporting more items on the AMSTAR checklist are more likely to be cited by other authors and lead to better future impact. The research community, including journal editors should be encouraged to conduct, report and publish reviews to the highest standards, enabling healthcare decisions to be informed by the best evidence.
Patient or healthcare consumer involvement: healthcare consumers were not involved in the conduct of this analysis or the preparation of the abstract.