Building more robust search strategies in Cochrane systematic reviews: towards improving inter-information specialists’ agreement


Oral session: Searching and information retrieval (1)


Tuesday 22 October 2019 - 11:00 to 12:30


All authors in correct order:

1 Kleijnen Systematic Reviews Ltd, .
Presenting author and contact person

Presenting author:

Pawel Posadzki

Contact person:

Abstract text
Background: in Cochrane systematic reviews (CSRs), two review authors carry out screening and data extraction to improve accuracy and minimize any potential biases. However, typically only one librarian/information specialist is involved in building a search strategy. This might lead to bias, especially in fields where there is a lack of clear categorization of interventions, constantly evolving concepts, pedagogies, or theories as well as a multitude of methods, features, technologies, or delivery settings.

Objectives: the objective of this research was to compare search strategy-building techniques/methods between two experienced librarians; and calculate the amount of overlap between them.

Methods: comparative study. Two experienced librarians/information specialists (one at Imperial College London (ICL) and the other at Karolinska Institutet (KI), Stockholm) independently of each other worked on developing a search strategy for the following PICO: what is the effectiveness of e-learning (digital learning) for education in the health professions. These strategies were then run on MEDLINE (via Ovid).

Results: the ICL librarian included 100 search terms (718 words) whereas the KI librarian included 160 search terms (621 words). There was a 62.5% overlap between the librarians. The KI search strategy yielded 659 hits, whereas the ICL one retrieved 1399 results. The most frequent disagreements (n = 10) pertained to ‘exploding’ of the following terms: education, evidence-based practice, healthcare personnel, medical personnel, student, software, telecommunication, computer, audiovisual equipment, and simulation. Other disagreements (n = 7) pertained to phrasing (used by Ovid) and inclusion of unnecessary search terms.

Conclusions: results presented herein are preliminary and require independent replication. However, I argue that building a search strategy for CSRs should be done by two review authors to minimize any potential biases; and to increase inter-librarian agreement. This can have an impact on the number of studies retrieved/included in the review and subsequent conclusions drawn.

Patient or healthcare consumer involvement: none