Challenges in the methodology and implementation of an exemplar Cochrane Review of prognostic factor studies

ID: 

OS44.5

Session: 

Oral session: Statistical Methods (1)

Date: 

Tuesday 22 October 2019 - 14:00 to 15:30

Location: 

All authors in correct order:

Aldin A1, Umlauff L1, Estcourt LJ2, Collins G3, Moons K4, Engert A5, Kobe C6, von Tresckow B5, Haque M1, Foroutan F7, Kreuzberger N1, Trivella M8, Skoetz N9
1 University of Cologne, Faculty of Medicine and University Hospital Cologne, Department I of Internal Medicine, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Cochrane Haematological Malignancies, Cologne, Germany
2 Haematology/Transfusion Medicine, NHS Blood and Transplant, Oxford, UK
3 Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
4 Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
5 Department I of Internal Medicine, University of Cologne, Faculty of Medicine and University Hospital of Cologne, Cologne, Germany
6 Department for Nuclear Medicine, University of Cologne, Faculty of Medicine and University Hospital of Cologne, Cologne, Germany
7 Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada
8 Centre for Statistics in Medicine, University of Oxford, Oxford, UK
9 University of Cologne, Faculty of Medicine and University Hospital Cologne, Department I of Internal Medicine, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Cochrane Cancer, Cologne, Germany
Presenting author and contact person

Presenting author:

Angela Aldin

Contact person:

Abstract text
Background: prognosis is the prediction of a patient’s health outcome. Individual factors can be used to predict the probability of an outcome (e.g. progression, mortality). Despite growing popularity, prognosis research still shows serious shortcomings regarding methodology, statistical analyses and reporting. Our prognostic factor of interest was interim positron emission tomography (PET) in newly diagnosed patients with Hodgkin Lymphoma. Interim PET identifies the state of disease after a few cycles of chemotherapy and is a good predictor of outcome, distinguishing between those with a poor prognosis and those with a better prognosis.

Objectives: to conduct an exemplary Cochrane Review of prognostic factor studies by assessing their risk of bias, meta-analyzing individual study results and grading the evidence.

Methods: we conducted a sensitive search in MEDLINE and the Cochrane Central Register of Controlled Trials (CENTRAL), with indication of the factor and disease of interest, but without a study filter. We developed a data extraction form according to the design and methodology of the included studies. We assessed risk of bias using the Quality in Prognosis Studies (QUIPS) tool, and meta-analyzed results on the association between interim PET, and overall survival and progression-free survival. We assessed our certainty of the evidence according to the recommendations of the GRADE Prognosis Working Group. To prevent bias in this review, two teams of two reviewers independently performed all relevant steps.

Results: we screened 6273 search results in detail and included 22 studies. Data extraction and meta-analysis of study results required contact with numerous principal investigators for further information, and estimating effect measures due to insufficient reporting of outcomes and results in the studies. Meta-analysis of adjusted results was not feasible; we meta-analyzed 15 primary studies with unadjusted results. Reaching a consensus in the 'Risk of bias' assessment was challenged by a lack of guidance and standardized methods for studies of prognosis. Grading the evidence from retrospective observational studies required elaborate discussions between review authors.

Conclusion: a lack of study filters, insufficient reporting of results, and inconsistencies within and between primary studies complicated the summary of results and questions our certainty in the evidence. Conducting a review of prognosis studies, following Methodological Expectations for Cochrane Intervention Reviews (MECIR) guidelines for traditional reviews and using a flexible Cochrane Review template, requires extensive discussion with experts in the field of prognosis in order to tackle difficulties in the methodology and implementation of reviews of prognosis studies.

Patient or healthcare consumer involvement: summarizing the evidence from prognosis studies can enable accurate predictions and aid healthcare users and providers to make an informed decision on treatment approaches in order to achieve the best possible health outcome.