Title:
|
Outcome reporting bias in meta-analysis
|
Introduction: The increased use of meta-analysis in systematic reviews of healthcare
interventions has highlighted several types of bias that can arise during the completion
of a randomised controlled trial (RCT). Publication bias has been recognised as a
potential threat to the validity of meta-analysis and can make the readily available
evidence unreliable for decision making. Until recently, outcome reporting bias
(ORB) has received less attention. ORB occurs when outcomes are selectively
reported for publication based on their results. This can impact upon the results of a
meta-analysis, biasing the pooled treatment effect.
Methods: Empirical evidence from a series of cohort studies that have assessed study
publication bias and ORB in RCTs is reviewed and summarised. Methods for
identifying ORB in a review and trial reports are discussed using several motivating
examples. Guidelines are obtained and assessed from worldwide organisations and
UK based charities that fund RCTs; with regards to trial registration, protocol
adherence and trial publication. Statistical, educational and policy solutions are
reviewed. A novel weighted linear regression is used to predict missing outcome data
when bias is suspected in a review containing structurally related outcomes. A
maximum bias bound previously proposed is further assessed using a simulation study
to test the sensitivity of the method when heterogeneity is present. The outcome
reporting bias in trials (ORBln study is introduced and the impact ORB has on this
cohort of reviews from the Cochrane Library is assessed using the bound for maximum
bias.
Results: Four empirical studies that examined the association between outcome
reporting bias and statistical significance found that statistically significant outcomes
were more likely to be completely reported than non-significant outcomes (range of
odds ratios: 2.2 to 4.7). Guidelines for 73 organisations and charities were reviewed;
only eleven of these organisations or charities mentioned the publication of negative as
well as positive outcomes. The simulation study indicates that the bound for
maximum bias is a useful approach for reviewers to apply to assess the robustness of
the conclusions of a review to ORB and can be applied in the presence of
heterogeneity. Assessing the impact of ORB in the cohort of reviews in the ORBIT
study indicates that ORB had an impact in 15% of the reviews considered.
Conclusions: This work summarises the direct empirical evidence for the existence of
study publication bias and ORB. Researchers need to be aware of the problems of
both types of bias and efforts should be concentrated on improving the reporting of
trials. There is a need to provide more detailed guidance for those conducting and
reporting RCTs to help prevent the selective reporting of results. There are several
complementary initiatives to either prevent ORB or to reduce the occurrence of ORB.
Therefore, with the implementation and development of these solutions it is hoped that
in the future ORB will become less of a problem. However, methods that detect ORB
and adjust results for ORB can be useful while the initiatives are given time to make an
impact.
|