Leading medical journal The Lancet has launched an attack on what it calls the systematic review ‘gravy train’ saying it is a fundamentally flawed process due to the inclusion of small, biased trials.
In a scathing commentary article, the Lancet’s editor Dr Richard Horton takes aim at the Cochrane Library and funding bodies such as Australia’s NHMRC for supporting the growth of a mammoth industry of systematic reviews and meta-analyses that may be ‘poisoning’ medical practice rather than nurturing it.
The Lancet editorial says it is no longer possible to ignore the strong doubts about the scientific validity of systematic reviews raised by leaders of evidence-based medicine such as Ian Roberts, Professor of Epidemiology and Public Health and Co-Director of the Clinical Trials Unit at the London School of Hygiene & Tropical Medicine.
Professor Roberts and others have described the widespread belief and support for systematic reviews in healthcare as an “anachronistic religion” because reviews often include low quality trials with a strong risk of bias.
One of his strongest criticisms is that systematic reviewers ‘sweep up’ small and underpowered trials into their meta-analyses without applying sufficiently rigorous inclusion criteria.
Systematic reviews may also be giving the wrong answer due to selection bias and because their findings are strongly influenced on the inclusion of one or two large but false ‘randomised trials’ , it adds.
This highlights the other fatal flaw of systematic reviews – that they are based on trust in the integrity of trial design and results.