A meta-analysis published in Lancet Respiratory Medicine that looks at whether e-cigarettes help smokers quit has caused quite a stir. The paper, which reviewed 38 studies suggests those who use e-cigarettes are less likely to quit than smokers who don’t.

The review has been described by the experts as: ‘grossly misleading’, ‘a major failure of the peer review system’ and ‘not scientific’.

Here’s what they had to say:

Prof Robert West, Professor of Health Psychology, University College London, said:

“If use of e-cigarettes caused fewer smokers to quit, the quit rate in England would have decreased as use of e-cigarettes has increased. Data from The Smoking Toolkit Study – a large ongoing population survey of smoking in England – shows, if anything, the opposite (www.smokinginengland.info).

Publication of this study represents a major failure of the peer review system in this journal.”

Prof Linda Bauld, Professor of Health Policy, University of Stirling; Deputy Director, UK Centre for Tobacco and Alcohol Studies; Chair in Behavioural Research for Cancer Prevention, Cancer Research UK, said:

“E-cigarettes have been called a ‘distruptive technology’ not just because of the controversy they have created but also because they pose significant challenges for research. The current paper represents the latest attempt to bring together the existing literature on e-cigarettes for smoking cessation. While its breath is to be commended, its conclusions (that e-cigarettes don’t work for smoking cessation) are at best tentative and at worst incorrect. The main reason for this is that attempting to directly compare the results of a body of literature that uses such a wide range of study designs and includes such variable (and often poorly defined) populations and outcomes is difficult, if not impossible. Some of the observational studies included in the review, in particular, suffer from a range of limitations that don’t allow us to reliably assess whether e-cigarettes help smokers quit.

“For example, these studies: don’t properly assess whether participants have used e-cigarettes enough to make a difference for smoking cessation (such as including measures of ‘ever’ rather than ‘regular’ use);  may be biased in how participants in the studies were selected (i.e. not representative of e-cigarette users in the population): and, perhaps most importantly, have confounding factors including that smokers in the studies are these who have tried to quit many times in the past and may therefore be more likely to try the remaining new product  (e-cigarettes), or that they gave up using these devices early in the conduct of the study but were still included in the final results with the assumption that e-cigarettes didn’t ‘work’ for them whereas there could be multiple reasons why they stopped using the devices.

“Some of the more recent studies included in the review do point to the types of measures that should be used to assess e-cigarettes for smoking cessation. These categorise the type of device (as e-cigarettes are many products not one product), look carefully at when and for how long e-cigarettes were used, and ask whether participants were using them to stop smoking or for other reasons. These more carefully conducted studies shed light on how e-cigarettes could help smokers stop – for example if they contain sufficient nicotine, are used often and for long enough, and are more advanced (‘tank’) devices than earlier ‘ciga-like’ e-cigarettes.  However, the current review does not separate out these studies or draw these distinctions but treats the body of evidence as a consistent and comparable set of studies when in fact it is not.

“The review also omits an important part of the picture when assessing any benefits of e-cigarettes in helping smokers quit – and that is the sheer reach of these devices. They are now the most popular aid to stopping smoking in the UK, for example, used by over 2.6 million people. Even low quit rates (which, it has to be said, are also found for licensed smoking cessation aids like Nicotine Replacement Therapy) for groups in studies could translate to many smokers quitting when applied at the population level.

“Other systematic reviews of the literature on e-cigarettes for smoking cessation are underway as this is such a rapidly moving area of research. These should not repeat the mistake of this current review which is to not take account of the huge variability in the characteristics of e-cigarette users, the devices themselves, and patterns of use. Alternative, more carefully conducted, reviews are needed if  we are to provide health professionals and, most importantly, smokers with high quality information about the circumstances in which e-cigarettes may help people to move away from the more deadly alternative – combustible tobacco.”

Rosanna O’ Connor, Director of Tobacco, Alcohol & Drugs, Public Health England, said:

“There are over a million ex-smokers using an e-cigarette in Britain and we need to provide those who continue to smoke with accurate, balanced information on different quitting methods.

“Evidence from practice in England shows that two out of three smokers who combined e-cigarettes with additional expert support from a local stop smoking service quit successfully and while dual use is a complex issue, many vapers report using an e-cigarette to cut down and ultimately quit. Smokers who have struggled to quit in the past could try vaping, and we encourage vapers to take that next step and stop smoking completely.”

Prof Peter Hajek, Director of the Tobacco Dependence Research Unit, Queen Mary University of London (QMUL), said:

“This review is grossly misleading in my opinion. There are several problems with the way studies were selected and used, but the main flaw is simple, though not easy to spot. The studies that are presented as showing that vaping does not help people quit only recruited people who were currently smoking and asked them if they used e-cigarettes in the past.  This means that people who used e-cigarettes and stopped smoking were excluded.  The same approach would show that proven stop-smoking medications do not help or even undermine quitting.

“Here is an analogy: Imagine you recruit people who absolutely cannot play piano. There will be some among them who had one piano lesson in the past. People who acquired any skills at all are not in the sample, only those that were hopeless at it are included. You compare musical ability in those who did and those who did not take a lesson, find a difference, and report that taking piano lessons harms your musical ability. The reason for your finding is that all those whose skills improved due to the lessons are not in the sample, but it would not necessarily be obvious to readers.

“E-cigarettes are a major development in public health. It is unfortunate that their potentially huge positive impact is being hindered by excessive regulations triggered by misleading suggestions.”

Prof Ann McNeill, Professor of Tobacco Addiction, National Addiction Centre, Institute of Psychiatry Psychology & Neuroscience, King’s College London (IoPPN), said:

“This review is not scientific. The information included about two studies that I co-authored is either inaccurate or misleading. In addition, the authors have not included all previous studies they could have done in their meta-analysis. I believe the findings should therefore be dismissed. I am concerned at the huge damage this publication may have – many more smokers may continue smoking and die if they take from this piece of work that all evidence suggests e-cigarettes do not help you quit smoking; that is not the case.

“The problems with the authors’ interpretation of the two papers mentioned above are as follows: The first study (Adkson et al) is not longitudinal as has been reported here – e-cigarette use was measured at follow up, the same time as quit status was ascertained. The second study (Hitchman et al) included smokers who were using e-cigarettes at baseline and therefore included smokers who may have tried to use e-cigarettes to quit and failed, and excluded smokers who successfully used e-cigarettes to quit. The authors of this meta-analysis had been previously informed by the authors of the Adkison paper that they were misreporting the findings.”

Quotes courtesy of The Australian Science Media Centre.