Can a smartwatch coupled with a deep learning AI algorithm beat a cardiologist with an ECG when it comes to detecting atrial fibrillation?
That’s the question being contested on social media this week following the publication of trial results in JAMA Cardiology of an Apple Watch for passive detection of atrial fibrillation.
In a validation study involving 9750 participants including 347 participants with AF, researchers at the University of California San Francisco used smartwatch data to develop and validate a deep neural network to detect AF.
Based on more than 139 million heart rate measurements from the Cardiogram app, the deep neural network exhibited a C statistic of 0.97 to detect AF against the reference standard 12-lead ECG–diagnosed AF in an external validation cohort of 51 patients undergoing cardioversion. The sensitivity was 98.0% and specificity was 90.2%. I
In a further exploratory analysis relying on self-report of persistent AF in ambulatory participants, the C statistic was 0.72, sensitivity was 67.7% and specificity was 67.6%.
The researchers said their findings showed that smartwatches combined with AI algorithms can passively detect AF in sedentary individuals “with excellent performance characteristics”.
The public health potential for AF screening was substantial, they suggested.
“Our deep neural network substantially outperforms standard techniques to detect self-reported persistent AF from ambulatory data, albeit with modest accuracy in these free-living natural environments,” they wrote.
“Given the broad and growing use of smartwatches and ready accessibility of downloadable mobile applications, this approach may ultimately be applied to perform AF detection at large scale, ultimately leveraging common wearable devices to guide AF management and rhythm assessment.”
However the findings provoked scepticism from clinicians via social media, with Dr Eric Topol, cardiologist at the Scripps Research Institute, California saying the results were exciting but flaws in the study design meant it was too early to be used as the basis for recommending screening
Failure of a deep learning #AI algorithm to detect atrial fibrillation via smartwatch prospectively @JAMACardio @UCSF @GeoffTison https://t.co/vvU23bStKZ pic.twitter.com/lw0RV6AkGi
— Eric Topol (@EricTopol) March 21, 2018
“[AI] really just doesn’t perform,” he told the Washington Post.
“This doesn’t pass muster for use in detection of atrial fibrillation.”
Dr Benjamin Mazer of Yale Pathology said studies of AI applications in medicine were plagued by publication bias.
“When was the last time you saw a machine learning paper that showed a computer COULDN’T beat or match a doctor’s skill?” he wrote.
“AI is almost certainly going to have an important place in medicine, especially pathology. I’m not dismissing the technology. But I would love to award internet points to the first pathology colleague who dares publish a negative study.
Others said the smartwatch screening could lead to high rates of false positives, overinvestigation and overtreatment.
Drs Mazer and Topol are spot on. AI is not close in cardiology. Some day (not my lifetime), AF detection may lead to positive outcomes.
But in near-term, since (too many) of these pts will get doused w fear, radiation, contrast, pills, etc, it will be am overRx epidemic. https://t.co/9tFWpqcSDA— John Mandrola, MD (@drjohnm) March 22, 2018