Victorian researchers have developed an artificial intelligence (AI) algorithm that is as accurate in diagnosis of skin cancers as a dermatologist.
If validated in further trials the software could become a useful tool to support GPs and skin cancer clinics in dermatology consultations where there is lack of access to dermatologists, according to Associate Professor Victoria Mar, Director of the Victorian Melanoma Service at The Alfred, Melbourne.
Associate Professor Mar led a pilot study, the Improving Skin Cancer Management With Artificial Intelligence (SMARTI) trial, which showed the system performed above 90% at accurately identifying a melanoma when compared to a dermatologist’s judgement.
“There’s potential for the tool to not only be used to support a doctor’s diagnosis, but as a triage tool, which is significant for regional Australians where we know there is a chronic shortage of dermatologists and doctors,” said Associate Professor Mar.
“With only 550 practicing dermatologists in Australia, mostly in metropolitan settings, we urgently need to ensure all Australians can access reliable diagnostic advice, regardless of where they live,” she added.
The ‘SMARTI’ trial was conducted across two Melbourne dermatology clinics, the Victorian Melanoma Service at The Alfred and the Skin Health Institute. It used an convolutional neural network algorithm developed by MoleMap and the Monash University eResearch team to examine images of 743 skin lesions from 214 adults who had one or more skin lesion of concern.
For comparison, the lesions were also reviewed by a dermatologist, and later a teledermatologist, who recorded their diagnosis and management plan for each lesion without knowing the AI algorithm results.
For the primary endpoint, the AI tool had a similar degree of accuracy for diagnosis of melanoma as a dermatologist: the area under the receiver operating characteristic curve (AUC) for the tool was 0·837 compared to 0·807 for teledermatologists review (p=0·050).
The false negative rate for the AI tool was 3·7%, which included two one metastatic deposits of melanoma (MDM), one melanoma in situ (MIS) and three actinic keratoses (AK) compared to 9.3% for teledermatologists (2 MDM, 6 MIS, 1 BCC, 2 intraepidermal carcinomas, and 4 AKs) and 0.6% for treating dermatologists (1 MIS).
The study found that overall, the AI tool would have a positive impact on lesion management decisions, shifting them to be more in line with those of the treating dermatologist (AUC 0·847 improved to 0·879 with CNN assistance, p=0·009).
While the proportion of biopsied lesions confirmed as benign was higher with the AI tool (31% vs 21% with dermatologist input only), the difference was not statistically significant (p=0.12).
The study investigators acknowledged that the AI tool would lead to unnecessary biopsies in some patients, with seven benign lesions biopsied which the treating dermatologist had recommended leaving or monitoring but the AI tool had classified the lesion as uncertain or malignant, and the patient consequently preferred biopsy.
The study investigators said the AI tool was worth evaluating further as an adjunct to clinician examination rather than as a replacement.
“The first port-of-call for anyone concerned about change to your skin is your GP,” A/Prof Mar said.
“The SMARTI trial’s results are very promising and give us confidence to further examine this AI tool in a larger cohort across more Australian sites and in the general practice setting,” she suggested.
“Testing this technology in regional Australia would also provide more equitable access to cutting-edge early interventions and help obtain the large-scale data we need to be confident that the algorithm will have a positive impact on improving patient outcomes,” she added.
Since the trial, the algorithm continues to be developed under the new technology group known as Kāhu.
The results are published in the Journal of the American Academy of Dermatology.