Pilot prospective assessment of the concordance of Melanoscope AI decisions with expert dermoscopic evaluation

Introduction. Early diagnosis of malignant skin neoplasms (MSN) is critical for prognosis, but a shortage of oncodermatologists in Russian regions limits screening coverage. Clinical decision support systems (CDSS) based on mobile dermoscopy represent a promising tool; proof of diagnostic effectiveness and equipping of primary care remain primary preconditions, while at the stage of clinical integration the interpretability of system output and standardization of patient routing become critical.

Aim: To assess the concordance between the conclusions of the Melanoscope AI clinical decision support system (hereafter, the system) and independent expert dermoscopic evaluation, and the change in the conclusions of a general practitioner after exposure to the system output in a screening setting.

Material and Methods. A single-center prospective pilot study was performed across four screening sessions (June 2025–April 2026). The material comprised dermoscopic images – one per examinee, of the lesion judged most suspicious on visual examination – from 176 participants. Concordance was assessed between the system's conclusions (malignant/benign) and the expert conclusion formed before exposure to the system's output. Lesions assessed by the expert as malignant were confirmed histologically; benign cases were not verified morphologically (partial differential verification). The change in concordance between a general practitioner's (GP) conclusions and the expert conclusion on the same cases, before and after exposure to the system's output, was assessed using the exact McNemar test.

Results. Agreement between the binary classification of the system and the independent expert conclusion across 176 lesions was 88.6% (95% CI 83.0–92.9), specificity 88.3% (82.5–92.7), positive predictive value 20.0% (6.8–40.7). All five histologically confirmed malignant neoplasms (3 melanomas and 2 basal cell carcinomas) were assigned by the system to the red routing zone; false-negative decisions could not be identified under the reference standard applied, since benign cases were not verified histologically. Six patients with dysplastic nevi were assigned to dynamic follow-up. Concordance between the GP's conclusions and the expert conclusion increased from 71.0% (63.9–77.2) to 82.4% (76.1–87.3); the gain was 11.4 percentage points (6.7–16.3), exact McNemar test p = 1.9·10^–⁶.

Conclusion. All histologically confirmed malignant neoplasms were assigned by the system to the red routing zone; however, sensitivity and the safety of ruling out malignancy are not established by this pilot study, owing to the small number of events and the absence of morphological verification of negative cases. Concordance between the GP's conclusions and the expert conclusion increased after exposure to the system's output. Multicenter verification of the system's decision concordance is needed on a larger cohort of examinees under population screening conditions.

1. Sung H., Ferlay J., Siegel R.L., et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: Cancer J. Clin. 2021;71(3):209–249. DOI: 10.3322/caac.21660

2. Kaprin A.D., Starinskiy V.V., Shakhzadova A.O. Malignant neoplasms in Russia in 2023 (morbidity and mortality). Moscow: P.A. Herzen MORI – branch of the National Medical Research Radiological Centre, 2024:276. (In Russ.) ISBN 978-5-85502-298-8.

3. Shakhzadova A.O., Starinskiy V.V., Lisichnikova I.V. State of cancer care for the population of Russia in 2022. Siberian Journal of Oncology. 2023;22(5):5–13. (In Russ.) DOI: 10.21294/1814-4861-2023-22-5-5-13 EDN: PESHHL

4. Chen J.Y., Fernandez K., Fadadu R.P., et al. Skin cancer diagnosis by lesion, physician, and examination type: a systematic review and meta-analysis. JAMA Dermatology. 2025;161(2):135–146. DOI: 10.1001/jamadermatol.2024.4382

5. Dinnes J., Deeks J.J., Chuchu N., et al. Dermoscopy, with and without visual inspection, for diagnosing melanoma in adults. Cochrane Database of Systematic Reviews. 2018;12(12):CD011902. DOI: 10.1002/14651858.CD011902.pub2

6. Esteva A., Kuprel B., Novoa R.A., et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115–118. DOI: 10.1038/nature21056

7. Brinker T.J., Hekler A., Enk A.H., et al. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur. J. Cancer. 2019;113:47–54. DOI: 10.1016/j.ejca.2019.04.001

8. Tschandl P., Rinner C., Apalla Z., et al. Human-computer collaboration for skin cancer recognition. Nature Medicine. 2020;26(8):1229–1234. DOI: 10.1038/s41591-020-0942-0

9. Udrea A., Mitra G.D., Costea D., et al. Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms. J. Eur. Acad. Dermatol. Venereol. 2020;34(3):648–655. DOI: 10.1111/jdv.15935

10. Liu Y., Jain A., Eng C., et al. A deep learning system for differential diagnosis of skin diseases. Nature Medicine. 2020;26(6):900–908. DOI: 10.1038/s41591-020-0842-7

11. Wysocki O., Davies J.K., Vigo M., et al. Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making. Artificial Intelligence. 2023;319:103839. DOI: 10.1016/j.artint.2022.103839

12. Reyes M., Meier R., Pereira S., et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiology: Artificial Intelligence. 2020;2(3):e190043. DOI: 10.1148/ryai.2020190043

13. Korabelnikov D.I., Lamotkin A.I. Analysis of the causes of misclassification of non-melanocytic skin tumor images by artificial-intelligence-based software. Siberian Journal of Clinical and Experimental Medicine. 2026;41(1):221–231. (In Russ.) DOI: 10.29001/2073-8552-2026-2881 EDN: DLMLOC

14. Combalia M., Codella N., Rotemberg V., et al. Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration grand challenge. The Lancet Digital Health. 2022;4(5):e330–e339. DOI: 10.1016/S2589-7500(22)00021-8

15. Kozachok E.S. Methodology for Creating a Clinically Verified Dermoscopic Image Dataset [Preprint]. 2026. arXiv:2605.25168 [cs.CV]. DOI: 10.48550/arXiv.2605.25168

16. Kozachok E.S., Seregin S.S., Kozachok A.V., et al. An intelligent clinical decision support system for diagnosis of skin lesions based on dermoscopic image analysis. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Seriya: Upravlenie, vychislitel’naya tekhnika, informatika. Meditsinskoe priborostroenie. 2025;15(3):50–65. (In Russ.) DOI: 10.21869/2223-1536-2025-15-3-50-65

17. Sounderajah V., Guni A., Liu X., et al.; STARD-AI Steering Committee. The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence. Nature Medicine. 2025;31(10):3283–3289. DOI: 10.1038/s41591-025-03953-8

18. Argenziano G., Soyer H.P., Chimenti S., et al. Dermoscopy of pigmented skin lesions: results of a consensus net meeting via the Internet. J. Am. Acad. Dermatol. 2003;48(5):679–693. DOI: 10.1067/mjd.2003.281

19. Kozachok E.S., Seregin S.S., Kozachok A.V., et al. Cascade classification of dermoscopic images of skin neoplasms with controllable sensitivity and external clinical validation [Preprint]. 2026. arXiv:2606.13135 [cs. CV]. (In Russ.) DOI: 10.48550/arXiv.2606.13135

20. Tun H.M., Abdul Rahman H., Naing L., Malik O.A. Trust in artificial intelligence-based clinical decision support systems among health care workers: systematic review. J. Med. Internet Res. 2025;27:e69678. DOI: 10.2196/69678