Diagnostic efficiency of individual systems for automatic analysis of computed tomography images in the detection of ischemic stroke in the basin of the middle cerebral artery

Background. The diagnosis of ischemic stroke is of high importance in modern medical practice. One of the most promising methods for solving this problem is the introduction of machine learning algorithms into physicians’ work as an auxiliary tool for the interpretation of beam images.

Aim: To compare automated computed tomography (CT) image analysis systems in detecting middle cerebral artery stroke.

Material and Methods. The study included three anonymized (A, B, C) machine learning algorithms. Analytical validation was carried out on a database of one hundred patients admitted in St. Petersburg vascular center with suspected middle cerebral artery stroke, who underwent noncontrast head CTs. Ischemic stroke in half of the patients was confirmed on the basis of clinical examination findings and CT-angiography and CT-perfusion. The study evaluated the performance indicators (sensitivity, specificity, positive predictive value, negative predictive value, accuracy) for detecting a set of signs of early ischemic changes (by automatic segmentation and predicting a score on the ASPECTS scale). The article also provides a graph that allows you to evaluate the quality of a binary classification – characteristic curves (ROC-curves).

Results. The meta-analyses showed all the considered automated algorithms did not reach the threshold values of accuracy (range from 0.67 to 0.75) required for programs according to clinical guidelines (0.80). The algorithms showed variability in sensitivity and specificity. One of the automatic analysis systems (A) had a high sensitivity (0.88), but at the same time a low specificity (0.46), which indicates its overtraining and a tendency to overdiagnoses. The remaining algorithms (B, C) showed low sensitivity (0.6; 0.55) and high specificity (0.9; 0.8).

1. Decree of the President of the Russian Federation dated October 10, 2019 No. 490 “On the development of artificial intelligence in the Russian Federation” (in conjunction with the National Artificial Intelligence Strategy until 2030). (In Russ.) URL: http://www.kremlin.ru/acts/bank/44731 (31.08.2023).

2. Andropova P.L., Gavrilov P.V., Kazantseva I.P., Domienko O.M., Narkevich A.N., Kolesnikova P.A. et al. Interexpert agreement between neuroradiologists in the diagnosis of middle cerebral artery stroke by computed tomography. Medical Visualization. 2023;27. (In Russ.) DOI: 10.24835/1607-0763-1315.

3. Morozov S.P., Vladzimirsky A.V., Klyashtornsky V.G., Andreichenko A.E., Kulberg N.S., Gombolevsky V.A. et al. (compilers). Clinical trials of intelligent software (radiation diagnostics). Series “The best practices of radiation and instrumental diagnostics”. Moscow; 2019:34. (In Russ.)

4. Vasiliev A.Yu., Maly A.Yu., Serova N.S. Analysis of these radiation methods of research on the basis of the principles of evidence-based medicine: a training manual. Moscow: GEOTAR-Media; 2008:32. (In Russ.)

5. Gavrilov P.V., Ushkov A.D., Smolnikova U.A. Detection of lumps in the lungs with digital X-ray: the role of the work experience of the radiologist. Medical Alliance. 2019;(2):51–56. (In Russ.) URL: https://elibrary.ru/item.asp?id=38073049 (31.08.2023).

6. Meldo A.A. Development and implementation of an artificial intelligence system in the radiation diagnostics of lung focal formations: dis. ... doctor of science; 3.1.25. St. Petersburg; 2022:235. (In Russ.) URL: http://www.almazovcentre.ru/wp-content/uploads/%D0%94%D0%B8%D1%81%D1%81%D0%B5%D1%80%D1%82%D0%B0%D1%86%D0%B8%D1%8F-%D0%9C%D0%B5%D0%BB%D0%B4%D0%BE-%D0%90%D0%90.pdf (31.08.2023).