ABSTRACT
BACKGROUND AND PURPOSE: Intracranial steno-occlusive lesions are responsible for acute ischemic stroke. However, the clinical benefits of artificial intelligence-based methods for detecting pathologic lesions in intracranial arteries have not been evaluated. We aimed to validate the clinical utility of an artificial intelligence model for detecting steno-occlusive lesions in the intracranial arteries.
MATERIALS AND METHODS: Overall, 138 TOF-MRA images were collected from two institutions, which served as internal (n = 62) and external (n = 76) test sets, respectively. Each study was reviewed by five radiologists (two neuroradiologists and three radiology residents) to compare the usage and non-usage of our proposed artificial intelligence model for TOF-MRA interpretation. They identified the steno-occlusive lesions and recorded their reading time. Observer performance was assessed using the area under the Jackknife free-response receiver operating characteristic curve and reading time for comparison.
RESULTS: The average area under the Jackknife free-response receiver operating characteristic curve for the five radiologists demonstrated an improvement from 0.70 without artificial intelligence to 0.76 with artificial intelligence (P = .027). Notably, this improvement was most pronounced among the three radiology residents, whose performance metrics increased from 0.68 to 0.76 (P = .002). Despite an increased reading time upon using artificial intelligence, there was no significant change among the readings by radiology residents. Moreover, the use of artificial intelligence resulted in improved inter-observer agreement among the reviewers (the intraclass correlation coefficient increased from 0.734 to 0.752).
CONCLUSIONS: Our proposed artificial intelligence model offers a supportive tool for radiologists, potentially enhancing the accuracy of detecting intracranial steno-occlusion lesions on TOF-MRA. Less-experienced readers may benefit the most from this model.
ABBREVIATIONS: AI = Artificial intelligence; AUC = Area under the receiver operating characteristic curve; AUFROC = Area under the Jackknife free-response receiver operating characteristic curve; DL = Deep learning; ICC = Intraclass correlation coefficient; IRB = Institutional Review Boards; JAFROC = Jackknife free-response receiver operating characteristic.
Footnotes
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Dongjun Choi is currently an employee of lululab Inc., Seoul, Korea. Leonard Sunwoo is currently employed part-time at JLK Inc., Seoul, Korea. Tackeun Kim reports a relationship with TALOS Corp., Seoul, Korea, that includes: equity or stocks. Kyong Joon Lee reports a relationship with Monitor Corporation, Seoul, Korea, that includes: equity or stocks. Dongjun Choi, Leonard Sunwoo, Tackeun Kim, Kyong Joon Lee are listed as inventors on a patent related to the work. The patentee is Seoul National University Hospital, Seoul, Korea.
- © 2024 by American Journal of Neuroradiology