Skip to main content
Advertisement
  • Main menu
  • User menu
  • Search
  • English ▼
    • English
    • Afrikaans
    • Albanian
    • Amharic
    • Arabic
    • Armenian
    • Azerbaijani
    • Basque
    • Belarusian
    • Bengali
    • Bosnian
    • Bulgarian
    • Catalan
    • Cebuano
    • Chichewa
    • Chinese (Simplified)
    • Chinese (Traditional)
    • Corsican
    • Croatian
    • Czech
    • Danish
    • Dutch
    • Esperanto
    • Estonian
    • Filipino
    • Finnish
    • French
    • Frisian
    • Galician
    • Georgian
    • German
    • Greek
    • Gujarati
    • Haitian Creole
    • Hausa
    • Hawaiian
    • Hebrew
    • Hindi
    • Hmong
    • Hungarian
    • Icelandic
    • Igbo
    • Indonesian
    • Irish
    • Italian
    • Japanese
    • Javanese
    • Kannada
    • Kazakh
    • Khmer
    • Korean
    • Kurdish (Kurmanji)
    • Kyrgyz
    • Lao
    • Latin
    • Latvian
    • Lithuanian
    • Luxembourgish
    • Macedonian
    • Malagasy
    • Malay
    • Malayalam
    • Maltese
    • Maori
    • Marathi
    • Mongolian
    • Myanmar (Burmese)
    • Nepali
    • Norwegian
    • Pashto
    • Persian
    • Polish
    • Portuguese
    • Punjabi
    • Romanian
    • Russian
    • Samoan
    • Scottish Gaelic
    • Serbian
    • Sesotho
    • Shona
    • Sindhi
    • Sinhala
    • Slovak
    • Slovenian
    • Somali
    • Spanish
    • Sudanese
    • Swahili
    • Swedish
    • Tajik
    • Tamil
    • Telugu
    • Thai
    • Turkish
    • Ukrainian
    • Urdu
    • Uzbek
    • Vietnamese
    • Welsh
    • Xhosa
    • Yiddish
    • Yoruba
    • Zulu

Main menu

  • Home
  • Content
    • Current Issue
    • Accepted Manuscripts
    • Article Preview
    • Past Issue Archive
    • Video Articles
    • AJNR Case Collection
    • Case of the Week Archive
    • Case of the Month Archive
    • Classic Case Archive
  • Special Collections
    • AJNR Awards
    • Low-Field MRI
    • Alzheimer Disease
    • ASNR Foundation Special Collection
    • Photon-Counting CT
    • View All
  • Multimedia
    • AJNR Podcasts
    • AJNR SCANtastic
    • Trainee Corner
    • MRI Safety Corner
    • Imaging Protocols
  • For Authors
    • Submit a Manuscript
    • Submit a Video Article
    • Submit an eLetter to the Editor/Response
    • Manuscript Submission Guidelines
    • Statistical Tips
    • Fast Publishing of Accepted Manuscripts
    • Graphical Abstract Preparation
    • Imaging Protocol Submission
    • Author Policies
  • About Us
    • About AJNR
    • Editorial Board
    • Editorial Board Alumni
  • More
    • Become a Reviewer/Academy of Reviewers
    • Subscribers
    • Permissions
    • Alerts
    • Feedback
    • Advertisers
    • ASNR Home

User menu

  • Alerts
  • Log in

Search

  • Advanced search
American Journal of Neuroradiology
American Journal of Neuroradiology

American Journal of Neuroradiology

ASHNR American Society of Functional Neuroradiology ASHNR American Society of Pediatric Neuroradiology ASSR
  • Alerts
  • Log in

Advanced Search

English ▼
  • English
  • Afrikaans
  • Albanian
  • Amharic
  • Arabic
  • Armenian
  • Azerbaijani
  • Basque
  • Belarusian
  • Bengali
  • Bosnian
  • Bulgarian
  • Catalan
  • Cebuano
  • Chichewa
  • Chinese (Simplified)
  • Chinese (Traditional)
  • Corsican
  • Croatian
  • Czech
  • Danish
  • Dutch
  • Esperanto
  • Estonian
  • Filipino
  • Finnish
  • French
  • Frisian
  • Galician
  • Georgian
  • German
  • Greek
  • Gujarati
  • Haitian Creole
  • Hausa
  • Hawaiian
  • Hebrew
  • Hindi
  • Hmong
  • Hungarian
  • Icelandic
  • Igbo
  • Indonesian
  • Irish
  • Italian
  • Japanese
  • Javanese
  • Kannada
  • Kazakh
  • Khmer
  • Korean
  • Kurdish (Kurmanji)
  • Kyrgyz
  • Lao
  • Latin
  • Latvian
  • Lithuanian
  • Luxembourgish
  • Macedonian
  • Malagasy
  • Malay
  • Malayalam
  • Maltese
  • Maori
  • Marathi
  • Mongolian
  • Myanmar (Burmese)
  • Nepali
  • Norwegian
  • Pashto
  • Persian
  • Polish
  • Portuguese
  • Punjabi
  • Romanian
  • Russian
  • Samoan
  • Scottish Gaelic
  • Serbian
  • Sesotho
  • Shona
  • Sindhi
  • Sinhala
  • Slovak
  • Slovenian
  • Somali
  • Spanish
  • Sudanese
  • Swahili
  • Swedish
  • Tajik
  • Tamil
  • Telugu
  • Thai
  • Turkish
  • Ukrainian
  • Urdu
  • Uzbek
  • Vietnamese
  • Welsh
  • Xhosa
  • Yiddish
  • Yoruba
  • Zulu
  • Home
  • Content
    • Current Issue
    • Accepted Manuscripts
    • Article Preview
    • Past Issue Archive
    • Video Articles
    • AJNR Case Collection
    • Case of the Week Archive
    • Case of the Month Archive
    • Classic Case Archive
  • Special Collections
    • AJNR Awards
    • Low-Field MRI
    • Alzheimer Disease
    • ASNR Foundation Special Collection
    • Photon-Counting CT
    • View All
  • Multimedia
    • AJNR Podcasts
    • AJNR SCANtastic
    • Trainee Corner
    • MRI Safety Corner
    • Imaging Protocols
  • For Authors
    • Submit a Manuscript
    • Submit a Video Article
    • Submit an eLetter to the Editor/Response
    • Manuscript Submission Guidelines
    • Statistical Tips
    • Fast Publishing of Accepted Manuscripts
    • Graphical Abstract Preparation
    • Imaging Protocol Submission
    • Author Policies
  • About Us
    • About AJNR
    • Editorial Board
    • Editorial Board Alumni
  • More
    • Become a Reviewer/Academy of Reviewers
    • Subscribers
    • Permissions
    • Alerts
    • Feedback
    • Advertisers
    • ASNR Home
  • Follow AJNR on Twitter
  • Visit AJNR on Facebook
  • Follow AJNR on Instagram
  • Join AJNR on LinkedIn
  • RSS Feeds

AJNR Awards, New Junior Editors, and more. Read the latest AJNR updates

Research ArticleNEUROVASCULAR/STROKE IMAGING

A Clinical and Imaging Fused Deep Learning Model Matches Expert Clinician Prediction of 90-Day Stroke Outcomes

Yongkai Liu, Preya Shah, Yannan Yu, Jai Horsey, Jiahong Ouyang, Bin Jiang, Guang Yang, Jeremy J. Heit, Margy E. McCullough-Hicks, Stephen M. Hugdal, Max Wintermark, Patrik Michel, David S. Liebeskind, Maarten G. Lansberg, Gregory W. Albers and Greg Zaharchuk
American Journal of Neuroradiology February 2024, DOI: https://doi.org/10.3174/ajnr.A8140
Yongkai Liu
aFrom the Department of Radiology (Y.L., P.S., Y.Y., J.O., B.J., J.J.H., S.M.H., G.Z.), Stanford University, Stanford, Calfornia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Yongkai Liu
Preya Shah
aFrom the Department of Radiology (Y.L., P.S., Y.Y., J.O., B.J., J.J.H., S.M.H., G.Z.), Stanford University, Stanford, Calfornia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yannan Yu
aFrom the Department of Radiology (Y.L., P.S., Y.Y., J.O., B.J., J.J.H., S.M.H., G.Z.), Stanford University, Stanford, Calfornia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jai Horsey
bMeharry Medical College (J.H.), Nashville, Tennessee
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jiahong Ouyang
aFrom the Department of Radiology (Y.L., P.S., Y.Y., J.O., B.J., J.J.H., S.M.H., G.Z.), Stanford University, Stanford, Calfornia
cDepartment of Electrical Engineering (J.O.), Stanford University, Stanford, California
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jiahong Ouyang
Bin Jiang
aFrom the Department of Radiology (Y.L., P.S., Y.Y., J.O., B.J., J.J.H., S.M.H., G.Z.), Stanford University, Stanford, Calfornia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Bin Jiang
Guang Yang
dNational Heart and Lung Institute (G.Y.), Imperial College London, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Guang Yang
Jeremy J. Heit
aFrom the Department of Radiology (Y.L., P.S., Y.Y., J.O., B.J., J.J.H., S.M.H., G.Z.), Stanford University, Stanford, Calfornia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jeremy J. Heit
Margy E. McCullough-Hicks
eDepartment of Neurology (M.E.M.-H.), University of Minnesota Medical School, Minneapolis, Minnesota
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stephen M. Hugdal
aFrom the Department of Radiology (Y.L., P.S., Y.Y., J.O., B.J., J.J.H., S.M.H., G.Z.), Stanford University, Stanford, Calfornia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Max Wintermark
fDepartment of Neuroradiology (M.W.), University of Texas MD Anderson Center, Houston, Texas
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Max Wintermark
Patrik Michel
gNeurology Service (P.M), Department of Clinical Neurosciences, Lausanne University Hospital and University of Lausanne, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Patrik Michel
David S. Liebeskind
hDepartment of Neurology (D.S.L.), University of California, Los Angeles, Los Angeles, Calfornia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for David S. Liebeskind
Maarten G. Lansberg
iDepartment of Neurology (M.G.L., G.W.A.), Stanford, Stanford, Calfornia.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Maarten G. Lansberg
Gregory W. Albers
iDepartment of Neurology (M.G.L., G.W.A.), Stanford, Stanford, Calfornia.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Greg Zaharchuk
aFrom the Department of Radiology (Y.L., P.S., Y.Y., J.O., B.J., J.J.H., S.M.H., G.Z.), Stanford University, Stanford, Calfornia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Greg Zaharchuk
  • Article
  • Figures & Data
  • Info & Metrics
  • Responses
  • References
  • PDF
Loading

Article Figures & Data

Figures

  • Tables
  • FIG 1.
    • Download figure
    • Open in new tab
    • Download powerpoint
    FIG 1.

    Flow chart for patients in the current study.

  • FIG 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    FIG 2.

    MR images (the first and second columns represent DWI and B0 images, respectively) for 3 patients with diverse clinical histories and 90-day mRS scores. Patient A is a 48-year-old man with a baseline NIHSS of 11, 24-hour NIHSS of 5, and a 90-day mRS of 1. He has no medical history of either diabetes or hypertension. The DL model accurately predicted his score. However, the readers overestimated his score by 1 point. Patient B, a 75-year-old woman, has a medical history that includes diabetes and hypertension and a 90-day mRS of 5. Both the DL model and the readers accurately predicted her 90-day mRS score of 5. Patient C, a 41-year-old man with no history of diabetes or hypertension, has a 90-day mRS score of 6. However, both the DL model and the readers incorrectly predicted his 90-day mRS score as 1. HTN indicates hypertension; DM. diabetes mellitus.

  • FIG 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    FIG 3.

    The AUC of the DL-based predictive model for predicting unfavorable outcomes (mRS >2) is shown alongside data points representing the performance of individual clinicians and the consensus of clinicians. The translucent blue region denotes the 95% confidence interval for the ROC curve, constructed using bootstrapping.

Tables

  • Figures
    • View popup
    Table 1:

    Summary of the characteristics of patients with AIS included in the Stanford University Hospital cohort (n = 80)a

    Summary
    Characteristics
     Male44 (55.0)
    Age (yr) (median) (IQR)62 (51–75)
     History of hypertension53 (66.3)
     History of diabetes20 (25.0)
    Baseline NIHSS (median) (IQR)12 (7–17)
     24-Hour NIHSS9 (4–17) 3.8%b
    Days after stroke for MR imaging (median) (IQR)1 (1–3)
    90-Day outcome
    Favorable outcome (90-Day mRS≤2)36 (45.0)
    Unfavorable outcome (90-Day mRS >2)44 (55.0)
    • ↵a Unless otherwise mentioned, data are expressed as number (percentage) of patients.

    • ↵b Percentage of variables missing. If no data are missing, then there will be no percentage reported.

    • View popup
    Table 2:

    MRS scorea

    ScalePre morbid mRS90-Day mRS
    067 (83.8)6 (7.5)
    16 (7.5)17 (21.3)
    23 (3.8)13 (16.3)
    34 (5.0)19 (23.8)
    40 (0.0)12 (15.0)
    50 (0.0)10 (12.5)
    60 (0.0)3 (3.8)
    • ↵a Data are expressed as number (percentage) of patients.

    • View popup
    Table 3:

    Performance comparisons for ordinal mRS prediction between the DLPD and the clinical readersa

    Gwet ACMAE±1ACC (%)ACC (%)
    Model/readers
     Neuroradiologist I0.70 (0.60–0.80)1.15 (0.94–1.38)71 (60–81)26 (18–36)
     Neuroradiologist II0.69 (0.59–0.79)1.14 (0.93–1.38)70 (60–80)29 (19–39)
     Neuroradiology fellow0.73 (0.65–0.81)1.04 (0.85–1.24)74 (6–84)31 (21–41)
    Clinical readers
     Neurologist I0.75 (0.66–0.84)1.03 (0.83–1.25)75 (65–84)32 (22–44)
     Neurologist II0.77 (0.67–0.86)0.91 (0.70–1.15)79 (69–88)41 (30–51)
    Consensus read0.76 (0.67–0.84)0.95 (0.75–1.17)79 (70–88)36 (25–46)
    DLPD0.79 (0.71–0.86)0.89 (0.70–1.11)81 (73–90)36 (26–46)
    P valueP < .001P = .02P < .001P = .07
    • ↵a The data in the parentheses represent the 95% confidence interval. The P value is for the noninferiority test between the consensus clinical reads and the DLPD with the predefined margin of 0.1 (MAE)/10% (±1ACC, ACC). The Gwet AC for agreement among 5 clinical readers is 0.83 (95% CI, 0.80–0.86), justifying the comparison with a consensus.

    • View popup
    Table 4:

    Performance comparisons for unfavorable-outcome prediction (mRS >2) between the DLPD and the clinical readersa

    AUCSensitivitySpecificity
    Model/readers
     Neuroradiologist I0.76 (0.64–0.85)0.68 (0.53–0.81)0.69 (0.54–0.83)
     Neuroradiologist II0.81 (0.71–0.89)0.61 (0.47–0.77)0.86 (0.73–0.97)
     Neuroradiology fellow0.82 (0.72–0.89)0.75 (0.62–0.87)0.69 (0.53–0.83)
    Clinical readers
     Neurologist I0.79 (0.69–0.88)0.82 (0.7–0.93)0.53 (0.36–0.68)
     Neurologist II0.77 (0.67–0.86)0.64 (0.49–0.78)0.83 (0.7–0.95)
    Consensus read0.79 (0.68–0.87)0.70 (0.56–0.83)0.75 (0.60–0.88)
    DLPD0.81 (0.72–0.89)0.68 (0.54–0.81)0.81 (0.67–0.92)
    P valueP = .005P = .25P = .03
    • ↵a The data in the parentheses represent the 95% confidence interval. The P value is for the noninferiority test between the consensus clinical read and the DLPD (predefined margin, .05).

PreviousNext
Back to top
Advertisement
Print
Download PDF
Email Article
Cite this article
0 Responses
Respond to this article
Share
Bookmark this article
A Clinical and Imaging Fused Deep Learning Model Matches Expert Clinician Prediction of 90-Day Stroke Outcomes
Yongkai Liu, Preya Shah, Yannan Yu, Jai Horsey, Jiahong Ouyang, Bin Jiang, Guang Yang, Jeremy J. Heit, Margy E. McCullough-Hicks, Stephen M. Hugdal, Max Wintermark, Patrik Michel, David S. Liebeskind, Maarten G. Lansberg, Gregory W. Albers, Greg Zaharchuk
American Journal of Neuroradiology Feb 2024, DOI: 10.3174/ajnr.A8140
del.icio.us logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
Purchase

Jump to section

  • Article
    • Abstract
    • ABBREVIATIONS:
    • MATERIALS AND METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSIONS
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • Responses
  • References
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • No citing articles found.
  • Crossref
  • Google Scholar

This article has not yet been cited by articles in journals that are participating in Crossref Cited-by Linking.

More in this TOC Section

  • 90-day Outcome Prediction in VBAO after EVT
  • Circle of Willis Variants and Stroke Outcomes
  • IVIM MRI in the Ischemic Penumbra
Show more Neurovascular/Stroke Imaging

Similar Articles

Advertisement

Indexed Content

  • Current Issue
  • Accepted Manuscripts
  • Article Preview
  • Past Issues
  • Editorials
  • Editor's Choice
  • Fellows' Journal Club
  • Letters to the Editor
  • Video Articles

Cases

  • Case Collection
  • Archive - Case of the Week
  • Archive - Case of the Month
  • Archive - Classic Case

More from AJNR

  • Trainee Corner
  • Imaging Protocols
  • MRI Safety Corner
  • Book Reviews

Multimedia

  • AJNR Podcasts
  • AJNR Scantastics

Resources

  • Turnaround Time
  • Submit a Manuscript
  • Submit a Video Article
  • Submit an eLetter to the Editor/Response
  • Manuscript Submission Guidelines
  • Statistical Tips
  • Fast Publishing of Accepted Manuscripts
  • Graphical Abstract Preparation
  • Imaging Protocol Submission
  • Evidence-Based Medicine Level Guide
  • Publishing Checklists
  • Author Policies
  • Become a Reviewer/Academy of Reviewers
  • News and Updates

About Us

  • About AJNR
  • Editorial Board
  • Editorial Board Alumni
  • Alerts
  • Permissions
  • Not an AJNR Subscriber? Join Now
  • Advertise with Us
  • Librarian Resources
  • Feedback
  • Terms and Conditions
  • AJNR Editorial Board Alumni

American Society of Neuroradiology

  • Not an ASNR Member? Join Now

© 2025 by the American Society of Neuroradiology All rights, including for text and data mining, AI training, and similar technologies, are reserved.
Print ISSN: 0195-6108 Online ISSN: 1936-959X

Powered by HighWire
Email this Article

Thank you for your interest in spreading the word on American Journal of Neuroradiology.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
A Clinical and Imaging Fused Deep Learning Model Matches Expert Clinician Prediction of 90-Day Stroke Outcomes
(Your Name) has sent you a message from American Journal of Neuroradiology
(Your Name) thought you would like to see the American Journal of Neuroradiology web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Cite this article
Yongkai Liu, Preya Shah, Yannan Yu, Jai Horsey, Jiahong Ouyang, Bin Jiang, Guang Yang, Jeremy J. Heit, Margy E. McCullough-Hicks, Stephen M. Hugdal, Max Wintermark, Patrik Michel, David S. Liebeskind, Maarten G. Lansberg, Gregory W. Albers, Greg Zaharchuk
A Clinical and Imaging Fused Deep Learning Model Matches Expert Clinician Prediction of 90-Day Stroke Outcomes
American Journal of Neuroradiology Feb 2024, DOI: 10.3174/ajnr.A8140

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero

We use cookies on this site to enhance your user experience. By clicking any link on this page you are giving your consent for us to set cookies.