AI-Assisted Summarization of Radiologic Reports: Evaluating GPT3davinci, BARTcnn, LongT5booksum, LEDbooksum, LEDlegal, and LEDclinical

REFERENCES

1. 1.↵
   2. Shen Y,
   3. Heacock L,
   4. Elias J, et al

   . ChatGPT and other large language models are double-edged swords. Radiology 2023;307:e230163 doi:10.1148/radiol.230163 pmid:36700838

   CrossRefPubMed
2. 2.↵
   2. Akinci D’Antonoli T,
   3. Stanzione A,
   4. Bluethgen C, et al

   . Large language models in radiology: fundamentals, applications, ethical considerations, risks, and future directions. Diagn Interv Radiol 2023 Oct 3. [Epub ahead of print] doi:10.4274/dir.2023.232417 pmid:37789676

   CrossRefPubMed
3. 3.↵
   2. Tippareddy C,
   3. Jiang S,
   4. Bera K, et al

   . Radiology reading room for the future: harnessing the power of large language models like ChatGPT. Curr Probl Diagn Radiol 2023 Aug 30. [Epub ahead of print] doi:10.1067/j.cpradiol.2023.08.018 pmid:37758604

   CrossRefPubMed
4. 4.↵
   2. Moharasan G,
   3. Ho TB

   . Extraction of temporal information from clinical narratives. J Healthc Inform Res 2019;3:220–44 doi:10.1007/s41666-019-00049-0 pmid:35415423

   CrossRefPubMed
5. 5.↵
   2. Mishra R,
   3. Bian J,
   4. Fiszman M, et al

   . Text summarization in the biomedical domain: a systematic review of recent research. J Biomed Inform 2014;52:457–67 doi:10.1016/j.jbi.2014.06.009 pmid:25016293

   CrossRefPubMed
6. 6.↵
   2. Sarzynski E,
   3. Hashmi H,
   4. Subramanian J, et al

   . Opportunities to improve clinical summaries for patients at hospital discharge. BMJ Qual Saf 2017;26:372–80 pmid:27154878

   Abstract/FREE Full Text
7. 7.↵
   2. Chien A,
   3. Callender RA,
   4. Yokota H, et al

   . Unruptured intracranial aneurysm growth trajectory: occurrence and rate of enlargement in 520 longitudinally followed cases. J Neurosurg 2019;132:1077–87 doi:10.3171/2018.11.JNS181814 pmid:30835694

   CrossRefPubMed
8. 8.↵
   2. Eskey CJ,
   3. Meyers PM,
   4. Nguyen TN

   , American Heart Association Council on Cardiovascular Radiology and Intervention and Stroke Council, et al. Indications for the performance of intracranial endovascular neurointerventional procedures: a scientific statement from the American Heart Association. Circulation 2018;137:e661–89 doi:10.1161/CIR.0000000000000567 pmid:29674324

   Abstract/FREE Full Text
9. 9.↵
   2. Lewis M,
   3. Liu Y,
   4. Goyal N, et al

   . BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020 doi:10.18653/v1/2020.acl-main.703

   CrossRef
10. 10.↵
    2. Guo M,
    3. Ainslie J,
    4. Uthus D, et al

    . Longt5: efficient text-to-text transformer for long sequences. Findings of the Association for Computational Linguistics: NAACL 2022. 2022 doi:10.18653/v1/2022.findings-naacl.55

    CrossRef
11. 11.↵
    2. Alfattni G,
    3. Peek N,
    4. Nenadic G

    . Extraction of temporal relations from clinical free text: a systematic review of current approaches. J Biomed Inform 2020;108:103488 doi:10.1016/j.jbi.2020.103488 pmid:32673788

    CrossRefPubMed
12. 12.↵
    2. Bhandari M,
    3. Gour PN,
    4. Ashfaq A, et al

    . Re-evaluating evaluation in text summarization. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020 doi:10.18653/v1/2020.emnlp-main.751
13. 13.↵
    2. Goldstein A,
    3. Shahar Y

    . An automated knowledge-based textual summarization system for longitudinal, multivariate clinical data. J Biomed Inform 2016;61:159–75 doi:10.1016/j.jbi.2016.03.022 pmid:27039119

    CrossRefPubMed
14. 14.↵
    2. Beltagy I,
    3. Peters ME,
    4. Cohan A

    . Longformer: the long-document transformer. arXiv 2020;2004.05150
15. 15.↵
    2. Adams G,
    3. Shing HC,
    4. Sun Q, et al

    . Learning to revise references for faithful summarization. arXiv 2022;2204.10290
16. 16.↵
    2. Goyal T,
    3. Jessy Li J,
    4. Durrett G

    . News summarization and evaluation in the era of gpt-3. arXiv 2022;2209.12356
17. 17.↵
    2. Lin CY

    . Rouge: a package for automatic evaluation of summaries. Text Summarization Branches Out 2004:74–81
18. 18.↵
    2. Zhang T,
    3. Kishore V,
    4. Wu F, et al

    . Bertscore: evaluating text generation with Bert. arXiv 2019;1904.09675
19. 19.↵
    2. Hirsch JS,
    3. Tanenbaum JS,
    4. Lipsky Gorman S, et al

    . Harvest, a longitudinal patient record summarizer. J Am Med Inform Assoc 2015;22:263–74 doi:10.1136/amiajnl-2014-002945 pmid:25352564

    CrossRefPubMed
20. 20.↵
    2. Percha B

    . Modern clinical text mining: a guide and review. Annu Rev Biomed Data Sci 2021;4:165–87 doi:10.1146/annurev-biodatasci-030421-030931 pmid:34465177

    CrossRefPubMed
21. 21.↵
    2. Sun W,
    3. Cai Z,
    4. Li Y, et al

    . Data processing and text mining technologies on electronic medical records: a review. J Healthc Eng 2018;2018:4302425 doi:10.1155/2018/4302425 pmid:29849998

    CrossRefPubMed
22. 22.↵
    2. Johnson AE,
    3. Pollard TJ,
    4. Shen L, et al

    . Mimic-III, a freely accessible critical care database. Sci Data 2016;3:160035 doi:10.1038/sdata.2016.35 pmid:27219127

    CrossRefPubMed
23. 23.↵
    2. Fu S,
    3. Leung LY,
    4. Wang Y, et al

    . Natural language processing for the identification of silent brain infarcts from neuroimaging reports. JMIR Med Inform 2019;7:e12109 doi:10.2196/12109 pmid:31066686

    CrossRefPubMed
24. 24.↵
    2. Kim C,
    3. Zhu V,
    4. Obeid J, et al

    . Natural language processing and machine learning algorithm to identify brain MRI reports with acute ischemic stroke. PLoS One 2019;14:e0212778 doi:10.1371/journal.pone.0212778 pmid:30818342

    CrossRefPubMed
25. 25.↵
    2. Wheater E,
    3. Mair G,
    4. Sudlow C, et al

    . A validated natural language processing algorithm for brain imaging phenotypes from radiology reports in UK electronic health records. BMC Med Inform Decis Mak 2019;19:184 doi:10.1186/s12911-019-0908-7 pmid:31500613

    CrossRefPubMed
26. 26.↵
    2. Sun W,
    3. Rumshisky A,
    4. Uzuner O

    . Evaluating temporal relations in clinical text: 2012 i2b2 challenge. J Am Med Inform Assoc 2013;20:806–13 doi:10.1136/amiajnl-2013-001628 pmid:23564629

    CrossRefPubMedWeb of Science
27. 27.↵
    2. Moskovitch R,
    3. Shahar Y,
    4. Wang F, et al

    . Temporal biomedical data analytics. J Biomed Inform 2019;90:103092 doi:10.1016/j.jbi.2018.12.006 pmid:30654029

    CrossRefPubMed
28. 28.↵
    2. Devlin J,
    3. Chang MW,
    4. Lee K, et al

    . Bert: pre-training of deep bidirectional transformers for language understanding. arXiv 2019;1810.04805
29. 29.↵
    2. Huang K,
    3. Altosaar J,
    4. Ranganath R

    . Clinicalbert: modeling clinical notes and predicting hospital readmission. arXiv 2020;1904.05342
30. 30.↵
    2. Yan A,
    3. McAuley J,
    4. Lu X, et al

    . Radbert: adapting transformer-based language models to radiology. Radiol Artif Intell 2022;4:e210258 doi:10.1148/ryai.210258 pmid:35923376

    CrossRefPubMed
31. 31.↵
    2. Styler IW,
    3. Bethard S,
    4. Finan S, et al

    . Temporal annotation in the clinical domain. TACL 2014;2:143–54 doi:10.1162/tacl_a_00172 pmid:29082229

    CrossRefPubMed
32. 32.↵
    2. Monajatipoor M,
    3. Rouhsedaghat M,
    4. Li LH, et al

    . Berthop: an effective vision-and-language model for chest x-ray disease diagnosis. Med Image Comput Comput Assist Interv 2022;13435:725–34 doi:10.1007/978-3-031-16443-9_69 pmid:37093922

    CrossRefPubMed