Toward Improved Radiologic Diagnostics: Investigating the Utility and Limitations of GPT-3.5 Turbo and GPT-4 with Quiz Cases

References

1. 1.↵
   1. Youssef A,
   2. Ng MY,
   3. Long J, et al

   . Organizational factors in clinical data sharing for artificial intelligence in health care. JAMA Netw Open 2023;6:e2348422 doi:10.1001/jamanetworkopen.2023.48422 pmid:38113040

   CrossRefPubMed
2. 2.↵
   1. Clusmann J,
   2. Kolbinger FR,
   3. Muti HS, et al

   . The future landscape of large language models in medicine. Commun Med (Lond) 2023;3:141 doi:10.1038/s43856-023-00370-1 pmid:37816837

   CrossRefPubMed
3. 3.↵
   1. Shaikh O,
   2. Zhang H,
   3. Held W, et al

   . On second thought, let’s not think step by step! Bias and toxicity in zero-shot reasoning. Presented at: Annual Meeting of the Association for Computational Linguistics; July 2023; Toronto, Canada. https://aclanthology.org/2023.acl-long.244/ doi:10.18653/v1/2023.acl-long.244

   CrossRef
4. 4.↵
   OpenAI. GPT-4 Technical Report. arXiv [csCL] 2023 Mar 15; epub ahead of print.
5. 5.↵
   1. Suthar PP,
   2. Kounsal A,
   3. Chhetri L, et al

   . Artificial intelligence (AI) in radiology: a deep dive into ChatGPT 4.0’s accuracy with the American Journal of Neuroradiology’s (AJNR) “Case of the Month. Cureus 2023;15:e43958 doi:10.7759/cureus.43958 pmid:37746411

   CrossRefPubMed
6. 6.↵
   1. Ueda D,
   2. Mitsuyama Y,
   3. Takita H, et al

   . ChatGPT’s diagnostic performance from patient history and imaging findings on the diagnosis please quizzes. Radiology 2023;308:e231040 doi:10.1148/radiol.231040 pmid:37462501

   CrossRefPubMed
7. 7.↵
   1. Horiuchi D,
   2. Tatekawa H,
   3. Shimono T, et al

   . Accuracy of ChatGPT generated diagnosis from patient’s medical history and imaging findings in neuroradiology cases. Neuroradiology 2024;66:73–79 doi:10.1007/s00234-023-03252-4 pmid:37994939

   CrossRefPubMed
8. 8.↵
   1. Albrecht J,
   2. Meves A,
   3. Bigby M

   . Case reports and case series from Lancet had significant impact on medical literature. J Clin Epidemiol 2005;58:1227–32.doi:10.1016/j.jclinepi.2005.04.003 pmid:16291466

   CrossRefPubMed
9. 9.↵
   1. Nissen T,
   2. Wynn R

   . The clinical case report: a review of its merits and limitations. BMC Res Notes 2014;7:264 doi:10.1186/1756-0500-7-264 pmid:24758689

   CrossRefPubMed
10. 10.↵
    1. Cohen JF,
    2. Korevaar DA,
    3. Altman DG, et al

    . STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open 2016;6:e012799 doi:10.1136/bmjopen-2016-012799 pmid:28137831

    Abstract/FREE Full Text
11. 11.↵
    1. Grewal H,
    2. Dhillon G,
    3. Monga V, et al

    . Radiology gets chatty: the ChatGPT saga unfolds. Cureus 2023;15:e40135 doi:10.7759/cureus.40135 pmid:37425598

    CrossRefPubMed
12. 12.↵
    1. Kim S,
    2. Lee CK,
    3. Kim SS

    . Large language models: A guide for radiologists. Korean J Radiology 2024;25:126–33 doi:10.3348/kjr.2023.0997 pmid:38288895

    CrossRefPubMed
13. 13.↵
    1. Nakaura T,
    2. Yoshida N,
    3. Kobayashi N, et al

    . Preliminary assessment of automated radiology report generation with generative pre-trained transformers: comparing results to radiologist-generated reports. Jpn J Radiology 2024;42:190–200 doi:10.1007/s11604-023-01487-y pmid:37713022

    CrossRefPubMed
14. 14.↵
    1. Adams LC,
    2. Truhn D,
    3. Busch F, et al

    . Leveraging GPT-4 for post hoc transformation of free-text radiology reports into structured reporting: A multilingual feasibility study. Radiology 2023;307:e230725 doi:10.1148/radiol.230725 pmid:37014240

    CrossRefPubMed
15. 15.↵
    1. Lyu Q,
    2. Tan J,
    3. Zapadka ME, et al

    . Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential. Vis Comput Ind Biomed Art 2023;6:9 doi:10.1186/s42492-023-00136-5 pmid:37198498

    CrossRefPubMed
16. 16.↵
    1. Nakamura Y,
    2. Kikuchi T,
    3. Yamagishi Y, et al

    . ChatGPT for Automating Lung Cancer Staging: Feasibility Study on Open Radiology Report Data set. medRxiv doi:10.1101/2023.12.11.23299107

    Abstract/FREE Full Text
17. 17.↵
    1. Gunes YC,
    2. Cesur T

    . A Comparative Study: Diagnostic Performance of ChatGPT. Google, Microsoft Bing, and Radiologists in Thoracic Radiology cases. bioRxiv:3.5. doi:10.1101/2024.01.18.24301495

    CrossRef
18. 18.↵
    1. Schramowski P,
    2. Turan C,
    3. Andersen N, et al

    . Large pre-trained language models contain human-like biases of what is right and wrong to do. Nat Mach Intell 2022;4:258–68 doi:10.1038/s42256-022-00458-8

    CrossRef
19. 19.↵
    1. Pal A,
    2. Umapathi LK,
    3. Sankarasubbu M

    . Med-HALT: Medical Domain Hallucination Test for Large Language Models. arXiv: 2307.15343 [csCL] doi:10.18653/v1/2023.conll-1.21

    CrossRef
20. 20.↵
    1. Karabacak M,
    2. Margetis K

    . Embracing large language models for medical applications: opportunities and challenges. Cureus 2023;15:e39305 doi:10.7759/cureus.39305 pmid:37378099

    CrossRefPubMed
21. 21.↵
    1. Walters WH,
    2. Wilder EI

    . Fabrication and errors in the bibliographic citations generated by ChatGPT. Sci Rep 2023;13:14045 doi:10.1038/s41598-023-41032-5 pmid:37679503

    CrossRefPubMed
22. 22.↵
    1. McGowan A,
    2. Gui Y,
    3. Dobbs M, et al

    . ChatGPT and Bard exhibit spontaneous citation fabrication during psychiatry literature search. Psychiatry Res 2023;326:115334 doi:10.1016/j.psychres.2023.115334 pmid:37499282

    CrossRefPubMed
23. 23.↵
    1. Meskó B

    . The impact of multimodal large language models on health care’s future. J Med Internet Res 2023;25:e52865 doi:10.2196/52865 pmid:37917126

    CrossRefPubMed
24. 24.↵
    1. Meskó B

    . Prompt engineering as an important emerging skill for medical professionals: tutorial. J Med Internet Res 2023;25:e50638 doi:10.2196/50638 pmid:37792434

    CrossRefPubMed