Back to the list
Congress: ECR25
Poster Number: C-25294
Type: Poster: EPOS Radiologist (scientific)
Authorblock: J. F. Ojeda Esparza1, D. Botta1, A. Fitisiori1, C. Santarosa1, M. Pucci1, C. Meinzer2, Y-C. Yun2, K-O. Loevblad1, F. T. Kurz1; 1Geneva/CH, 2Heidelberg/DE
Disclosures:
Jose Federico Ojeda Esparza: Nothing to disclose
Daniele Botta: Nothing to disclose
Aikaterini Fitisiori: Nothing to disclose
Corrado Santarosa: Nothing to disclose
Marcella Pucci: Nothing to disclose
Clara Meinzer: Nothing to disclose
Yeong-Chul Yun: Nothing to disclose
Karl-Olof Loevblad: Nothing to disclose
Felix T Kurz: Nothing to disclose
Keywords: Artificial Intelligence, CNS, Catheter arteriography, CT, MR, Computer Applications-General, Diagnostic procedure, Technology assessment, Education and training, Image verification
Purpose Artificial intelligence (AI) tools are evolving rapidly, with large language models (LLMs) now demonstrating the ability to process both text and image-based data. In radiology, AI has primarily focused on task-specific solutions for image analysis and diagnostic support [1]. However, recent developments in multimodal models have introduced LLMs capable of integrating text and image inputs, opening new possibilities for clinical applications.Unlike conventional AI algorithms, which are designed for specific imaging tasks mainly interpretative and workflow-enhancing deep learning algorithms, LLMs offer...
Read more Methods and materials A total of 104 image-based multiple-choice questions (MCQs) from Radiopaedia.org were analyzed, covering various neuroradiological imaging modalities, including CT, MRI, DSA, and conventional radiography (Fig. 1). Each question was presented five times to two large language models (LLMs) to ensure consistent evaluation. Additionally, global response data from Radiopaedia.org were collected as a comparative benchmark [8].The study included both human participants and artificial intelligence models. Among the human participants were three expert radiologists (experienced neuroradiologists from an university hospital with over...
Read more Results Accuracies were calculated for each question across all groups, including neuroradiologists, trainee radiologists, large language models (LLMs), and global response data (Fig. 5). Neuroradiologists achieved the highest accuracy (0.911 ± 0.02), significantly outperforming all other groups. In contrast, LLM-GAG demonstrated the lowest accuracy (0.50 ± 0.03), close to chance level, while LLM-GPT (0.64 ± 0.05) and global response data (0.69 ± 0.18) showed intermediate performance, with comparable accuracy levels. Trainee radiologists had a moderate accuracy rate (0.57 ± 0.04), but...
Read more Conclusion Expert neuroradiologists outperformed all other groups. However, global response data appeared more accurate than that of residents, possibly due to variability in response volume. LLM-GPT achieved higher accuracy than residents, while LLM-GAG showed the lowest performance, emphasizing the need for further improvements in AI tools. These findings are consistent with previous studies evaluating the accuracy of large language models (LLMs) in medical question-answering tasks [5–7]. In this study, the inclusion of images in the questions did not appear to affect...
Read more References Clinical Imaging. Perceptions and applications of artificial intelligence in radiology: A cross-sectional survey study. Clin Imaging. 2021;42(5):1–10. doi:10.1016/j.clinimag.2021.09.018. Bajaj S, Gandhi D, Nayar D. Potential applications and impact of ChatGPT in radiology. Acad Radiol. 2024;31(4):1256–61. doi:10.1016/j.acra.2023.08.039. OpenAI. GPT-4 Technical Report. Available at: https://openai.com/research/gpt-4. Accessed February 2025. Google DeepMind. Gemini AI Overview. Available at: https://www.deepmind.com/gemini. Accessed February 2025. Nguyen M, Lee J. Evaluating large language models in dermatology question-answering tasks. Skin Health Dis. 2023;8(5):15–21. doi:10.25251/skin.8.5.5. Smith A, Patel K. Comparison of AI models in medical question-based...
Read more
GALLERY