Back to the list
Congress: ECR25
Poster Number: C-18397
Type: Poster: EPOS Radiologist (scientific)
Authorblock: A. Fink, M. Russe, A. Rau, J. Weiß, F. Bamberg, T. Stein; Freiburg/DE
Disclosures:
Anna Fink: Nothing to disclose
Maximilian Russe: Nothing to disclose
Alexander Rau: Nothing to disclose
Jakob Weiß: Nothing to disclose
Fabian Bamberg: Nothing to disclose
Thomas Stein: Nothing to disclose
Keywords: Artificial Intelligence, CT, Experimental investigations, Trauma
Purpose Purpose: To evaluate the ability of an interactive chatbot to identify optimal CT protocols and contrast phases based on unstructured clinical queries and context-specific knowledge.Background: Selecting personalized CT protocols and contrast phases is critical for accurate diagnoses and standardized, radiation-reduced patient care. Prior studies have demonstrated a chatbots' ability to recommend imaging protocols using American College of Radiology guidelines [1]. However, real-world clinical scenarios involve unstructured queries requiring advanced reasoning and context comprehension. This study explores the application of large...
Read more Methods and materials Model: LLM: OpenAI's GPT-4 [2] was provided with specialized knowledge using Retrieval-Augmented Generation (RAG), an approach that combines pre-trained LLMs with a real-time retrieval mechanism to enhance reasoning capabilities in unstructured settings. Embedding Process: Our local standard operating procedure (SOP) on CT protocols was indexed using OpenAI’s embedding model (text-embedding-ada-002-v2) [3]. Each sentence, alongside five preceding and five following sentences, was converted into a numerical embedding. Approach: Cases: We created 100 synthetic cases of unstructured clinical requests, each containing a short patient history,...
Read more Results Performance: Accuracy: Correct recommendations in 77% of cases Minor Errors: Occurred in 12% of cases Significant Errors: Found in 11%, particularly in complex cases involving multiple contrast phases and scan regions Hallucination-Free Responses: No out-of-context hallucinations were observed. Efficiency: Average Decision Time: 14.39 seconds, suggesting potential time and cost savings in clinical workflows
Read more Conclusion Limitations: Challenges in Complexity: The chatbot struggled with intricate cases requiring nuanced reasoning. Future refinements are needed to improve consistency in handling complex clinical scenarios. Outlook: The two-step reasoning approach successfully standardized CT protocol selection, showcasing potential as a supportive tool in clinical practice. Impact: Streamlines patient management, potentially enhances patient safety by minimizing unnecessary imaging, contrast use, and radiation exposure.
Read more References References: [1] Rau A, Rau S, Zoeller D, Fink A, Tran H, Wilpert C, Nattenmueller J, Neubauer J, Bamberg F, Reisert M, Russe MF. A Context-based Chatbot Surpasses Trained Radiologists and Generic ChatGPT in Following the ACR Appropriateness Guidelines. Radiology. 2023 Jul;308(1):e230970. doi: 10.1148/radiol.230970. PMID: 37489981.[2] OpenAI. GPT-4 Research. https://openai.com/gpt-4. Accessed October 22, 2024.[3] OpenAI. Text embedding Ada 002 V2. https://openai.com/blog/new-and-improved-embedding-model. Accessed October 30, 2024.[4] Liu J. LlamaIndex. 2022. doi: 10.5281/zenodo.1234.
Read more
GALLERY