Artificial intelligence (AI) tools are evolving rapidly, with large language models (LLMs) now demonstrating the ability to process both text and image-based data. In radiology, AI has primarily focused on task-specific solutions for image analysis and diagnostic support [1]. However, recent developments in multimodal models have introduced LLMs capable of integrating text and image inputs, opening new possibilities for clinical applications.Unlike conventional AI algorithms, which are designed for specific imaging tasks mainly interpretative and workflow-enhancing deep learning algorithms, LLMs offer broader utility by generating detailed responses to image-related questions and assisting in decision-making processes [2]. This versatility could make them a valuable tool in radiology, especially with the emergence of commercially widely available AI models like GPT-4V and Google’s Gemini that can analyze images and provide context-aware explanations [3,4].

Multiple studies have shown that these models perform well on text-based medical exams, including in fields such as dermatology, neurology, and ophthalmology [5–7]. However, their ability to interpret radiological images and match the expertise of human radiologists remains under investigation. This presents an opportunity to evaluate their potential role in radiology beyond traditional text-based applications.

In our study, we compared the performance of commercially widely available large language AI models (LLMs) with that of neuroradiologists and radiology residents in answering image-based multiple-choice questions (MCQs) from a vastly consulted free radiological educational online resource like Radiopaedia [8]. The evaluation included response accuracy, overall success rates, and the number of correct answers across various imaging categories.