On the test set, the overall accuracy of the CNN (ResNet-18) ensemble model among all sequence types was 97.9% (Figure 1), ranging from 84.2% for susceptibility-weighted images (95%CI: 81.8%, 86.6%) to 99.8% for T2-weighted images (95%CI: 99.7%, 99.9%). Confusion matrix is illustrated in Figure 2. The ResNet-18 model achieved significantly better accuracy compared with ResNet-50 despite its simpler architecture (97.9% versus 97.1%, p=<0.001).

Fig 2: Classification results on the test set based on the ResNet-18 architecture. Confusion matrices indicate the true and predicted values for each MRI sequence class as percentages. ADC = apparent diffusion coefficient, CT1 = postcontrast T1 weighted, FLAIR = fluid-attenuated inversion recovery, High-B-DWI = high-b-value diffusion-weighted imaging, Low-B-DWI = low-b-value diffusion-weighted imaging, SWI = susceptibility-weighted imaging, T2* and DSC-related = T2*-weighted and dynamic susceptibility contrast related.

To assess the applicability of our model beyond tumor cohorts and its ability to generalize to other diseases or healthy subjects, we conducted two additional analyses. First, we generated Grad-CAM visualizations to gain insights into the model’s focus on tumors versus the healthy brain itself. The best model’s attention patterns were visualized for the last layer (representative examples are shown in Figure 3).

Fig 3: Gradient-weighted class activation mapping visualization of the last layer of the best model. Correctly predicted test images with and without visible tumor or lesion in the brain were selected for each MRI sequence type. ADC = apparent diffusion coefficient, CT1 = postcontrast T1 weighted, FLAIR = fluid-attenuated inversion recovery, Grad-CAM = gradient-weighted class activation mapping, high-B-DWI = high-b-value diffusion-weighted imaging, low-B-DWI = low-b-value diffusion-weighted imaging, T2*/DSC-related = T2*-weighted and dynamic susceptibility contrast related.

Second, we quantified the area of the tumor within the midsection of the brain that was used as input for training or test data. The tumor segmentation was done using HD-GLIO segmentation tool [11-12], which requires T1, CT1, T2 and FLAIR volumes and 5378/8544 exams (63%) fulfilled these criteria. Within the 5378 MRI exams a total of 23849 sequences were available for quantifying the area of the tumor within the midslice of the brain. We compared the prediction accuracy of the ensemble model on the images with presence of tumor (3690/23849 exams [15.5%]) and without presence of tumor (20159/23849 exams [84.5%]) in the brain mask for each sequence type. To examine the hypothesis of whether our model performs effectively on images from healthy subjects (considering that our CNN model was trained solely on glioblastoma cohorts), we conducted χ2 tests followed by false discovery rate (FDR) correction between subgroups of tumor presence vs. absence for each MRI sequence type. The accuracy of the ResNet-18 ensemble model was not affected by the presence vs. absence of tumor on the two-dimensional-midsection images for any sequence type. Results are illustrated in Figure 4.

Fig 4: (A) Illustration of the extraction of tumor area from the two-dimensional midsection image. (B) Chart shows the effect of tumor presence in the input image (two-dimensional midsection) on prediction accuracy of the ensemble model. For each MRI sequence type, prediction accuracy for images with or without tumor presence is illustrated (with tumor category: whole tumor area/brain mask ratio >0%; without tumor category: whole tumor area/brain mask ratio = 0%). ADC = apparent diffusion coefficient, CE = contrast enhanced, CNN = convolutional neural network, CT1 = postcontrast T1 weighted, ED = edema, FLAIR = fluid attenuated inversion recovery, High-B-DWI = high-b-value diffusion-weighted imaging, Low-B-DWI = ow-b-value diffusion-weighted imaging, SWI = susceptibility-weighted imaging, T2* and DSC-related = T2*-weighted and dynamic susceptibility contrast related, 3D = three-dimensional, 2D = two-dimensional.