Data

The 2024 BraTS-PTG dataset consists of retrospective, multi-institutional cohorts of patients diagnosed with diffuse gliomas having already undergone treatment, which may include surgery, radiation, and/or systemic therapy. The patients have been clinically scanned with multiparametric MRI (mpMRI) acquisition protocols, including the following sequences:

1. pre-contrast T1-weighted (T1)
2. contrast-enhanced T1-weighted (T1-Gd)
3. T2-weighted (T2)
4. T2-weighted fluid-attenuated inversion recovery (FLAIR)

Data Preprocessing 

The preprocessing pipeline (Figure 1) was similar to the one evaluated and followed by the BraTS 2017-2023 challenges [6].

Fig 1: Data processing and annotation workflow for creating the 2024 BraTS post-treatment glioma challenge dataset.

The T1, T1-Gd, T2, and FLAIR sequences were extracted and named according to the standard BraTS naming convention. The dcm2niix software was applied to the four sequences to convert the raw scans from their original Digital Imaging and Communications in Medicine (DICOM) file format to the Neuroimaging Informatics Technology Initiative (NIfTI) file format [7]. Following the conversion to NIfTI files, we performed brain extraction to remove any apparent non-brain tissue using HD-BET (https://github.com/CCI-Bonn/HD-BET) [8]. The brain-extracted sequences were registered to the Linear Symmetrical MNI Atlas using affine registration via the CapTK/Greedy software [9]. Several institutions that contributed to the dataset processed their data in a similar in-house pipeline [10–12] or in the FeTS 2.0 platform (https://fets-ai.github.io/FL-PoST/). 

Prior to manual correction, automated segmentation models were used to generate pre-segmentations. Some sites used their in-house pipelines [10,12], while others applied a standardized pipeline utilizing five pre-segmentation approaches based on nnU-Net [13] and SegResNet [14] models trained on post-treatment glioma data. The five segmentations produced by these networks were combined using the Simultaneous Truth and Performance Level Estimation (STAPLE) fusion algorithm [15]. In addition, digital subtraction images were created between the T1-Gd and T1 sequences to facilitate the annotation process for radiologists in cases of more subtle enhancement or confounding areas of T1 intrinsic hyperintensity. 

Tumor Annotation Protocol

The data considered in the BraTS-PTG was similar to the paradigm of the BraTS 2021-2023 challenge data [6], though with modifications specific to the post-treatment setting. The annotation (Figure 2) followed a pre-defined clinically approved annotation protocol (defined by expert neuroradiologists and radiation oncologists). This protocol was provided to all clinical annotators, describing in detail instructions on what the segmentations of each tumor sub-region should include, with numerous examples of more challenging cases.

Fig 2: Tumor sub-regions considered in the 2024 BraTS post-treatment glioma challenge. The enhancing tissue (blue) visible on T1-Gd, the non-enhancing tumor core (red) visible on T1-Gd, the surrounding non-enhancing FLAIR hyperintensity (green) visible on FLAIR, and the resection cavity (yellow) visible on T2. The combined segmentations generating the final tumor sub-region labels visible on mpMRI, as provided to the challenge participants.

Summary of specific instructions:

1. Enhancing tissue (ET): Pathological hyperintense signal on T1-Gd. Areas of thick or nodular enhancement were included, though typical treatment-related thin linear enhancement along and within resection cavities and along the dura were not included.  
2. Non-enhancing tumor core (NETC): Hypointense regions on both T1 and T1-Gd images (denoting necrosis/cysts) and dark regions on T1-Gd that appear brighter on T1 and not clearly represented by a prior resection cavity.
3. Surrounding non-enhancing FLAIR hyperintensity (SNFH): Includes edema and infiltrating tumor as well as other tumor-related FLAIR signal abnormalities, including treatment-related changes and gliosis. Symmetric or patchy white matter hyperintensities clearly related to chronic microvascular ischemic disease or periventricular capping are not included.
4. Resection cavity (RC): Consists of both recent and chronic resection cavities. Chronic cavities appear isointense to cerebrospinal fluid on T1 and T2/FLAIR, while recent cavities may show air, blood, or proteinaceous material with variable signals.

Thirty-two annotators worked with the annotation (Figure 3).

Fig 3: Map showing the distribution of the 32 annotators who worked with the segmentations to create the final reference standard annotations.

The annotators were provided with the four mpMRI sequences as well as a T1-Gd - T1 subtraction image and given the flexibility to use their tool of preference for making the segmentations, following a hybrid approach where the pre-segmentations were refined manually. After annotators refined the tumor segmentations, board-certified neuroradiologists from the organizing committee reviewed them. If the segmentations were unsatisfactory, they were sent back for further refinement. This iterative process continued until segmentations met the required quality for public release as final reference standard labels. Based on observations, we have identified some common errors in automated segmentations (Figure 4).

Fig 4: Common errors in automated segmentations in the 2024 BraTS post-treatment glioma challenge. The top row shows typical segmentation errors, and the bottom row shows manually corrected labels. Additional detailed examples are available in the annotation protocol.

Performance Evaluation 

The challenge was hosted on the Synapse Platform (Sage Bionetworks). Following the paradigm of algorithmic evaluation in machine learning, the data was divided into training, validation, and testing datasets. Challenge participants received reference standard labels exclusively for the training dataset, while the validation dataset was provided without any associated labels, and the testing dataset remained completely hidden. The evaluation metrics used in this challenge were the same as from BraTS 2023 challenges (https://github.com/rachitsaluja/BraTS-2023-Metrics):

1. Lesion-wise Dice similarity coefficient (L-DSC) measures voxelwise segmentation overlap between predicted and reference standard segmentations, ignoring true negative voxels.  
2. Lesion-wise 95% Hausdorff distance (L-HD95) measures the distance between the center of the predicted and reference standard segmentations.

The L-DSC and L-HD95 metrics were designed to assess model performance at the level of individual lesions rather than across the entire image. This prevents our evaluation from favoring models that only detect larger lesions, a limitation often seen with standard DSC, and enables a lesion-by-lesion evaluation to better assess segmentation of multifocal and multicentric disease.

In terms of the assessed and evaluated tumor sub-regions:

1. ET - regions of active tumor as well as nodular areas of enhancement.
2. NETC - necrosis and cysts within the tumor.
3. SNFH - edema, infiltrating tumor, and post-treatment changes.
4. RC - both recent and chronic resection cavities.
5. Tumor core (ET plus NETC) describes what is typically resected during a surgical procedure.
6. Whole tumor (ET plus NETC plus SNFH) defines the whole extent of the tumor, including the tumor core, infiltrating tumor, peritumoral edema, and treatment-related changes.

For ranking teams based on multiple metrics, we calculated the summation of their ranks across the average of the metrics described above. This combined score decided the overall ranking for each team.