Interrater Agreement of BT-RADS for Evaluation of Follow-up MRI in Patients with Treated Primary Brain Tumor

BACKGROUND AND PURPOSE: The Brain Tumor Reporting and Data System (BT-RADS) is a structured radiology reporting algorithm that was introduced to provide uniformity in posttreatment primary brain tumor follow-up and reporting, but its interrater reliability (IRR) assessment has not been widely studied. Our goal is to evaluate the IRR among neuroradiologists and radiology residents in the use of BT-RADS.

MATERIALS AND METHODS: This retrospective study reviewed 103 consecutive MR studies in 98 adult patients previously diagnosed with and treated for primary brain tumor (January 2019 to February 2019). Six readers with varied experience (4 neuroradiologists and 2 radiology residents) independently evaluated each case and assigned a BT-RADS score. Readers were blinded to the original score reports and the reports from other readers. Cases in which at least 1 neuroradiologist scored differently were subjected to consensus scoring. After the study, a post hoc reference score was also assigned by 2 readers by using future imaging and clinical information previously unavailable to readers. The interrater reliabilities were assessed by using the Gwet AC2 index with ordinal weights and percent agreement.

RESULTS: Of the 98 patients evaluated (median age, 53 years; interquartile range, 41–66 years), 53% were men. The most common tumor type was astrocytoma (77%) of which 56% were grade 4 glioblastoma. Gwet index for interrater reliability among all 6 readers was 0.83 (95% CI: 0.78–0.87). The Gwet index for the neuroradiologists’ group (0.84 [95% CI: 0.79–0.89]) was not statistically different from that for the residents’ group (0.79 [95% CI: 0.72–0.86]) (χ² = 0.85; P = .36). All 4 neuroradiologists agreed on the same BT-RADS score in 57 of the 103 studies, 3 neuroradiologists agreed in 21 of the 103 studies, and 2 neuroradiologists agreed in 21 of the 103 studies. Percent agreement between neuroradiologist blinded scores and post hoc reference scores ranged from 41%–52%.

CONCLUSIONS: A very good interrater agreement was found when tumor reports were interpreted by independent blinded readers by using BT-RADS criteria. Further study is needed to determine if this high overall agreement can translate into greater consistency in clinical care.