Abstract
BACKGROUND AND PURPOSE: Children with cerebral malaria have an elevated risk of mortality and neurologic morbidity. Both mortality and morbidity are associated with initially increased brain volume on MR imaging, as graded by the Brain Volume Score, a subjective ordinal rating scale created specifically for brain MRIs in children with cerebral malaria. For the Brain Volume Score to be more widely clinically useful, we aimed to determine its independent reproducibility and whether it can be applicable to lower-resolution MRIs.
MATERIALS AND METHODS: To assess the independent reproducibility of the Brain Volume Score, radiologists not associated with the initial study were trained to score MRIs from a new cohort of patients with cerebral malaria. These scores were then compared with survival and neurologic outcomes. To assess the applicability to lower-resolution MRI, we assigned Brain Volume Scores to brain MRIs degraded to simulate a very-low-field (64 mT) portable scanner and compared these with the original scores assigned to the original nondegraded MRIs.
RESULTS: Brain Volume Scores on the new cohort of patients with cerebral malaria were highly associated with outcomes (OR for mortality = 16, P < .001). Scoring of the simulated degraded images remained consistent with the Brain Volume Scores assigned to the original higher-quality (0.35 T) images (intraclass coefficients > 0.86).
CONCLUSIONS: Our findings demonstrate that the Brain Volume Score is externally valid in reproducibly predicting outcomes and can be reliably assigned to lower-resolution images.
ABBREVIATIONS:
- BVS
- Brain Volume Score
- CM
- cerebral malaria
Malaria continues to be a leading cause of childhood mortality in sub-Saharan Africa.1 A subset of children bitten by infectious mosquitoes develop cerebral malaria (CM), clinically defined as coma, Plasmodium falciparum infection, and no other obvious cause of illness. In endemic areas, CM is a driver of public health, contributing to both child mortality and poor neurodevelopmental outcomes among survivors.2,3 Although many of the steps between P falciparum infection and death from CM are clear, the lack of an animal model has resulted in an incomplete pathophysiologic understanding of the disease.4 Standard of care treatments are IV antimalarial drugs and supportive care. Currently, there are no adjunctive therapies proved to improve outcomes.
Brain MR imaging has been used to better understand CM pathogenesis.5⇓⇓⇓⇓-10 Some children with clinical CM develop increased brain volume on MRIs, identified by the presence of ventricular, sulcal, and/or cisternal effacement; blurring of the gray-white junction; and/or brain herniation. Brain volume in CM is graded using an 8-point scale, the Brain Volume Score (BVS) (Table 1).7 Scores of 1 and 2 indicate atrophy, 3 is normal brain volume, and 4–8 indicate increasing severity of ventricular, sulcal, and cisternal effacement; blurring of the gray-white junction; and (with a score of 8) brain herniation. Scores of 7 and 8, termed “highly increased brain volume,” are strongly associated with increased mortality risk.7 The original studies that created and validated this 8-point scale were performed using a Signa Ovation (GE Healthcare) fixed magnet. In these studies, interrater reliability was very good-to-excellent.7,11 Given these findings, the BVS is now intended to be used as a clinical tool to estimate prognosis and for enrichment of clinical trial samples (eg, selecting participants at higher risk of death).
MRI criteria for assignment of brain volume scores
The BVS was formulated and evaluated using a single 0.35 T MR imaging scanner.7 Because fixed magnets of this strength (or stronger) are not widely available in malaria endemic regions, there is a need to test the validity of the BVS and its association with outcomes on lower-resolution MRIs. Very-low-field (64 mT) portable MR imaging scanners have recently become available and may provide an opportunity to expand the availability and usefulness of neuroimaging in resource-limited settings.12,13 Portable scanners are less expensive than fixed magnets, but at a compromise of resolution and signal-to-noise.
We recently showed that BVS values assigned by different radiologists were similar to those assigned by an objective statistical model.11 Combined with human interrater reliability results, this finding suggests that the BVS is internally valid. To our knowledge, external validation of the association of BVS with outcomes has not been previously assessed.
Here, to evaluate the external validity of the BVS, we aimed to determine the association between outcomes and the BVSs obtained in a new, independent cohort of children with CM. Additionally, to evaluate whether BVS assignment is valid using lower-resolution images, we compared the scores assigned to brain MRIs degraded to simulate a very-low-field MR imaging with the scores previously assigned to the original higher-resolution MRIs.
MATERIALS AND METHODS
We performed a retrospective analysis of MRIs collected from children 3 months to 12 years of age with a clinical diagnosis of CM (Blantyre coma scale score ≤2, peripheral parasitemia with P. falciparum of any density, and no other identifiable cause of coma). Enrollment in the parent studies occurred on the Pediatric Research Ward, a hospital unit specialized in the evaluation and care of children with CM, located at Queen Elizabeth Central Hospital in Blantyre, Malawi. Participants received standard of care therapies for pediatric CM, including IV artesunate (according to Malawi national guidelines) and close supportive care.
Seizures, both convulsive and nonconvulsive, were treated with IV antiseizure medications. Medical complications were treated as they arose. At the time of hospital discharge, survivors were examined by a physician experienced in the care of children with CM for the presence of neurologic sequelae including motor, special sensory (blindness, deafness), or developmental abnormalities. If the parent thought that the child had returned to his or her prehospital baseline and the neurologic examination findings were normal, the survivor was classified as alive and normal. If findings of either the parental report or the examination were abnormal, the child was classified as alive with neurologic sequelae.
A parent or guardian accompanying the patient provided written informed consent at study enrollment. Ethics approval was obtained from the University of Malawi College of Medicine and the Michigan State University institutional review boards.
Brain MRI Protocol
Children with CM who had baseline brain MRIs and recorded clinical outcomes suitable for testing the external validity of the BVS were included in this study. Participants underwent MR imaging as soon as possible after clinical stabilization, typically within 4 hours of hospital presentation. Images were obtained on a Signa Ovation 0.35 T MR imaging scanner. Imaging sequences in children admitted before and including 2014 have been previously described.7 In 2014 and after, the brain MRI protocol included a sagittal 1.5-mm3-resolution T1 spin-echo sequence, an axial T2 spin-echo sequence, a coronal T2 spin-echo sequence, an axial T1 sequence, and an axial DWI sequence. If scanning protocols needed to be shortened due to patient factors (clinical instability or motion), the axial and coronal T2 sequences and sagittal T1 sequence were preferentially acquired. Contrast was not administered.
BVS Education and Methodology of Radiologists
Three radiologists participated in this study, all practicing, medical school–affiliated attendings now with at least 6 years of experience (M.S.G., neuroradiology fellowship completed 2014; L.V., pediatric neuroradiology fellowship completed 2017; and K.C., pediatric radiology fellowship completed 2015). All 3 radiologists underwent initial training with one of the radiologists who developed the BVS. Training included a virtual didactic session and review of 50 MRIs including the full range of possible BVSs (the “training set”). The radiologists tested their BVS assignment abilities by next independently scoring 47 test case MRIs (the “test set”) blinded to the original (reference standard) BVS assigned by one of the radiologists who developed the BVS.
After the 3 radiologists completed this training, 2 of them (M.S.G. and L.V.) re-assigned consensus BVSs to the same 47 test set MRIs but blinded to the original scores, approximately 2 years later. This was to confirm that their consensus scores remained closely calibrated with the reference standard BVS scores. Consensus was achieved after concurrent review of images using teleconferencing software (Zoom; Zoom Video Communications) and arriving at a single consensus score. During this process, several cases were identified in which differentiating between a BVS of 6 and 7 was challenging. Accordingly, a score of 6.5 was allowed in the consensus scores for these cases (Table 1).
Evaluating the Reproducibility and External Validity of the Association between BVS and Outcomes in Pediatric CM
After completion of training, the same 2 radiologists (M.S.G. and L.V.) also independently assigned consensus BVS ratings to 94 brain MRIs obtained from a new cohort of children with CM. The radiologists were blinded to patient outcomes when assigning the BVS. To determine the potential benefit of modification of the BVS criteria by adding a score of 6.5 to the current whole-number-based system, we evaluated whether the outcomes of children whose BVS was 6.5 were more like outcomes seen in children whose MRIs were scored as 6, 7, or neither.
Testing the External Validity of BVS Assignment in Lower-Resolution MRIs
To evaluate whether BVS can be validly assigned using lower-resolution MRIs, we obtained degraded versions of 37 of the training set and 47 of the test set MRIs (all from the 0.35 T MR imaging scanner and used previously for BVS assignment training) based on a proprietary algorithm from Hyperfine Research (Guilford, Connecticut). This step resulted in a set of brain images with lower resolution and greater noise, aiming to simulate scans obtainable with the 64 mT Swoop portable MRI (Hyperfine).
To first calibrate BVS assignment to the now-degraded images, 3 radiologists (M.S.G., L.V., and K.C.) reviewed the 37 degraded training set MRIs along with their corresponding nondegraded images. Once they finished this calibration step, each radiologist then independently assigned a BVS to each of the 47 degraded test set MRIs, while blinded to their corresponding nondegraded images. This step allowed us to test the reliability of the BVS assignment of the degraded MRIs by determining the concordance between the BVS assigned to the degraded test set MRIs and the original BVS assigned to their corresponding nondegraded images, which had been obtained using the 0.35 T scanner.
Statistical Analysis
Statistical analyses were performed in Excel 2016 (Microsoft) and R Version 4.0.2 or higher (https://www.r-project.org/). The Fisher exact test was used to calculate ORs and 95% CIs. To assess the concordance of BVS assignment between the original resolution and degraded MRIs, we used the intraclass correlation coefficient function from the irr package for R (https://cran.r-project.org/web/packages/irr/index.html). For intraclass correlation coefficients, lower-bound 95% CI values between 0.5 and 0.75 were considered moderate; 0.75 and 0.90, good; and >0.90, excellent estimates of reliability.14
RESULTS
The mean age of the 94 children with CM whose MRIs were evaluated for external validity in this study was 49.5 months (range, 3–148 months), and 46% were male.
Calibration of the 2 Independent Radiologists’ Consensus BVS Scoring
Two-radiologist consensus BVS scoring of the nondegraded 47 test set brain MRIs had a good intraclass coefficient (0.925; 95% CI, 0.870–0.958) compared with reference standard scores assigned by the original radiologist who helped develop the BVS. The mean difference between the consensus scores and the original reference standard scores was very small (0.043; paired Student t test, P = .69), confirming that the independent consensus BVS scoring was well-calibrated against the reference standard scoring.
Reproducibility and External Validity of the Association of BVS and Outcomes in Pediatric CM
Two-radiologist consensus BVSs for the 94 newly obtained brain MRIs ranged from 2 to 8. Six was the most frequent score. All children whose MRIs had scores of ≤4 were alive and neurologically normal at hospital discharge (n = 12). The 4 children whose MRIs had a score of 8 either died (n = 3) or were alive with neurologic sequelae (n = 1). Outcomes of those with BVS values of 5–7 were intermediate. In those children with a score of 5 (n = 20), one (5%) died and 3 of the 19 survivors (16%) were alive with neurologic sequelae. Of the 27 children whose MRIs had a BVS of 6, five (19%) died; the remainder were alive and neurologically normal. Of the 16 children with images assigned a BVS of 7, nine (56%) died, one was alive with neurologic sequelae (6%), and 6 were alive and neurologically normal. Among 15 cases given an intermediate score of 6.5, none died, one was alive with neurologic sequelae (7%), and the rest (93%) were alive and neurologically normal. The outcomes of children with a BVS of 6.5 were more similar to those in children whose BVS was 6 and less similar to children with scores of 7. Outcomes according to 3 categories (≤5, 6 or 6.5, and 7 or 8) are shown in Fig 1.
Outcomes of children by the initial consensus BVS. Proportional outcomes of consensus BVS are demonstrated when trichotomized into 3 categories, ≤5, 6 or 6.5, and 7 or 8.
In the original outcome study, children with BVSs of 7 or 8, described as having “highly increased brain volume,” had a high mortality risk.7 To evaluate the reproducibility and external validity of this finding, we compared the outcomes of children with scores of 7 or 8 (n = 20) with the outcomes of children with a BVS of ≤6.5 (n = 74). Children whose scans had a BVS of 7 or 8 had a 60% mortality risk and, if they survived, a 70% chance of a poor neurologic outcome. Conversely, those with a BVS of ≤6.5 had an 8% mortality risk and an 86% chance of survival with a good neurologic outcome. Compared with the outcomes of children with scores of ≤6.5, the odds of death in children whose BVS was 7 or 8 was 16.2 (95% CI, 4.3–69.7). These odds are similar to those reported in previous studies (Table 2). If we defined “highly increased brain volume” as a BVS of 6–8, the odds of death for children with higher scores was 11.5 (95% CI, 1.6–503.7), compared with children with a BVS of ≤5.
ORs for mortality at a BVS threshold of 7 or 6
Consistency of BVS Assignment on Lower-Resolution MRIs
Despite the images being of lower resolution, MRIs degraded to simulate the output of lower-field scanners showed similar characteristics compared with the original images obtained at 0.35 T (Fig 2). The CSF spaces, including the sulci, cisterns, and ventricles, could be differentiated from brain parenchyma, allowing assessment of whether they were effaced. In contrast, blurring of the gray-white junction was less visible using the degraded images compared with those that were not degraded.
Sample T2w original and simulated MRIs from children with CM. The original brain MRIs were obtained on a Signa Ovation 0.35 T magnet (upper row) and corresponding axial images were obtained after image degradation to simulate the resolution of a very-low-field scanner (lower row). A, A BVS of 8 was assigned to the original scan as well as to the degraded images by all 3 radiologists. B, A BVS of 6 was assigned to the original scan, and 6 or 7, to the degraded images. C, A BVS of 4 was assigned to the original scan, and 5 or 3, to the degraded images.
For the degraded 47 test set MRIs, the intraclass coefficients for BVS scores assigned by each of the 3 radiologists compared with the original BVS scores for the nondegraded MRIs ranged from 0.867 to 0.901 (Table 3). The mean bias for all 3 was <0.5, and the mean absolute deviation was approximately 0.6. If we used the median BVS from the 3 radiologists assigned on the lower-resolution images and compared it with the BVS score assigned using higher-resolution images, this step modestly improved the intraclass coefficient to 0.928, while slightly reducing the bias (0.128) and mean absolute deviation (0.468).
BVS assigned to degraded MRIs compared with original criterion standard BVS assigned to nondegraded scans
DISCUSSION
Brain MR imaging is a prognostic biomarker in pediatric CM and can be used to identify children who may most benefit from aggressive or targeted interventions. The whole-number system of BVS assignment is prognostically valid. In our study, noninteger intermediate scoring (ie, BVS = 6.5) yielded no additional prognostic information. Using 2-radiologist consensus scoring, our study confirms the reproducibility and external validity of the association of the integer-based BVS and outcomes in CM. Moreover, assignment of BVS remains consistent when applied to degraded images with lower resolution.
A fundamental challenge in applying any scale to describe the severity of a pathologic process is in determining the appropriate resolution of that scale. With respect to the BVS, if gradations finer than the currently used ordinal integer scale are used, one radiologist could assign a score of 6.4 to a case, while a second could assign a score of 6.6. Although the assigned grades differ minimally, if one is forced to round to the nearest whole number, the original scores would more widely differ. This difference becomes important at thresholds (eg, between BVS of 6 and 7) at which mortality risk changes. Conversely, using a more finely graded scale may not improve the reliability or accuracy. Here, we assessed the validity of a single in-between score, 6.5, and found that outcomes were more like those in children with a score of 6 than with a score of 7. On this basis, we suggest that the original 8-integer-based ordinal scale for the assignment of BVS currently remains appropriate, though future refinements should continue to be investigated.
In our study, children with CM whose MR imaging scans were scored independently as a BVS of 7 or 8 had a mortality risk of 60%. Conversely, only 1 of 32 children with a score of ≤5 died. For cases assigned a score of 6, mortality was intermediate. Our results suggest that the BVS may be more useful if trichotomized into “no significant increase in brain volume” (BVS of ≤5), “moderate increase in brain volume” (BVS of 6), and “highly increased brain volume” (BVS of 7 or 8). Children with the lowest range of scores have low mortality, those with a score of 6 have intermediate mortality, and scores of 7 or 8 are associated with a high mortality risk. Of note, among survivors, the relationship between morbidity and BVS scores is less clear. Thus, future studies to confirm and extend these findings are warranted.
Images acquired for this study were from a single 0.35 T MR imaging scanner with a minimally varying imaging protocol. Portable lower-field scanners are increasingly being tested and used in a variety of clinical settings, including evaluating brain volume in children15 with pathogenic processes that result in brain injury and edema16 and when assessing brain herniation.17 To evaluate whether this more easily scalable imaging technique might work in the assignment of BVSs to children with CM, we determined score assignment concordance when images from the 0.35 T MR imaging scanner were degraded to simulate a 64 mT MR imaging scanner. Despite image degradation, BVS scoring showed good-to-excellent reliability between images obtained on a 0.35 T fixed magnet and those simulating output of a portable 64 mT scanner. Our findings should be considered preliminary because they used proprietary-algorithm-based simulated images rather than those obtained independently by the very-low-field scanner itself. Optimally, MRIs from a single patient using both a 0.35 T fixed magnet and a portable 64 mT magnet should be compared. Ethically, this comparison may be challenging due to the fragile clinical status of critically ill children with CM and the long scanning times required. Another method to validate BVSs derived from very-low-field scanners as prognostic biomarkers in pediatric CM would be to assess the association between BVS and outcomes using images only obtained from a portable 64 mT magnet. Nevertheless, our current findings suggest but cannot establish that the BVS is a valid prognostic biomarker across MR imaging scanners of varying field strengths.
Our study has several limitations. Most important, the study radiologists are all experienced in BVS assignment in children with CM. The generalizability of the findings of our study to radiologists who have not undergone a rigorous educational program of BVS assignment is unknown. The training and test set MRIs in this study were used repeatedly for training, calibration of BVS scores, and testing of the BVS on degraded images. Although months or years separated each of these tasks, it is possible that memory of the previously assigned BVS affected a radiologist’s later BVS scoring. Our reuse of images through time was required due to the numeric rarity of MR imaging scans obtained from children with CM. The validity of the BVS as a prognostic biomarker was derived using images obtained at hospital admission. It is possible that changes in BVS through time may be a better prognostic biomarker or provide additional insights into CM pathogenesis. We used the neurologic status at hospital discharge to indicate the presence of neurologic sequelae. The duration of posthospital follow-up for children included in this study varied across the clinical protocols in which they were enrolled. Because these posthospital follow-up times were not standardized for all participants included here, only the hospital discharge status was used in our analyses. However, because neurologic sequelae in CM survivors may appear or resolve through time, it is possible that the estimates of the strength of the association between BVS and neurologic sequelae may change if a standard posthospital neurologic status was used in the analyses.
CONCLUSIONS
In pediatric CM, independent MRI-based assessment of the BVS reproducibly predicts outcomes and can be reliably performed with lower-resolution images.
Acknowledgments
We are grateful to all families whose children took part in this study. We would also like to thank Dr Sam Kampondeni for assisting in the initial training of the study radiologists on assigning brain volume scores.
Footnotes
This work was supported by National Institutes of Health grants R01 AI034969 and U01 AI126610.
Disclosure forms provided by the authors are available with the full text and PDF of this article at www.ajnr.org.
References
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- Received June 28, 2023.
- Accepted after revision November 6, 2023.
- © 2024 by American Journal of Neuroradiology