Abstract
BACKGROUND AND PURPOSE: The nucleus basalis of Meynert is a key subcortical structure that is important in arousal and cognition and has been explored as a deep brain stimulation target but is difficult to study due to its small size, variability among patients, and lack of contrast on 3T MR imaging. Thus, our goal was to establish and evaluate a deep learning network for automatic, accurate, and patient-specific segmentations with 3T MR imaging.
MATERIALS AND METHODS: Patient-specific segmentations can be produced manually; however, the nucleus basalis of Meynert is difficult to accurately segment on 3T MR imaging, with 7T being preferred. Thus, paired 3T and 7T MR imaging data sets of 21 healthy subjects were obtained. A test data set of 6 subjects was completely withheld. The nucleus was expertly segmented on 7T, providing accurate labels for the paired 3T MR imaging. An external data set of 14 patients with temporal lobe epilepsy was used to test the model on brains with neurologic disorders. A 3D-Unet convolutional neural network was constructed, and a 5-fold cross-validation was performed.
RESULTS: The novel segmentation model demonstrated significantly improved Dice coefficients over the standard probabilistic atlas for both healthy subjects (mean, 0.68 [SD, 0.10] versus 0.45 [SD, 0.11], P = .002, t test) and patients (0.64 [SD, 0.10] versus 0.37 [SD, 0.22], P < .001). Additionally, the model demonstrated significantly decreased centroid distance in patients (1.18 [SD, 0.43] mm, 3.09 [SD, 2.56] mm, P = .007).
CONCLUSIONS: We developed the first model, to our knowledge, for automatic and accurate patient-specific segmentation of the nucleus basalis of Meynert. This model may enable further study into the nucleus, impacting new treatments such as deep brain stimulation.
ABBREVIATIONS:
- DBS
- deep brain stimulation
- DnSeg
- deep nuclei segmentation network
- NBM
- nucleus basalis of Meynert
- TLE
- temporal lobe epilepsy
The nucleus basalis of Meynert (NBM) is a basal forebrain nucleus and is one of the major sources of cholinergic signal in the brain.1,2 It has been implicated as abnormal in several disorders with cognitive decline such as Parkinson disease dementia, Alzheimer disease, and temporal lobe epilepsy (TLE).3,4 These abnormal changes include prediction of cognitive impairment in Parkinson disease, neuronal loss and atrophy in Alzheimer disease, and abnormal functional connectivity in TLE.3⇓⇓⇓-7 Furthermore, there has recently been increased interest in studying the functional and structural network abnormalities involving NBM connections in these and other disease states.3,6,7 Finally, several studies have started investigating neurostimulation of the NBM to improve cognitive outcomes in Alzheimer disease using deep brain stimulation (DBS).8,9
While the NBM is thought to be important in several disease states, study into it has been limited because it exhibits patient-specific changes and no accurate patient-specific segmentation exists.10 Patient-specific changes may be accounted for with manual segmentation, but the NBM is difficult to visualize on commonly available 3T T1-weighted MR imaging. More accurate manual segmentations can be performed on high-resolution 7T MR imaging; however, these high-resolution scans are rare (Online Supplemental Data). To overcome this limitation, previous groups have developed a probabilistic atlas of the NBM, derived from postmortem histologic staining of 10 healthy subject’s brains.11 The segmented histologic slices were then aligned to the Montreal Neurological Institute 152 atlas space to provide a probabilistic atlas of the NBM. The probabilistic atlas has gained popularity and is the most used atlas of the NBM.12 While the probabilistic atlas provides an accurate segmentation for most cases, it does not capture patient-specific anatomic variability (Fig 1). Considering that the anatomy of the NBM varies among patients and changes throughout the disease states of interest, this limitation substantially impacts nearly all studies of the NBM.3⇓⇓⇓-7,13⇓⇓-16 Furthermore, recent studies have demonstrated that the probabilistic atlas is unable to capture the patient-specific differences, and atlas innovation is needed to address this limitation.17
Comparison of the probabilistic and manual labels on a similar coronal section of the Montreal Neurological Institute 152 atlas space brain and a subject’s 7T brain. The left NBM is shown in blue, and the right NBM is shown in red.
Deep learning methods can learn subtle features for medical imaging segmentation that are beyond human perception.18 Thus, to expand study of the NBM, we propose a novel deep learning—based method for patient-specific NBM segmentation using only 3T MR imaging.
MATERIALS AND METHODS
Data Set
We trained the network using only 3T MR imaging and both 7T and 3T MR imaging scans of the same healthy subjects. Data from a total of 21 healthy subjects were available from 2 separate institutions (Vanderbilt University and the University of Amsterdam).19⇓-21 A total of 6 healthy subjects from this data set were completely withheld as a test data set.
An additional external data set of 14 paired 3T and 7T MR images of patients with TLE was obtained. Half of the patients had lesional changes on MR imaging, and the other half had nonlesional changes. This data set was withheld from the training and validation process and was used only for testing of the deep learning network on brains with neurologic disorders. The demographics of both data sets can be seen in the Table.
Data set demographicsa
Manual Segmentation
Although the anatomic borders of the NBM cannot be easily discerned on 3T MR imaging, 7T MR imaging enables enhanced contrast in the NBM. According to the landmarks described in the literature, the anatomic borders of the NBM were identified for manual segmentation.1,22 The left and right NBMs were segmented for each subject and patient using our in-house CRAnial Vault Explorer (CRAVE) software (https://www.sciencedirect.com/science/article/abs/pii/S1361841510001015).23 The segmentations of the NBM were completed by 2 authors and were verified by 2 neurological surgeons.
Data Preprocessing
The 7T and 3T paired images were converted to Right Anterior Superior (RAS) orientation, were skull-stripped, and were rigidly registered to the same patient’s 7T scan with SPM8 (https://www.fil.ion.ucl.ac.uk/spm/software/spm8/).24,25 The registration accuracy was then individually verified for every patient (Fig 2).
Flow chart describing how the 7T NBM segmentation is used for training the network to segment the NBM on 3T MR imaging.
Data Augmentation
To enhance model generalization and promote learning of anatomic variability, we augmented the healthy subject data set from 21 to 210 samples. Augmentations were performed with torchIO (https://torchio.readthedocs.io/) and included random rigid affine transformations, random elastic transformations, and random bias field additions, examples of which can be seen in the Online Supplemental Data.26
Volume Reduction
Considering the large amount of video RAM required to train a deep learning network using the whole brain, the size of the images was reduced for ease of training. Given that the mamillary bodies are the posterior anatomic borders of the NBM, we used the patient-specific segmentation package implemented in FreeSurfer ScLimbic (surfer.nmr.mgh.harvard.edu/fswiki/ScLimbic) to segment the mamillary bodies.15 The mamillary bodies were used as the center of a 64 × 64 × 64 mm region, which includes the entire NBM in all cases. The final network performs this step automatically, allowing for an end-to-end network that requires only 3T MR imaging for accurate segmentation.
Convolutional Neural Network
The modified 3D U-Net architecture for biomedical image segmentation was used.27 A complete visualization of the architecture can be seen in the Online Supplemental Data. The network was trained using the soft Dice loss function.28
The data were divided into train, test, and validation sets. The data sets were split on the subject level. The train data set consisted of 12 subjects (120 scans), the validation data set consisted of 3 subjects (30 scans), and the test data set consisted of 6 subjects (60 scans). The test data set was completely held out from model selection and only used for the evaluation after the final model was selected.
A 5-fold cross-validation was used on the train and validation data sets. The average Dice performance from each fold on the validation data set was used as the performance measure for model selection. Hyperparameter tuning during model selection was performed.29 The final network comprised an ensemble of 5 trained and optimized models.
Evaluation Metrics
Evaluation of the deep nuclei segmentation network (DnSeg) encompassed methods to assess volumetric overlap of the NBM and localization of the NBM. The Dice coefficient, mean surface distance, and centroid distance were calculated between the predicted and ground truth segmentation, defined as the overlap of both rater segmentations.13,14,30
The performance of DnSeg was compared against that of the probabilistic atlas. The probabilistic atlas was thresholded such that all voxels with a >50% probability of being the NBM were assigned as NBM. The probabilistic atlas was registered from Montreal Neurological Institute 152 atlas space to patient-specific space using SPM8 (https://www.fil.ion.ucl.ac.uk/spm/software/spm8/). The same metrics were then computed on the probabilistic atlas segmentation of the NBM.
RESULTS
All results were computed in the patient’s 3T MR imaging space on nonaugmented data with 1-mm isotropic slices. The results were all computed on the test data sets of 6 healthy subjects, which were held out from the training and validation of the DnSeg. Results were also computed on held-out external data sets of 14 subjects with diagnosed TLE that were not used in development of the DnSeg.
Dice Coefficient
The Dice coefficient was computed between the predicted and ground truth–segmented NBM.30 The Dice coefficient is known to underestimate the performance of small, complex structures, for example, segmentation studies of other small structures such as the ventral tegmental area and the interposed nuclei report Dice coefficients of 0.52 and 0.69, respectively.13⇓⇓-16,31 The Dice coefficient of the NBM computed from the DnSeg (mean, 0.68 [SD, 0.10]) significantly outperformed that of the probabilistic atlas (0.45 [SD, 0.11]) in the completely withheld data sets of 6 healthy subjects (paired t test, P = .001), as seen in Fig 3B. Additionally, the Dice coefficient was significantly increased in DnSeg (mean, 0.64 [SD, 0.10]) compared with the probabilistic atlas Dice coefficient (mean, 0.37 [SD, 0.22) for the TLE data set (paired t test, P < .001). The probabilistic atlas demonstrated substantial variability in the Dice coefficient for the TLE data set, likely due to anatomic changes that have been noted in the NBM for TLE.
Example of favorable (A, right), average (A, middle), and poor (A, left) performance. Blue is the correct prediction of NBM, red is the overpredicted NBM (false-positive), and yellow is the ground truth label that was not predicted (false-negative). The Dice coefficient of the favorable example (right) was 0.82, the Dice coefficient of the average example (middle) was 0.63, and the Dice coefficient of the poor example (left) was 0.55. B, The Dice coefficient. C, The mean surface distance. D, The centroid distance of DnSeg versus the probabilistic atlas for both healthy subjects and patients with TLE. The healthy subjects included the nonaugmented held-out test data set (n = 6). The patients with TLE were also not augmented and were held out until final analysis (n = 14). Paired t test: single asterisk indicates P < .05; double asterisks, P < .01; triple asterisks, P < .001.
Mean Surface Distance
The mean surface distance between the predicted surface and the ground truth surface of the NBM segmentation was calculated for the test data set and the TLE data set (Fig 3C). For healthy subjects, DnSeg demonstrated significantly lower mean surface distance (mean, 0.65 [SD, 0.24] mm) computed with the probabilistic atlas (mean, 1.18 [SD, 0.35] mm) in the completely withheld data set of 6 healthy subjects (paired t test, P = .002). Furthermore, the mean surface distance was significantly lower with DnSeg (0.69 [SD, 0.15] mm) compared with the probabilistic atlas (mean, 1.91 [SD, 2.03] mm) in the TLE data set (paired t test P = .027). The variability of the probabilistic atlas on the TLE data set was far more exaggerated than that of the healthy subject data set, as seen with the 95% confidence intervals of 1.2 and 0.2 mm, respectively.
Centroid Distance
The centroid distance represents the error in localization of the NBM that can occur due to patient-specific anatomic variability. It is most relevant for DBS applications because it could contribute to a targeting error. The centroid distance performance of DnSeg (mean, 1.35 [SD, 0.86] mm) was not significantly different from the probabilistic atlas (mean, 2.19 [SD, 1.36] mm) in the completely withheld data sets of 6 healthy subjects (paired t test, P = .014), as can be seen in Fig 3D. However, the centroid distance of DnSeg (mean, 1.18 [SD, 0.43] mm) was significantly lower than that of the probabilistic atlas (mean, 3.09 [SD, 2.56] mm) in the TLE data set (paired t test, P = .007).
Qualitative Comparison
Because localization and overlap are difficult to quantify for small structures, the results were also analyzed qualitatively. An evaluation of NBM prediction of DnSeg compared with the ground truth manual segmentation was completed qualitatively by visual inspection (Fig 3A). The poor-performance example demonstrates that even the poor-performing examples perform relatively well. Additionally, it is seen that the NBM shape differs slightly from patient to patient and that DnSeg successfully accounts for heterogeneous patient anatomy (Online Supplemental Data).
DISCUSSION
The purpose of this work was to enable study of the NBM by accounting for patient-specific differences. DnSeg may have widespread impact across several diseases, perhaps enabling new treatment modalities targeting the NBM.
DnSeg Can Distinguish Subject-Specific Anatomic Differences
The NBM has anatomic variability among healthy subjects and has been shown to change in size in patients with Parkinson disease, Alzheimer disease, and TLE.32⇓⇓⇓⇓⇓⇓⇓-40 This variability presents a substantial challenge in studying the NBM and a barrier for possible treatment innovations such as DBS. Manual segmentation to capture these differences would be preferable; however, the NBM cannot be accurately visualized on 3T MR imaging. Thus, current research is limited by the non-patient-specific atlas that does not capture variability in the NBM, as seen by the wide 95% confidence intervals for patients with TLE (Fig 3). DnSeg was able to capture patient-specific differences in the NBM and accurately segment it using only 3T T1-weighted MR imaging. The results of DnSeg in patients with TLE were comparable with those of healthy subjects, indicating that it can accurately segment the NBM despite anatomic changes (Fig 3B–D), demonstrating that DnSeg can capture pathologic changes in the NBM in at least 1 disease state.
DBS Targeting
DBS targeting for the NBM has been explored in several studies with mixed results.8,9 One possible factor in these studies is DBS targeting accuracy. In this investigation, DnSeg localized the NBM with far more accuracy than the probabilistic atlas, which can have a substantial impact on targeting (Fig 3). The impact of this distance can be seen in Fig 4, where an electrode planned with DnSeg seems to target the NBM accurately, while an electrode planned with the center of the 50% thresholded probabilistic atlas targets the border of the true NBM.
Increased accuracy of NBM electrode targeting with DnSeg. The orange electrode was placed in the center of the NBM segmentation generated by the probabilistic atlas. The blue electrode was placed in the center of the segmentation generated by DnSeg. D, Both electrode positions compared with the ground truth segmentation mesh. The ground truth NBM (manually segmented on 7T MR imaging) is shown in red. The outline of the ground truth NBM as well as the lead trajectories can be seen in A, B, and C. The sample DBS planning for this figure was generated using CRAnial Vault Explorer.23
In addition to this practical application, DnSeg also has 10× faster run-time than standard registration methods required for applying a Montreal Neurological Institute 152 atlas space to patient-specific space. With a modern CPU, it takes approximately 1 minute per scan if skull-stripping is not required and approximately 2 minutes if skull stripping is required. The pipeline does not require the patient data to be registered to any standard space; thus, it is much faster than other methods that require a registration step.
Limitations
Despite these benefits of the deep learning pipeline, there are some limitations. Most notably, the expert segmentations could not be verified with a postmortem histologic analysis. While the segmentations were verified by 2 neurological surgeons, a histologic analysis would be ideal. Furthermore, the current analysis validates only DnSeg in one disease state. The NBM is both of interest and has pathologic changes in several disease states. Thus, it would be preferred to validate DnSeg in other disease states. However, paired 7T and 3T MR imaging scans of the same patients needed to achieve this goal are rare. Furthermore, DnSeg is limited by the small sample size used for training. Although this limitation is common among studies of paired 3T and 7T MR imaging, it has the potential to limit the generalizability of DnSeg. Aggressive augmentation was used to attenuate this limitation and increase generalizability.
CONCLUSIONS
In this work, we have presented an accurate, patient-specific method of segmenting the NBM using only 3T T1-weighted MR imaging. The NBM has been implicated in Alzheimer disease, Parkinson disease dementia, and TLE.3,6,7 It has been shown to change in neuronal density and gray matter volume across age and disease states.32⇓⇓⇓⇓⇓⇓⇓-40 With the use of an expertly segmented NBM and aggressive data augmentation, we have trained a deep learning network to capture anatomic differences when segmenting the NBM. We have presented evidence that DnSeg is a powerful and accurate NBM segmentation model.
Methodologically, this work represents an innovative approach to the segmentation of regions with little contrast enhancement. The NBM represents a small region of the brain with little contrast on commonly used 3T MR imaging but with high importance to several disease states. Paired 3T and 7T MR imaging of the same subjects is a useful approach to providing accurate labels for training a deep learning network to segment structures that cannot be accurately visualized on 3T imaging. However, this approach is intrinsically limited by the rarity of paired 3T-7T data sets.
DnSeg, therefore, opens the possibility of further study of the NBM. The NBM has long been considered a region of key interest, but study of the NBM in vivo has been difficult, in part because of the lack of an adaptive, patient-specific atlas. The presented model is available (https://github.com/DerekDoss/DnSeg) and may greatly assist in novel studies of the NBM.
Acknowledgments
The authors would like to thank Laurie Cutting, PhD, for providing paired 7T-3T MR imaging scans of healthy subjects. The authors would also like to thank William (Bill) Rodriguez, MEd for his assistance with CRAVE and data set curation. A preprint of this article was originally released on biorxiv.
Footnotes
This work was supported by the National Institutes of Health (grant Nos. T32GM007347, T32EB001628-17, T32EB021937, F31NS106735, F31NS120401, R01NS112252, R01NS108445, R01NS110130, and R01NS116504).
Disclosure forms provided by the authors are available with the full text and PDF of this article at www.ajnr.org.
Indicates open access to non-subscribers at www.ajnr.org
References
- Received November 11, 2022.
- Accepted after revision June 25, 2023.
- © 2023 by American Journal of Neuroradiology
