MR Contrast Media in Neuroimaging: A Critical Review of the Literature

Gadolinium-based MR contrast agents have been widely applied since they were first available for clinical use in 1988. By March 1993, more than 5.4 million doses had been administered (1). At an approximate hospital charge of $150 per dose, the use of contrast material accounted for almost $1 billion during the first 6 years of its clinical application. Most contrast material was used in neuroimaging, with an established role in lesion detection, in characterization, and in improving radiologists' confidence regarding interpretation.

In the current environment of cost containment, an established role may not warrant continued government and corporate support for a specific technology. Increasingly, these financially involved entities require evidence-based practice guidelines and patient outcomes data based on rigorous technology assessment methodology. We sought to apply such criteria to the evidence for clinical efficacy; that is, to the probability that a patient will derive benefit, under optimum conditions, from contrast-enhanced MR neuroimaging. This analysis could yield evidence-based guidelines for the use of contrast material or serve as a demonstration of what is lacking in the evidence at this time.

Methods

Results

Of a total of 756 ratings (seven criteria for 108 articles), there were 50 disagreements (7%). The disagreements were most commonly regarding the ratings of the reference test, cohort assembly, and clinical description.

Of the 108 articles rated, one was rated B (4), six were rated C, and 101 were rated D. The distribution of ratings among the seven quality criteria is listed separately (see the table). Index test quality received the highest ratings (61 of 108) because the reporting institutions generally used state-of-the-art equipment. The authors focused most of the attention on the quality of imaging. Reference test quality also frequently earned high or intermediate ratings (33 of 108 high) because of the availability of pathologic specimens or close surgical collaboration. The availability of pathologic proof often was the means of selection into a study. The difficulty in uniformly applying an acceptable standard of reference led to lower quality ratings in the application of the reference test. Authors failed to apply stringently the requirement that all analyzed cases have the same reference standard. In some articles, a few cases with no standard of reference could have been eliminated while maintaining an adequate sample size. Approximately one third of the articles did not use a reference standard. These shortcomings precluded the calculation of accuracy statistics because positivity and negativity of the contrast-enhanced MR imaging could not be assigned.

Regarding the next four criteria, very few high ratings were assigned. Independence of interpretation, which rates the avoidance of review bias, is a central concern in study design. In general, the Methods section of articles did not document explicit separation between the interpretation of the contrast-enhanced MR imaging findings and the standard of reference. Intermediate articles usually described interpretation of the contrast-enhanced MR images without knowledge of the final diagnosis but did not describe the procedures used to prevent the MR findings from affecting the final diagnosis. The assessment of clinical description yielded only 11 high ratings. In many articles, the clinical description consisted only of summary statements, such as “suspected intracranial pathology.” It is likely that many studies could have been improved by including reviews of medical record. On the other hand, cohort assembly represents the most difficult aspect of study design for radiologists, who may have little influence over the spectrum of disease they see. Correspondingly, only six articles (5–10) were rated high for cohort assembly. Unless they have close clinician collaboration from the beginning of a study design, radiologists must focus on retrospective case selection. In addition, case accrual at a tertiary care center usually includes substantial referral filtering. Finally, almost all articles presented sample sizes that were inadequate to yield robust statistics and did not present control cases. Not one of the five articles with high ratings for sample size (5, 11–14) applied reference tests.

Discussion

This critical literature review, consisting of structured blinded ratings, was originally designed as a metaanalysis, to derive pooled estimates of sensitivity and specificity for the use of contrast material in neuroimaging. The available studies, unfortunately, did not themselves yield valid accuracy measures, which precluded metaanalysis. One finding of our study, therefore, is that no valid measures of sensitivity or specificity exist for the application of MR contrast material in neuroimaging.

We did apply our study design to uncover surprisingly prevalent methodological flaws. In a critical literature review evaluating diagnostic tests in general during a 16-year period, Reid et al (15) similarly found inadequate assessment of diagnostic tests, although use of methodological standards did increase during the study interval from 1978 to 1993. In their study, radiologic tests were the largest single category of diagnostic test evaluated. Although their results were described as particularly disturbing, our results document a much lower prevalence of acceptable methodological standards. For example, Reid et al reported that 47% of analyzed articles avoided review bias during the interval between 1990 and 1993, as compared with only 4% (four of 108) in our study.

The one article that was assigned a B rating contains basic methodological elements that any study of diagnostic accuracy should include. In 1992, Wiebe et al (4) reported their study using craniospinal MR imaging with contrast enhancement to serially examine patients with multiple sclerosis (MS). All patients underwent cranial and spinal MR imaging on at least three occasions at 13-week intervals, with additional imaging performed if clinical relapses occurred during the study period. The clinical judgment of a neurologist at a university MS clinic with respect to presence or absence of disease activity was used as the standard of reference. Patients with quiescent disease were included in the study. The examining neurologists were blinded to the MR findings, and the radiologists were blinded to the patients' clinical status. The article contains a table that clearly describes the spectrum of disease evaluated. In this manner, the authors were able to construct a standard two-by-two table showing test positivity and negativity, as well as the presence or absence of disease. The Methods section presents details that allow the reader to recognize the limitations in generalizing, which include a somewhat narrow spectrum of disease and a small sample size.

The reported accuracy statistics in the article by Wiebe et al (4), although favorable, were for the use of MR imaging overall. Contrast enhancement for cranial imaging was used only in those cases in which evidence of activity had already been noted on unenhanced sequences. However, contrast enhancement was the sole evidence of disease activity in 5% of all spinal cord images. These results parallel the clinical use of contrast material in the evaluation of patients with MS.

Because low-field MR systems are gaining acceptance, we undertook an informal analysis of our data and assigned high ratings to all articles for index test quality. We found that no article gained a higher overall rating, because the rating system values the presence and quality of a standard of reference over other criteria.

Conclusion

We have shown that strong evidence-based guidelines for the use of contrast material in neuroimaging cannot be derived from the current literature. In short, the clinical efficacy of contrast material remains unproved. Future investigations need to focus on constructing robust measures of diagnostic accuracy. These studies will require larger sources of funding to construct appropriate randomized controlled trials, as demonstrated by the highest rated article in our study. Researchers in neuroradiology need to apply such methodologies, which have been used successfully in advancing other areas of medicine. Only when we rigorously evaluate the ability of contrast material to aid in the diagnosis and exclusion of disease can we proceed to the evaluation of its diagnostic and therapeutic impact.