Factors Influencing Confidence in Diagnostic Ratings and Retreatment Recommendations in Coiled Aneurysms

BACKGROUND AND PURPOSE: Angiographic occlusion and retreatment of coiled aneurysms are commonly used as surrogate end points in clinical trials. We aimed to evaluate the influence of aneurysm, patient, and rater characteristics on the confidence of visual evaluation of aneurysm coiling and retreatment decisions.

MATERIALS AND METHODS: Twenty-six participants of the Advanced Course in Endovascular Interventional Neuroradiology of the European Society of Neuroradiology were asked to evaluate digital subtraction angiography examinations of patients who had undergone endovascular coiling, by determining the grade of aneurysm occlusion, the change between immediate postprocedural and follow-up angiograms, their level of confidence, the technical difficulty of retreatment, and the best therapeutic approach. The experience, knowledge, and skills of each participant were assessed. The influence of rater and case characteristics on indicated confidence in diagnostic ratings and retreatment recommendations was analyzed.

RESULTS: Interrater reliability was moderate regarding the assessment of aneurysm occlusion grade (intraclass correlation coefficient = 0.581) and substantial regarding change (intraclass correlation coefficient = 0.776). Overall confidence in the diagnostic rating was high (median, “very certain”). Confidence was statistically significantly higher in cases that were generally rated as “worse.” The odds of recommending retreatment were significantly higher in cases that were generally rated with higher mean confidence.

CONCLUSIONS: Although overall confidence in the diagnostic rating was high, our study confirms the suboptimal interrater reliability of visual assessment of aneurysm occlusion as well as retreatment recommendations, rendering both questionable as primary outcome measures. Besides recurrence status, recommendation of retreatment is significantly influenced by patient age, aneurysm neck width, and characteristics of the therapist.