System performance of BioBERT on the test data seta
| Category | Sensitivity (%) (95% CI) | Specificity (%) (95% CI) | AUC (95% CI) |
|---|---|---|---|
| Non-Definitive | 76.71 (56/73) (65.35–85.81) | 90.24 (148/164) (84.64–94.32) | 0.919 (0.874–0.964) |
| Definitive-Mild | 59.52 (25/42) (43.28–74.37) | 88.72 (173/195) (83.42–92.79) | 0.843 (0.76–0.92) |
| Definitive-Strong | 74.6 (47/63) (62.06–84.73) | 95.4 (166/174) (91.14–97.99) | 0.964 (0.931–0.997) |
| Other | 98.31 (58/59) (90.91–99.96) | 97.19 (173/178) (93.57–99.08) | 0.994 (0.979–1) |
| Macro Avg | 77.29 (65.4–86.22) | 92.89 (88.19–96.05) | 0.93 (0.888–0.972) |
Note:—Macro Avg indicates average on the macro level across different categories.
↵a Numerators and denominators for sensitivity and specificity are included in parentheses.