Performance of Neck Imaging Reporting and Data System (NI-RADS) for Diagnosis of Recurrence of Head and Neck Squamous Cell Carcinoma: A Systematic Review and Meta-analysis
===========================================================================================================================================================================

* Akira Baba
* Ryo Kurokawa
* Mariko Kurokawa
* Takafumi Yanagisawa
* Ashok Srinivasan

## Abstract

**BACKGROUND:** The Neck Imaging Reporting and Data System (NI-RADS) is a reporting template used in head and neck cancer posttreatment follow-up imaging.

**PURPOSE:** Our aim was to evaluate the pooled detection rates of the recurrence of head and neck squamous cell carcinoma based on each NI-RADS category and to compare the diagnostic accuracy between NI-RADS 2 and 3 cutoffs.

**DATA SOURCES:** The MEDLINE, Scopus, and EMBASE databases were searched.

**STUDY SELECTION:** This systematic review identified 7 studies with a total of 694 patients (1233 lesions) that were eligible for the meta-analysis.

**DATA ANALYSIS:** The meta-analysis of pooled recurrence detection rate estimates for each NI-RADS category and the diagnostic accuracy of recurrence with NI-RADS 3 or 2 as the cutoff was performed.

**DATA SYNTHESIS:** The estimated recurrence rates in each category for primary lesions were 74.4% for NI-RADS 3, 29.0% for NI-RADS 2, and 4.2% for NI-RADS 1. The estimated recurrence rates in each category for cervical lymph nodes were 73.3% for NI-RADS 3, 14.3% for NI-RADS 2, and 3.5% for NI-RADS 1. The area under the curve of the summary receiver operating characteristic for recurrence detection with NI-RADS 3 as the cutoff was 0.887 and 0.983, respectively, higher than 0.869 and 0.919 for the primary sites and cervical lymph nodes, respectively, with NI-RADS 2 as the cutoff.

**LIMITATIONS:** Given the heterogeneity of the data of the studies, the conclusions should be interpreted with caution.

**CONCLUSIONS:** This meta-analysis revealed estimated recurrence rates for each NI-RADS category for primary lesions and cervical lymph nodes and showed that NI-RADS 3 has a high diagnostic performance for detecting recurrence.

## ABBREVIATIONS:

AUC
:   area under the curve
CE-CT
:   contrast-enhanced CT
CE-MRI
:   contrast-enhanced MRI
DOR
:   diagnostic odds ratio
NI-RADS
:   Neck Imaging Reporting and Data System
sROC
:   summary receiver operating characteristic

Follow-up imaging after head and neck cancer treatment is used for the assessment of the treatment response and the detection of recurrence. Recurrences may involve the primary site and/or cervical lymph nodes, and early detection of such recurrences may facilitate subsequent salvage therapy.1,2 Posttreatment follow-up imaging of head and neck cancer is often challenging, however, because of the anatomic complexity of the head and neck region, complex resection and reconstruction operations, and the posttreatment effects of radiation and chemotherapy that mimic recurrent disease. These factors affect radiologists’ interpretations, rendering them nonuniform and potentially suboptimal.3,4

The Neck Imaging Reporting and Data System (NI-RADS) is a head and neck cancer posttreatment follow-up imaging reporting template that was proposed by the American College of Radiology in 2016 to standardize imaging interpretation and communication between clinicians and radiologists.5 NI-RADS provides standardized terminology, report structure, and evaluation categories to convey the degree of suspicion of recurrence in the interpretation of imaging studies. The NI-RADS lexicon established for the evaluation of both posttreatment primary sites and cervical lymph nodes has 4 categories (category 1 [no evidence of recurrence], category 2 [low suspicion], category 3 [high suspicion], and category 4 [definitive recurrence]). NI-RADS 1–3 provide linked recommendations for clinical management along with an estimate of the degree of suspicion for recurrent head and neck cancer.

Most previously published studies of the diagnostic performance of NI-RADS have been limited by small sample sizes. Therefore, the purpose of this systematic article was to summarize the existing data and estimate the detection rate for recurrent head and neck squamous cell carcinoma for each NI-RADS category and to compare diagnostic test accuracy estimates using NI-RADS 3 versus 2 cutoffs for detecting recurrent lesions.

## MATERIALS AND METHODS

### Study Selection

This study was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement.6 We searched the Cochrane database and confirmed that there were no reviews/meta-analyses similar to the present research design. On November 10, 2022, MEDLINE via PubMed, Scopus, and EMBASE databases were screened using the following search terms, without any language or date limits: “Neck Imaging Reporting and Data System” or “NI-RADS.”

Inclusion criteria for this evaluation were as follows: 

*   Data on the number of lesions in each NI-RADs category and the number of proven primary site or cervical lymph node recurrences

*   Data including either primary sites or lymph nodes in NI-RADs 1, 2, or 3

*   Data with contrast-enhanced CT (CE-CT), contrast-enhanced MRI (CE-MRI), PET/CT, or PET/MRI

*   Data with the pathology of squamous cell carcinoma only

*   Original studies that investigated human findings

*   In cases of duplicate publications, the highest quality or most recent publication was selected

*   Written in English

The exclusion criteria were as follows: 

*   Studies published before 2016

*   Studies without an identified imaging period from treatment

*   The full text was unavailable

*   Studies with incomplete data

*   Review, case reports, and systematic review/meta-analysis articles

*   Books and conference proceedings only that lacked an associated peer-reviewed full-fledged publication

### Data Extraction

Two board-certified radiologists with 13 and 9 years of experience, respectively, in head and neck radiology reviewed the full text of the eligible studies and extracted the following information from the included studies by consensus: first author’s name, study region, publication year, study period, study design, number of patients, age, sex, tumor subsite, pathology, treatment method, type of imaging technique, vendor and model of equipment used, imaging period from therapy, reference standard, and recurrent and nonrecurrent lesions for each NI-RADS category. Any disagreements were resolved by consensus agreement of the evaluators.

### Quality and Risk Assessment

The Newcastle-Ottawa Scale was used to assess the quality of the included studies in accordance with the Cochrane Handbook for Systematic Reviews of Interventions for included nonrandomized studies.7,8 The scale rates the following 3 factors: selection (0–4 points), comparability (0–2 points), and exposure (0–3 points), with total scores ranging from 0 (lowest) to 9 (highest). Studies with scores of >6 were identified as “high-quality” choices.

### Data Analyses

Proportional meta-analyses were performed using a random effects model or a common effects model to determine the estimated prevalence of recurrent disease for each NI-RADS category. The following data were available for this analysis: 2 articles for which all data from NI-RADS 1–3 of the primary lesions and cervical lymph nodes were available,9,10 1 article with only NI-RADS 3 data available for the primary lesions and cervical lymph nodes,11 1 article with only NI-RADS 2 and NI-RADS 3 data points available for the primary lesion,12 1 article with only primary lesion data from NI-RADS 1–3 due to an additional modification of the NI-RADS assessment for the cervical lymph node,13 1 article with only primary lesion data from NI-RADS 1–3 available,14 and 1 article with only cervical lymph node data from NI-RADS 1–3 available due to case overlap of primary lesion cases with another article.15 Forest plots were used to assess and summarize the data. Heterogeneity among the outcomes of the studies included in this article was evaluated using the I2 statistic.

Significant heterogeneity was indicated by a ratio >50% in I2 statistics. Publication bias was assessed using funnel plots. In 4 articles for which all case number data for recurrence and nonrecurrence from each of NI-RADS 1–3 at the primary lesions were available9,10,13,14 or in 3 articles for which all case number data for recurrence and nonrecurrence from each of NI-RADS 1–3 at the cervical lymph nodes were available,9,10,15 we divided the data for the diagnostic accuracy analysis into groups of NI-RADS 3 and NI-RADS 1/2 when NI-RADS 3 was used as the cutoff, and into groups of NI-RADS 2/3 and NI-RADS 1 when NI-RADS 2 was used as the cutoff. Data were pooled using random or fixed effects models to summarize the estimates of sensitivity, specificity, and diagnostic odds ratio (DOR). Bivariate models were used to construct summary receiver operator characteristic (sROC) curves and calculate the area under the curve (AUC). All statistical analyses were performed using R, Version 4.2.2 ([http://www.r-project.org/](http://www.r-project.org/)).

## RESULTS

### Study Selection and Characteristics

Our initial search identified 257 records, and 185 remained after removing duplicates and/or conference proceedings and book chapters. In the next screening, 85 articles published before 2016, non-English language reports, review articles, case reports, and systematic reviews/meta-analyses were excluded. After applying the inclusion/exclusion criteria, we identified 7 articles with 694 patients (1233 lesions) for this review (Fig 1).9⇓⇓⇓⇓⇓-15

![FIG 1.](http://www.ajnr.org/https://www.highwirestage.com/content/ajnr/early/2023/09/14/ajnr.A7992/F1.medium.gif)

[FIG 1.](http://www.ajnr.org/content/early/2023/09/14/ajnr.A7992/F1)

FIG 1. 
The PRISMA 2020 flow chart for the article-selection process. After applying the selection criteria, we identified 7 articles for the systematic review and the meta-analysis.

The data extracted from the 7 studies are outlined in the Online Supplemental Data. All were published between 2019 and 2022, with 2 studies from North America, 2 from South Asia, 2 from Africa, and 1 from Europe. The studies had a median Newcastle-Ottawa Scale score of 4 (range, 4−5). The 6 studies for which information regarding the sex of individual participants was available included 419 men and 191 women (male/female ratio = 2.2:1). The studies had a mean age range of 49–63.4 years and a median age range of 59–62 years. The primary tumor subsites in the studies included the nasopharynx, oropharynx, hypopharynx, larynx, oral cavity, sinonasal cavity, and salivary gland. The 6 studies for which treatment methods were available included radiation therapy, chemoradiotherapy, surgery, surgery plus radiation therapy, and surgery plus chemoradiotherapy. The imaging modalities in the studies were CE-CT, CE-MRI, PET/CT, or PET/CT with CE-CT. Imaging was performed >1.5−3 months after completion of therapy.

In the reference standard, histology and follow-up were used in all articles to determine recurrent lesions, whereas only follow-up was used in 3 articles to determine nonrecurrent lesions. In NI-RADS at the primary site, 6 articles were available for NI-RADS 3; five, for NI-RADS 2; and 4, for NI-RADS 1, for a total of 710 lesions evaluated. In NI-RADS for cervical lymph nodes, data for 4 articles were available for NI-RADS 3, three for NI-RADS 2, and 3 for NI-RADS 1, for a total of 523 lesions evaluated. For diagnostic performance evaluation using NI-RADS 3 or NI-RADS 2 as a cutoff, 4 articles were available for NI-RADS at the primary sites and 3 articles were available for NI-RADS at the lymph nodes. Seven studies reported 270 recurrent lesions and 963 nonrecurrent lesions at the primary sites and cervical lymph nodes.

### Meta-analysis of Summarized Recurrence Rate Estimates for Each NI-RADS Category

A forest plot of the summarized estimates of the prevalence for each NI-RADS category of recurrent lesions at the primary sites or cervical lymph nodes is shown in Figs 2 and 3. At the primary site, the estimated recurrence rate for NI-RADS 3 was 74.4% (95% CI, 59.2%−85.4%; I2 = 74%); for NI-RADS 2, it was 29.0% (95% CI, 21.2%−38.2%; I2 = 47%); and for NI-RADS 1, it was 4.2% (95% CI, 2.6%−6.7%; I 2= 0%). For cervical lymph nodes, the estimated recurrence rates for NI-RADS 3, NI-RADS 2, and NI-RADS 1 were 73.3% (95% CI, 63.0%−81.5%; I2 = 38%), 14.3% (95% CI, 6.1%−30.0%; I2 = 0%), and 3.5% (95% CI, 2.1%−5.8%; I2 = 0%), respectively. Funnel plots of these results are shown in the Online Supplemental Data.

![FIG 2.](http://www.ajnr.org/https://www.highwirestage.com/content/ajnr/early/2023/09/14/ajnr.A7992/F2.medium.gif)

[FIG 2.](http://www.ajnr.org/content/early/2023/09/14/ajnr.A7992/F2)

FIG 2. 
Forest plot of summary estimates of the prevalence of recurrence for each NI-RADS category at the primary site (*A*, NI-RADS 3. *B*, NI-RADS 2. *C*, NI-RADS 1).

![FIG 3.](http://www.ajnr.org/https://www.highwirestage.com/content/ajnr/early/2023/09/14/ajnr.A7992/F3.medium.gif)

[FIG 3.](http://www.ajnr.org/content/early/2023/09/14/ajnr.A7992/F3)

FIG 3. 
Forest plot of summary estimates of the prevalence of recurrence for each NI-RADS category at the neck node (*A*, NI-RADS 3. *B*, NI-RADS 2. *C*, NI-RADS 1).

### Meta-analysis of Diagnostic Test Accuracy of Recurrence with NI-RADS 3 or 2 as the Cutoff

#### Primary Lesion.

A forest plot of the summarized estimates of sensitivity, specificity, and DOR of recurrence detection with NI-RADS 3 or NI-RADS 2 as the cutoff in the primary lesion is shown in the Online Supplemental Data, and the sROC for diagnostic performance is shown in Fig 4. With NI-RADS 3 as the cutoff, the estimated sensitivity for recurrence was 60.6% (95% CI, 39.5%−78.4%), the estimated specificity was 94.6% (95% CI, 84.6%−98.3%), the estimated DOR was 26.6 (95% CI, 13.5−52.4), and the AUC in sROC was 0.887. With NI-RADS 2 as the cutoff, the estimated sensitivity for recurrence was 81.8% (95% CI, 54.5%−94.4%), the estimated specificity was 76.6% (95% CI, 29.5%−96.2%), the estimated DOR was 18.9 (95% CI, 9.4−37.9), and the AUC in sROC was 0.869.

![FIG 4.](http://www.ajnr.org/https://www.highwirestage.com/content/ajnr/early/2023/09/14/ajnr.A7992/F4.medium.gif)

[FIG 4.](http://www.ajnr.org/content/early/2023/09/14/ajnr.A7992/F4)

FIG 4. 
sROC for diagnostic performance in the primary site. *A*, sROC with NI-RADS 3 as the cutoff. *B*, sROC with NI-RADS 2 as the cutoff.

#### Lymph Nodes.

A forest plot of the summarized estimates of sensitivity, specificity, and the DOR of recurrence detection with NI-RADS 3 or NI-RADS 2 as the cutoff for cervical lymph nodes is shown in the Online Supplemental Data, and the sROC for diagnostic performance is shown in Fig 5. With NI-RADS 3 as the cutoff, the estimated sensitivity for recurrence was 66.9% (95% CI, 28.9%–90.9%), the estimated specificity was 98.6% (95% CI, 96.7%–99.4%), the estimated DOR was 96.0 (95% CI, 31.1–296.2), and the AUC in sROC was 0.983. With NI-RADS 2 as the cutoff, the estimated sensitivity for recurrence was 75.7% (95% CI, 38.5%–94.0%), the estimated specificity was 89.3% (95% CI, 78.8%–95.0%), the estimated DOR was 21.3 (95% CI, 9.9–45.9), and the AUC in the sROC was 0.919.

![FIG 5.](http://www.ajnr.org/https://www.highwirestage.com/content/ajnr/early/2023/09/14/ajnr.A7992/F5.medium.gif)

[FIG 5.](http://www.ajnr.org/content/early/2023/09/14/ajnr.A7992/F5)

FIG 5. 
sROC for diagnostic performance in the lymph node. *A*, sROC with NI-RADS 3 as the cutoff. *B*, sROC with NI-RADS 2 as the cutoff.

## DISCUSSION

This systematic review and meta-analysis used 7 studies reporting 694 patients and 1233 lesions to calculate the summarized estimated detection rates of head and neck cancer recurrence for each NI-RADS category and to compare the diagnostic accuracy of NI-RADS 2 and 3 cutoffs to define the optimal cutoff value. The estimated recurrence rates in the categories of primary lesions and cervical lymph nodes were 73.3%−74.4%, 14.3%−29.0%, and 3.5%−4.2% for NI-RADS 3, NI-RADS 2, and NI-RADS 1, respectively. Furthermore, the summarized estimates of specificity, the DOR, and the AUC of the sROC for recurrence detection were higher when NI-RADS 3 was used as a cutoff in primary lesions or cervical lymph nodes than when NI-RADS 2 was used.

The current lexicon in NI-RADS is based on standardized report templates specific to CE-CT and PET/CE-CT, though NI-RADS can also be used for the interpretation of CE-MRI or PET/MR imaging.5,16 Therefore, in addition to CE-CT and PET/CT, there have been a number of articles regarding NI-RADS using CE-MRI and PET/MR imaging.10,12,14,15,17⇓⇓⇓-21 The inclusion of T2-weighted images, DWI, and ADC findings improves the diagnostic performance of NI-RADS.12 Other important MR imaging findings include quantitative values such as ADC values and dynamic contrast-enhanced MR imaging parameters, which are reported to be useful in differentiating recurrent head and neck cancer and posttreatment effects.22,23 Further reporting on the utility of these qualitative and quantitative MR imaging findings for NI-RADS incorporation or the establishment of a revised NI-RADS lexicon that includes these findings and assessment parameters may be warranted in the future but is beyond the scope of this work.

Each NI-RADS category has a different set of clinical recommendations that have been proposed as follows: NI-RADS 2: clinical evaluation of the concerning mucosal region, relatively close follow-up (∼ 3 months) or FDG-PET; and NI-RADS 3 recommends tissue correlation.5 The results of the current study showed that the estimated specificity, estimated DOR, and AUC for head and neck cancer recurrence for NI-RADS 3 as a cutoff were higher than those for NI-RADS 2 for both primary lesions and cervical lymph nodes, with an estimated recurrence rate in NI-RADS 3 as high as 74.5%−74.6%. These results support using NI-RADS 3 as a cutoff for recurrent lesion detection and proceeding to the linked clinical recommendation for NI-RADS 3 of biopsy, an invasive procedure. However, the estimated recurrence rates of 12.2%−29.8% for NI-RADS 2 in the results of this study are a relatively cautionary prevalence that cannot be considered safe. Therefore, close follow-up and direct testing as recommended by the current lexicon for NI-RADS 2 is validated by this analysis.

The utility of liquid biopsy for monitoring recurrence in head and neck cancer is promising24,25 and may have the potential to be incorporated into future NIRADS clinical recommendations.

Our study does have certain limitations, most notably the limited number of available studies. The Newcastle-Ottawa Scale score for all studies was low (4–5), suggesting a high risk of bias. Variations in the type of imaging technique, the location of head and neck cancers, time from treatment to imaging studies, and the reference method among the included studies could have influenced the heterogeneous outcomes. One study that included only NI-RADS 3 cases11 and another study that included only primary NI-RADS 2 and NI-RADS 3 cases12 may have disproportionately affected the results of the meta-analysis of proportions. Some funnel plots exhibited asymmetry indicating potential publication bias. Due to the limited number of included studies, however, a precise evaluation of publication bias was not feasible. Such heterogeneity and bias among studies must be considered a potential limitation when assessing the significance of this analysis.

The NI-RADS assessments and outcome differences may also have been influenced by the treatment method. In the NI-RADS, only local lesions could be classified into NI-RADS 2a and 2b.5 Of the articles included in this study, the number of recurrences and nonrecurrences in NI-RADS 2a and 2b could be extracted for only 2 articles.10,13 Due to the limited number of available articles, this study could not perform a meta-analysis of the NI-RADS 2a/2b. Further studies are warranted to comprehensively evaluate the diagnostic performance of the NI-RADS 2a/2b cutoff. Last, although random effects models were used in some of the tests to treat heterogeneity across studies, our conclusions should still be interpreted with caution because the underlying studies on this topic were not strongly based.26 Appropriately designed prospective large-scale trials are needed to validate the results of this study.

## CONCLUSIONS

This meta-analysis demonstrated that NI-RADS 3 has a high diagnostic performance for detecting clinically significant recurrence and confirmed that a NI-RADS category 3 is the optimal cutoff value as a clinical recommendation linked to tissue sampling. Given that NI-RADS 2 lesions also have relatively high estimated recurrence rates, careful follow-up is mandatory in this group of patients. Considering the limitations, including procedural and methodologic heterogeneity among the eligible studies, further investigation and validation of this study are needed.

## Acknowledgments

We gratefully acknowledge Mr Mark MacEachern (Taubman Health Sciences Library, University of Michigan) for his advice and Dr Roberto Rivera-de Choudens (Division of Neuroradiology, Department of Radiology, University of Michigan) for his English proofreading.

## Footnotes

*   [Disclosure forms](http://www.ajnr.org/sites/default/files/additional-assets/Disclosures/October%202023/0269.pdf) provided by the authors are available with the full text and PDF of this article at [www.ajnr.org](http://www.ajnr.org).

## References

1.  1.Wong LY, Wei WI, Lam LK, et al. Salvage of recurrent head and neck squamous cell carcinoma after primary curative surgery. Head Neck 2003;25:953–59 doi:10.1002/hed.10310 pmid:14603456
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1002/hed.10310&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=14603456&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000186160500009&link_type=ISI) 

2.  2.van Weert S, Leemans CR. Salvage surgery in head and neck cancer. Oral Dis 2021;27:117–24 doi:10.1111/odi.13582 pmid:32738064
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1111/odi.13582&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=32738064&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

3.  3.Baba A, Ojiri H, Ikeda K, et al. Essentials on oncological imaging: postoperative computed tomography and magnetic resonance imaging of oral tongue cancer. Can Assoc Radiol J 2018;69:458–67 doi:10.1016/j.carj.2018.08.001 pmid:30390963
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.carj.2018.08.001&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=30390963&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

4.  4.Strauss SB, Aiken AH, Lantos JE, et al. Best practices: application of NI-RADS for posttreatment surveillance imaging of head and neck cancer. AJR Am J Roentgenol 2021;216:1438–51 doi:10.2214/AJR.20.23841 pmid:32876470
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.2214/AJR.20.23841&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=32876470&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

5.  5.Aiken AH, Rath TJ, Anzai Y, et al. ACR Neck Imaging Reporting and Data Systems (NI-RADS): a white paper of the ACR NI-RADS Committee. J Am Coll Radiology 2018;15:1097–108 doi:10.1016/j.jacr.2018.05.006 pmid:29983244
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.jacr.2018.05.006&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=29983244&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

6.  6.Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71 doi:10.1136/bmj.n71 pmid:33782057
    
    [FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE1OiIzNzIvbWFyMjlfMi9uNzEiO3M6NDoiYXRvbSI7czozODoiL2FqbnIvZWFybHkvMjAyMy8wOS8xNC9ham5yLkE3OTkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

7.  7.Stang A. Critical evaluation of the Newcastle-Ottawa Scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol 2010;25:603–05 doi:10.1007/s10654-010-9491-z pmid:20652370
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s10654-010-9491-z&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=20652370&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 
    
    [Web of Science](http://www.ajnr.org/lookup/external-ref?access_num=000282102200001&link_type=ISI) 

8.  8.Deeks JJ, Dinnes J, D'Amico R, et al; European Carotid Surgery Trial Collaborative Group. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7:iii–x. 1–173 doi:10.3310/hta7270 pmid:14499048
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.3310/hta7270&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=12633527&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

9.  9.Hsu D, Chokshi FH, Hudgins PA, et al. Predictive value of first posttreatment imaging using standardized reporting in head and neck cancer. Otolaryngol Head Neck Surg 2019;161:978–85 doi:10.1177/0194599819865235 pmid:31331239
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1177/0194599819865235&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=31331239&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

10. 10.Elsholtz FH, Ro SR, Shnayien S, et al. Impact of double reading on NI-RADS diagnostic accuracy in reporting oral squamous cell carcinoma surveillance imaging: a single-center study. Dentomaxillofac Radiol 2022;51:1–8 doi:10.1259/dmfr.20210168 pmid:34233509
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1259/dmfr.20210168&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=34233509&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

11. 11.Wangaryattawanich P, Branstetter BF, Ly JD, et al. Positive predictive value of neck imaging reporting and data system categories 3 and 4 posttreatment FDG-PET/CT in head and neck squamous cell carcinoma. AJNR Am J Neuroradiol 2020;41:1070–75 doi:10.3174/ajnr.A6589 pmid:32467187
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo5OiI0MS82LzEwNzAiO3M6NDoiYXRvbSI7czozODoiL2FqbnIvZWFybHkvMjAyMy8wOS8xNC9ham5yLkE3OTkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

12. 12.Jajodia A, Mandal G, Yadav V, et al. Adding MR diffusion imaging and T2 signal intensity to Neck Imaging Reporting and Data System categories 2 and 3 in primary sites of postsurgical oral cavity carcinoma provides incremental diagnostic value. AJNR Am J Neuroradiol 2022;43:1018–23 doi:10.3174/ajnr.A7553 pmid:35738671
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo5OiI0My83LzEwMTgiO3M6NDoiYXRvbSI7czozODoiL2FqbnIvZWFybHkvMjAyMy8wOS8xNC9ham5yLkE3OTkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

13. 13.Kumar I, Reza SO, Choudhary S, et al. Performance of NI-RADS on CECT alone to predict recurrent head and neck squamous cell carcinoma after chemoradiotherapy: added value of RECIST 1.1. Indian J Radiol Imaging 2022;32:151–58 doi:10.1055/s-0042-1754315 pmid:35924129
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1055/s-0042-1754315&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=35924129&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

14. 14.Ashour MM, Darwish EA, Fahiem RM, et al. MRI posttreatment surveillance for head and neck squamous cell carcinoma: proposed MR NI-RADS criteria. AJNR Am J Neuroradiol 2021;42:1123–29 doi:10.3174/ajnr.A7058 pmid:33707288
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo5OiI0Mi82LzExMjMiO3M6NDoiYXRvbSI7czozODoiL2FqbnIvZWFybHkvMjAyMy8wOS8xNC9ham5yLkE3OTkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

15. 15.Abdelrahman AS, Ashour MM, Abdelaziz TT. Predictive value of Neck Imaging Reporting and Data System (NIRADS) in CECT/CEMRI of laryngeal and oral cavity squamous cell carcinoma. Egypt J Radiol Nucl Med 2020;51:241 doi:10.1186/s43055-020-00366-0
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1186/s43055-020-00366-0&link_type=DOI) 

16. 16.Aiken AH, Hudgins PA. Neck Imaging Reporting and Data System. Magn Reson Imaging Clin N Am 2018;26:51–62 doi:10.1016/j.mric.2017.08.004 pmid:29128006
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1016/j.mric.2017.08.004&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=29128006&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

17. 17.Lee J, Kaht D, Ali S, et al. Performance of the Neck Imaging Reporting and Data System as applied by general neuroradiologists to predict recurrence of head and neck cancers. Head Neck 2022;44:2257–64 doi:10.1002/hed.27138 pmid:35801334
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1002/hed.27138&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=35801334&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

18. 18.Abdelaziz TT, Abdel Razk AA, Ashour MM et al. Interreader reproducibility of the Neck Imaging Reporting and Data system (NI-RADS) lexicon for the detection of residual/recurrent disease in treated head and neck squamous cell carcinoma (HNSCC). Cancer Imaging 2020;20:1–10 doi:10.1186/s40644-020-00337-8 pmid:32811559
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1186/s40644-020-00337-8&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=32811559&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

19. 19.Dinkelborg P, Ro SR, Shnayien S, et al. Retrospective evaluation of NI-RADS for detecting postsurgical recurrence of oral squamous cell carcinoma on surveillance CT or MRI. AJR Am J Roentgenol 2021;217:198–206 doi:10.2214/AJR.20.24209 pmid:32876497
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.2214/AJR.20.24209&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=32876497&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

20. 20.Elsholtz FH, Erxleben C, Bauknecht HC, et al. Reliability of NI-RADS criteria in the interpretation of contrast-enhanced magnetic resonance imaging considering the potential role of diffusion-weighted imaging. Eur Radiol 2021;31:6295–304 doi:10.1007/s00330-021-07693-4 pmid:33533989
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1007/s00330-021-07693-4&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=33533989&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

21. 21.Patel LD, Bridgham K, Ciriello J, et al. PET/MR imaging in evaluating treatment failure of head and neck malignancies: a Neck Imaging Reporting and Data System–based study. AJNR Am J Neuroradiol 2022;43:435–41 doi:10.3174/ajnr.A7427 pmid:35177543
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo4OiI0My8zLzQzNSI7czo0OiJhdG9tIjtzOjM4OiIvYWpuci9lYXJseS8yMDIzLzA5LzE0L2FqbnIuQTc5OTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

22. 22.Baba A, Kurokawa R, Kurokawa M, et al. Apparent diffusion coefficient for differentiation between benign post-treatment changes and recurrence in head and neck cancer: a systematic review and meta-analysis. AJNR Am J Neuroradiol 2022;43:442–47 doi:10.3174/ajnr.A7431 pmid:35210272
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo4OiI0My8zLzQ0MiI7czo0OiJhdG9tIjtzOjM4OiIvYWpuci9lYXJseS8yMDIzLzA5LzE0L2FqbnIuQTc5OTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

23. 23.Baba A, Kurokawa R, Kurokawa M, et al. Dynamic contrast‐enhanced MRI parameters and normalized ADC values could aid differentiation of skull base osteomyelitis from nasopharyngeal cancer. AJNR Am J Neuroradiol 2022;44:74–78 doi:10.3174/ajnr.A7740 pmid:36521963 
    
    [Abstract/FREE Full Text](http://www.ajnr.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpuciI7czo1OiJyZXNpZCI7czo3OiI0NC8xLzc0IjtzOjQ6ImF0b20iO3M6Mzg6Ii9ham5yL2Vhcmx5LzIwMjMvMDkvMTQvYWpuci5BNzk5Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

24. 24.Mishra V, Singh A, Chen X, et al. Application of liquid biopsy as multi-functional biomarkers in head and neck cancer. Br J Cancer 2022;126:361–70 doi:10.1038/s41416-021-01626-0 pmid:34876674
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1038/s41416-021-01626-0&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=34876674&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

25. 25.Flach S, Howarth K, Hackinger S, et al. Liquid BIOpsy for MiNimal RESidual DiSease Detection in Head and Neck Squamous Cell Carcinoma (LIONESS): a personalised circulating tumour DNA analysis in head and neck squamous cell carcinoma. Br J Cancer 2022;126:1186–95 doi:10.1038/s41416-022-01716-7 pmid:35132238
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1038/s41416-022-01716-7&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=35132238&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

26. 26.Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 2016;94:485–514 doi:10.1111/1468-0009.12210 pmid:27620683
    
    [CrossRef](http://www.ajnr.org/lookup/external-ref?access_num=10.1111/1468-0009.12210&link_type=DOI) 
    
    [PubMed](http://www.ajnr.org/lookup/external-ref?access_num=27620683&link_type=MED&atom=%2Fajnr%2Fearly%2F2023%2F09%2F14%2Fajnr.A7992.atom) 

*   Received March 31, 2023.
*   Accepted after revision August 12, 2023.


*   © 2023 by American Journal of Neuroradiology