Advertisement

A new prediction model for lateral cervical lymph node metastasis in patients with papillary thyroid carcinoma: Based on dual-energy CT

  • Author Footnotes
    1 Contributed equally as the first authors.
    Ying Zou
    Footnotes
    1 Contributed equally as the first authors.
    Affiliations
    Department of Radiology, First Central Clinical College, Tianjin Medical University, No. 24 Fu Kang Road, Nan Kai District, Tianjin 300192, China

    Department of Radiology, First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, No. 314 Anshan West Road, Nan Kai District, Tianjin 300193, China

    Department of Radiology, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion, No. 314 Anshan West Road, Nan Kai District, Tianjin 300193, China
    Search for articles by this author
  • Author Footnotes
    1 Contributed equally as the first authors.
    Shuangyan Sun
    Footnotes
    1 Contributed equally as the first authors.
    Affiliations
    Department of Radiology, First Central Clinical College, Tianjin Medical University, No. 24 Fu Kang Road, Nan Kai District, Tianjin 300192, China

    Department of Radiology, JiLin Cancer Hospital, No.1066 JinHu Road, ChaoYang District, ChangChun 130000, China
    Search for articles by this author
  • Qian Liu
    Affiliations
    Department of Radiology, The Second Hospital of Tianjin Medical University, No. 23, Pingjiang Road, Hexi District, Tianjin 300211, China
    Search for articles by this author
  • Jihua Liu
    Affiliations
    Department of Radiology, First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, No. 314 Anshan West Road, Nan Kai District, Tianjin 300193, China

    Department of Radiology, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion, No. 314 Anshan West Road, Nan Kai District, Tianjin 300193, China
    Search for articles by this author
  • Yan Shi
    Affiliations
    Department of Ultrasonography, Binzhou Medical University Hospital, No. 661 Huanghe 2nd Road, Binzhou City, Shandong 256603, China
    Search for articles by this author
  • Fang Sun
    Affiliations
    Department of Ultrasonography, Binzhou Medical University Hospital, No. 661 Huanghe 2nd Road, Binzhou City, Shandong 256603, China
    Search for articles by this author
  • Yan Gong
    Affiliations
    Department of Radiology, Tianjin Hospital of ITCWM Nan Kai Hospital, No.6 Changjiang Road, Nan Kai District, Tianjin 300100, China
    Search for articles by this author
  • Xiudi Lu
    Affiliations
    Department of Radiology, First Teaching Hospital of Tianjin University of Traditional Chinese Medicine, No. 314 Anshan West Road, Nan Kai District, Tianjin 300193, China

    Department of Radiology, National Clinical Research Center for Chinese Medicine Acupuncture and Moxibustion, No. 314 Anshan West Road, Nan Kai District, Tianjin 300193, China
    Search for articles by this author
  • Xuening Zhang
    Affiliations
    Department of Radiology, The Second Hospital of Tianjin Medical University, No. 23, Pingjiang Road, Hexi District, Tianjin 300211, China
    Search for articles by this author
  • Shuang Xia
    Correspondence
    Corresponding author.
    Affiliations
    Department of Radiology, Tianjin First Central Hospital, School of Medicine, Nankai University, No. 24 Fu Kang Road, Nan Kai District, Tianjin 300192, China
    Search for articles by this author
  • Author Footnotes
    1 Contributed equally as the first authors.
Open AccessPublished:November 21, 2021DOI:https://doi.org/10.1016/j.ejrad.2021.110060

      Highlights

      • DECT parameter could predict lateral lymph node metastasis in papillary thyroid carcinoma.
      • The new nomogram could facilitate the prediction risk with an AUC up to 0.912.
      • Individualized lateral lymph node dissection could be performed based on the nomogram.

      Abstract

      Purpose

      The current study aimed to develop and validate a prediction model to estimate the independent risk factors for lateral cervical lymph node metastasis (LLNM) in papillary thyroid carcinoma (PTC) patients based on dual-energy computed tomography (DECT).

      Method

      This study retrospectively conducted 406 consecutive patients from July 2015 to June 2019 to form the derivation cohorts and performed internal validation. 101 consecutive patients from July 2019 to June 2020 were included to create the external validation cohort. Univariable and multivariable logistic regression analyses were used to evaluate independent risk factors for LLNM. A prediction model based on DECT parameters was built and presented on a nomogram. The internal and external validations were performed.

      Results

      Iodine concentration (IC) in the arterial phase (OR 2.761, 95% CI 1.028–7.415, P 0.044), IC in venous phase (OR 3.820, 95% CI 1.430–10.209, P 0.008), located in the superior pole (OR 4.181, 95% CI 2.645–6.609, P 0.000), and extrathyroidal extension (OR 4.392, 95% CI 2.142–9.004, P 0.000) were independently associated with LLNM in the derivation cohort. These four predictors were incorporated into the nomogram. The model showed good discrimination in the derivation (AUC, 0.899), internal (AUC, 0.905), and external validation (AUC, 0.912) cohorts. The decision curve revealed that more advantages would be added using the nomogram to estimate LLNM, which implied that the lateral lymph node dissection was recommended.

      Conclusions

      DECT parameters could provide independent indicators of LLNM in PTC patients, and the nomogram based on them may be helpful in treatment decision-making.

      Graphical abstract

      Keywords

      Abbreviations:

      ATA (American Thyroid Association), DCA (Decision curve analysis), DECT (Dual-energy computed tomography), ETE (Extrathyroidal extension), IC (Iodine concentration), LLND (Lateral cervical lymph node dissection), LLNM (Lateral cervical lymph node metastasis), PTC (Papillary thyroid carcinoma), US-FNAB (Ultrasound-guided fine-needle aspiration biopsy)

      1. Introduction

      The incidence of papillary thyroid carcinoma (PTC) has dramatically increased during recent years [
      • Carlson R.W.
      The NCCN 2019 annual conference: improving the quality, effectiveness, and efficiency of cancer care.
      ], and lateral cervical lymph node metastasis (LLNM) may increase the recurrence and shorten survival [
      • McLeod D.S.
      • Sawka A.M.
      • Cooper D.S.
      Controversies in primary treatment of low-risk papillary thyroid cancer.
      ]. According to the American Thyroid Association (ATA) guidelines [
      • Haugen B.R.
      • Alexander E.K.
      • Bible K.C.
      • Doherty G.M.
      • Mandel S.J.
      • Nikiforov Y.E.
      • et al.
      2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer.
      ], therapeutic lateral cervical lymph node dissection (LLND) should be performed in patients with N1 stage, which has been proved by invasive ultrasound-guided fine-needle aspiration biopsy (US-FNAB). Therefore, it is essential to identify the presence and range of LLNM before the operation as accurately as possible using non-invasive means.
      Preoperative imaging examination plays a vital role in detecting and staging LLNM in patients with PTC [
      • Oh H.S.
      • Kwon H.
      • Song E.
      • Jeon M.J.
      • Song D.E.
      • Kim T.Y.
      • et al.
      Preoperative clinical and sonographic predictors for lateral cervical lymph node metastases in sporadic medullary thyroid carcinoma.
      ]. Ultrasound (US) is the primary examination method for the evaluation of cervical lymph nodes (LNs) according to ATA Statement on Preoperative Imaging for Thyroid Cancer Surgery [
      • Yeh M.W.
      • Bauer A.J.
      • Bernet V.A.
      • Ferris R.L.
      • Loevner L.A.
      • Mandel S.J.
      • et al.
      American Thyroid Association statement on preoperative imaging for thyroid cancer surgery.
      ], however, which is greatly affected by the operators' experience and manipulation [
      • Han Z.
      • Lei Z.
      • Li M.
      • Luo D.
      • Ding J.
      Differential diagnosis value of the ultrasound gray scale ratio for papillary thyroid microcarcinomas and micronodular goiters.
      ]. And the sensitivity for detecting lymph node metastasis (LNM) on the preoperative US is not high [
      • Lee J.Y.
      • Na D.G.
      • Yoon S.J.
      • Gwon H.Y.
      • Paik W.
      • Kim T.
      • et al.
      Ultrasound malignancy risk stratification of thyroid nodules based on the degree of hypoechogenicity and echotexture.
      ,
      • Karkada M.
      • Costa A.F.
      • Imran S.A.
      • Hart R.D.
      • Bullock M.
      • Ilie G.
      • et al.
      Incomplete Thyroid Ultrasound Reports for Patients With Thyroid Nodules: Implications Regarding Risk Assessment and Management.
      ]. The sensitivity for central cervical lymph node metastasis (CLNM) is < 50%, whereas it is approximately 70% to 80% for LLNM [
      • Khokhar M.T.
      • Day K.M.
      • Sangal R.B.
      • Ahmedli N.N.
      • Pisharodi L.R.
      • Beland M.D.
      • et al.
      Preoperative High-Resolution Ultrasound for the Assessment of Malignant Central__Compartment Lymph Nodes In Papillary Thyroid Cancer.
      ]. Meanwhile, US is also limited for evaluating LNs at lower cervical levels, such as levels VI and VII, and upper mediastina in some patients [
      • Liu X.
      • Ouyang D.
      • Li H.
      • Zhang R.
      • Lv Y.
      • Yang A.
      • et al.
      Papillary thyroid cancer: dual-energy spectral CT quantitative parameters for preoperative diagnosis of metastasis to the cervical lymph nodes.
      ]. Besides, traditional CT has similar sensitivity and specificity to that of US [
      • Eun N.L.
      • Son E.J.
      • Kim J.A.
      • Gweon H.M.
      • Kang J.H.
      • Youk J.H.
      Comparison of the diagnostic performances of ultrasonography, CT and fine needle aspiration cytology for the prediction of lymph node metastasis in patients with lymph node dissection of papillary thyroid carcinoma: A retrospective cohort study.
      ], and necrosis, cystic degeneration, calcification, and enhancement degree are essential factors for CT to judge LNM [
      • Yeom J.A.
      • Roh J.
      • Jeong Y.J.
      • Lee J.C.
      • Kim H.Y.
      • Suh Y.J.
      • et al.
      Ultra-Low-Dose Neck CT With Low-Dose Contrast Material for Preoperative Staging of Thyroid Cancer: Image Quality and Diagnostic Performance.
      ]. However, these classic signs are sporadic in the presence of small metastatic LNs [
      • Liu C.
      • Chen S.
      • Yang Y.
      • Shao D.
      • Peng W.
      • Wang Y.
      • et al.
      The value of the computer-aided diagnosis system for thyroid lesions based on computed tomography images.
      ]. DWI has a high diagnostic value for benign and metastatic LNs, with an accuracy of 91.0–94.3% [
      • Kato H.
      • Kanematsu M.
      • Kato Z.
      • Teramoto T.
      • Mizuta K.
      • Aoki M.
      • et al.
      Necrotic cervical nodes: usefulness of diffusion-weighted MR imaging in the differentiation of suppurative lymphadenitis from malignancy.
      ]. However, ADC value does not apply to all cases, such as cystic degeneration and necrosis in LNs [
      • Zhang H.
      • Zhang C.
      • Zheng Z.
      • Ye F.
      • Liu Y.
      • Zou S.
      • et al.
      Chemical shift effect predicting lymph node status in rectal cancer using high-resolution MR imaging with node-for-node matched histopathological validation.
      ], and also another disadvantage from artifact can significantly affect the detection and sensitivity in application of DWI in the neck [
      • Sun H.
      • Zhou J.
      • Liu K.
      • Shen T.
      • Wang X.
      • Wang X.
      Pancreatic neuroendocrine tumors: MR imaging features preoperatively predict lymph node metastasis.
      ,
      • Hoang J.K.
      • Vanka J.
      • Ludwig B.J.
      • Glastonbury C.M.
      Evaluation of cervical lymph nodes in head and neck cancer with CT and MRI: tips, traps, and a systematic approach.
      ,
      • Noda Y.
      • Kanematsu M.
      • Goshima S.
      • Kondo H.
      • Watanabe H.
      • Kawada H.
      • et al.
      MRI of the thyroid for differential diagnosis of benign thyroid nodules and papillary carcinomas.
      ]. Although PET-CT is helpful in the follow-up of thyroid cancer [
      • Kim S.K.
      • So Y.
      • Chung H.W.
      • Yoo Y.B.
      • Park K.S.
      • Hwang T.S.
      • et al.
      Analysis of predictability of F-18 fluorodeoxyglucose-PET/CT in the recurrence of papillary thyroid carcinoma.
      ], it does not provide any additional benefit than US and CT for the preoperative diagnosis of cervical LNM in patients with PTC to data [
      • Seo Y.L.
      • Yoon D.Y.
      • Baek S.
      • Ku Y.J.
      • Rho Y.S.
      • Chung E.J.
      • et al.
      Detection of neck recurrence in patients with differentiated thyroid cancer: comparison of ultrasound, contrast-enhanced CT and (18)F-FDG PET/CT using surgical pathology as a reference standard: (ultrasound vs. CT vs. (18)F-FDG PET/CT in recurrent thyroid cancer).
      ]. Because of its expensive examination cost, it is not widely used in clinical applications. In short, conventional image examinations have different disadvantages for the detection of LLNM in patients with PTC. Therefore, a new non-invasive imaging examination method is needed clinically to solve the cervical assessment problem. In recent years, dual-energy computed tomography (DECT) imaging features are widely used to differentiate metastatic from benign LNs in PTC patients [
      • Liu X.
      • Ouyang D.
      • Li H.
      • Zhang R.
      • Lv Y.
      • Yang A.
      • et al.
      Papillary thyroid cancer: dual-energy spectral CT quantitative parameters for preoperative diagnosis of metastasis to the cervical lymph nodes.
      ,
      • Lu W.
      • Zhong L.
      • Dong D.
      • Fang M.
      • Dai Q.
      • Leng S.
      • et al.
      Radiomic analysis for preoperative prediction of cervical lymph node metastasis in patients with papillary thyroid carcinoma.
      ]. The reason why DECT is performed in PTC patients is explained in Appendix A.
      Prediction models are becoming recognized as valuable tools in recent years and are recommended in clinical practice guidelines [
      • Collins G.S.
      • Reitsma J.B.
      • Altman D.G.
      • Moons K.G.M.
      • members of the Tg
      Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement.
      ]. At present, the prediction models have been applied to the diagnosis and antidiastole of lung cancer [
      • Muller D.C.
      • Johansson M.
      • Brennan P.
      Lung cancer risk prediction model incorporating lung function: development and validation in the UK biobank prospective cohort study.
      ], breast cancer [
      • Recht A.
      Radiation-induced heart disease after breast cancer treatment: how big a problem, and how much can-and should-we try to reduce it?.
      ], and so on [
      • Martinez-Zayas G.
      • Almeida F.A.
      • Simoff M.J.
      • Yarmus L.
      • Molina S.
      • Young B.
      • et al.
      A prediction model to help with oncologic mediastinal evaluation for radiation: HOMER.
      ,
      • Kim J.R.
      • Hwang J.Y.
      • Yoon H.M.
      • Jung A.Y.
      • Lee J.S.
      • Kim J.S.
      • et al.
      Risk estimation for biliary atresia in patients with neonatal cholestasis: development and validation of a risk score.
      ,
      • Dreizin D.
      • Bodanapally U.
      • Boscak A.
      • Tirada N.
      • Issa G.
      • Nascone J.W.
      • et al.
      CT prediction model for major arterial injury after blunt pelvic ring disruption.
      ]. In the current study, we hypothesized that the quantitative parameters from DECT have potentially been associated with LLNM in patients with PTC, and the prediction model may be the most effective strategy for evaluating LLNM. The study's purpose was first to look for independent risk factors for LLNM in patients with PTC. Second, to develop a prediction model for predicting LLNM. Third, to validate the prediction model internally and externally.

      2. Material and methods

      The methods followed the enhancing the quality and transparency of health research reporting guidelines using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement [
      • Collins G.S.
      • Reitsma J.B.
      • Altman D.G.
      • Moons K.G.
      • Group T
      Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement The TRIPOD Group.
      ].

      2.1 Patients' population

      This retrospective study was approved by the ethics committee of Tianjin First Center Hospital (2020N046KY). The requirement for written informed consent was waived of the retrospective nature.
      Five hundred eighty-five consecutive patients diagnosed with PTC by US-FNAB from July 2015 to June 2019 were initially selected. They were all performed total thyroidectomy and cervical lymph node dissection due to suspected LNM according to the preoperative US and/or DECT. Check Appendix B for the specific US and DECT diagnostic criteria of LNM in patients with PTC. The inclusion criteria were patients with PTC confirmed by postoperative pathology, with cervical lymph node dissection, and complete postoperative pathological results. We excluded patients with medullary thyroid carcinoma (MTC) and follicular thyroid carcinoma (FTC), less than 18 years old, previous thyroid operation or other neck surgery, and a history of radiation therapy. Patients who did not undergo DECT examination or with noticeable artifacts in DECT images were also excluded. Also, to ensure model development accuracy and the independence of included parameters, we only included patients with a single lesion. Inclusion and exclusion criteria were detailed in the flowchart (Fig. 1). Totally 406 consecutive patients were retrospectively enrolled, which were randomly divided into derivation and internal validation cohorts in a ratio of 7:3 by SPSS software. After the model development, the study period was extended, and an affiliate of 101 consecutive patients between July 2019, and June 2020, was selected in the external validation cohort using the same criteria. Check Appendix C for specific cohorts sampling rules. Demographics, including sex, age, final surgical pathology diagnosis was thoroughly reviewed from the medical record. The surgical methods and the principles of LNs zoning were introduced in detail in Appendix D.
      Figure thumbnail gr1
      Fig. 1Patient inclusion and exclusion flowchart for model derivation and internal validation cohorts (left) and external validation cohort (right), along with respective summaries of the study design used for each cohort. * According to ATA guidelines, patients with suspected LNM in the central and lateral cervical regions based on preoperative US and/or DECT examination would undergo total thyroidectomy and cervical lymph node dissection. # Patients who were suspected of thyroid carcinoma with LNM by US were initially selected and underwent DECT to determine the extent of LNM before operation. US, ultrasound; FNA, fine needle aspiration; MTC, medullary thyroid carcinoma; FTC, follicular thyroid carcinoma; PTC, papillary thyroid carcinoma; DECT, dual-energy computed tomography; LLNM, lateral cervical lymph node metastasis; ROC, receiver operating characteristic curve; ATA, American Thyroid Association; LNM, lymph node metastasis.

      2.2 Image acquisition

      All data was scanned using a 64 multi-detector row CT scanner (SOMATOM Definition Flash, Siemens Healthcare, Forchheim, Germany) with dual-phase contrast-enhanced CT. Patients held their breath during eupnea before the horizontal scan in a transverse position. All patients were scanned craniocaudally in the supine position with the bilateral upper limbs placed on both sides, shoulders drooping as much as possible, head slightly tilted. And the longitudinal alignment of the positioning cursor was aligned on the central sagittal plane of the cervicothoracic region. The orthotopic scanning was performed firstly, and then the scanning baseline and range were confirmed according to the scout view. The whole neck was scanned from the upper edge of the aortic arch to the lower edge of the submandibular gland, covering the thyroid and cervical LNs area.
      Keep the patient's heart rate at an average level throughout the scan. After DECT scanning, the arterial and venous phases contrast-enhanced scanning was performed. The images were acquired in the dual-energy mode by using the following parameters: tube current, 600 mA; helical thickness, 6 mm; helical pitch, 0.9; rotation speed 0.28 s; detector width 40 mm; collimation, 64 × 0.6 mm. The scan parameters were set according to the concept of as low as reasonably achievable for radiation protection. A fast rotation speed and a moderate helical pitch were chosen to obtain fast scanning speed and to reduce motion artifacts of neck and radiation dose. For contrast-enhanced scanning, an iodinated nonionic contrast agent (iohexol; 350 mg/mL iodine, SOMATOM Definition Flash, Siemens Healthcare, Forchheim, Germany) was administered through the right elbow median vein by a dual-head injector. The dosage was 1 mL/kg with a flow rate of 3 mL/s, and the total injection dose was 60–70 mL, followed by a bolus injection of 40 mL saline given at the same flow rate. The arterial phase scanning was determined by automatic trigger technique, and the scanning delay was 25 s at the beginning of arterial phase scanning. The delay time of the venous scan was 20 s after the end of the arterial scan.
      All the original DECT data were reconstructed into contiguous axial images with a section thickness of 1 mm, a field of view of 200 mm, and a matrix of 512 × 512.

      2.3 Image interpretation

      The DECT data of arterial and venous phases were transferred to SIEMENS Syngovia workstation (Syngo DE, Siemens Healthcare, Forchheim, Germany) for analysis, the Liver VNC function keys for automatic computer processing, and then the iodine maps were obtained. Specific DECT image characteristics of primary foci included as following: tumor location (left lobe, right lobe or isthmus) and position (superior-middle-inferior, ventral-middle-dorsal, interior-middle-exterior), calcification, enhancement, extrathyroidal extension (ETE), iodine concentration (IC) in the arterial and venous phases. Manual freehand delineation of a region of interest (ROI) was performed on three different adjacent slices containing the largest lesion area to measure the IC of each PTC lesion. ROI was placed in the substantial part with a room at least greater than 2 mm2, including the whole lesion (Appendix Fig. A.1). The average value from 3 measurements was taken for the final evaluation. A week later, all lesions were retest. All data were evaluated independently by two head and neck radiologists with over ten years of clinical experience blinded to all clinical information and pathological diagnosis. Intra and inter-observer consistency analyses were performed.

      2.4 Prediction model development

      Variables with a P value less than 0.05 on univariate analysis were used as candidate variables for the multivariate binary logistic regression model. The binary logistic backward stepwise regression analysis was used to select the independent predictors. Tolerance and variance inflation factors were used to evaluate the multicollinearity of the multivariate model. Before constructing the nomogram, make sure that each variable was categorical. A nomogram was created based on independent predictor variables. The nomogram's ideal cut-off value was determined using receiver operating characteristic (ROC) analysis and the peak Youden index. To quantify the nomogram's discrimination, the area under the curve (AUC) was measured. The calibration of the nomogram was evaluated by the calibration curve to assess the fit's goodness, and the Hosmer-Lemeshow test was accompanied.

      2.5 Prediction model validation

      The cross-validation method was performed for the internal validation, and the temporal validation method was used for external validation. Evaluate the model's discriminative ability through AUC, and the Hosmer-Lemeshow test was used to examine the goodness of fit of the nomogram model. In order to reduce the overfitting bias, calibration was evaluated by comparing the actual probabilities and plotting a nomogram using 1,000 bootstrap samples. Decision curve analysis (DCA) was performed to estimate the net benefits of the prediction model for the validation cohorts under different threshold probabilities [
      • Kerr K.F.
      • Brown M.D.
      • Zhu K.
      • Janes H.
      Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use.
      ]. The maximum Youden index was selected as the cut-off value to evaluate the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the prediction model.

      2.6 Statistical analysis

      All the statistical analysis was performed with SPSS (version 21.0.0.0, Chicago, IL, USA) and R software (version 4.0.1; http://www.r-project.org). R software, GraphPad prism 8.3.0, and Medcalc (version 18.2.1) were used to draw graphs. A P-value of 0.05 was considered to be statistically significant. The data distribution of each group was determined by the Kolmogorov-Smirnov test. If it fitted the normal distribution, Mean ± Standard Deviation was used to describe it, and Student's t-test was used to compare the difference between LLNM and non-LLNM. If not, Median (interquartile range, IQR) and Mann-Whitney U test were used. Categorical variables were presented as numbers (%) and analyzed using Pearson's chi-squared test or Fisher's exact test as appropriate, including sex, age, tumor position and location, calcification, enhancement, and ETE. Multivariate logistic regression was performed to estimate the odds ratio (OR) with a 95% confidence interval (CI) and identify the independent predictors for LLNM. A nomogram was constructed according to independent predictors. The nomogram's performance was evaluated using the ROC curve and calibration curve (“RMS” package). DCA was performed using the “DCA.R” (decisioncurveanalysis.org).

      3. Results

      3.1 Patient demographics

      A total of 406 consecutive patients diagnosed as PTC by postoperative pathology from July 1, 2015 to June 30, 2019 were included in this retrospective analysis. We divided the patients into two parts: approximately 70% of the cases randomly selected by SPSS software were conducted as the derivation cohort, and the remaining around 30% were used as the internal validation cohort. Ultimately, the derivation cohort consisted of 280 patients (65 males, mean age, 45.86 years ± 13.98; 215 females, mean age, 47.14 years ± 12.56), and the internal validation cohort consisted of 126 patients (27 males, mean age, 48.29 years ± 11.34; 99 females, mean age, 45.29 years ± 10.53). A total of 101 consecutive patients (33 males, mean age, 47.64 years ± 13.80; 68 females, mean age, 47.46 years ± 12.00) confirmed PTC from July 1, 2019 to June 30, 2020 were collected to form the external validation cohort. No statistically significant difference in LLNM prevalence was observed between the three cohorts. Baseline epidemiologic and clinical characteristics for the derivation and validation cohorts were shown in Table 1, Table 2. DECT quantitative parameters for the derivation, internal and external validation cohorts were shown in Table 3. There was no statistically significant difference between the three groups for any primary lesion parameter, including diameter, IC in the arterial phase, and IC in the venous phase. The result of consistency analysis was shown in Appendix Table A.1.
      Table 1Baseline Epidemiologic and Clinical Characteristics in the derivation, internal and external cohorts.
      Variable, N (%)Derivation CohortInternal Validation CohortExternal Validation Cohort
      Total

      (n = 280)
      LN (-)

      (n = 190)
      LN (+)

      (n = 90)
      PTotal

      (n = 126)
      LN (-)

      (n = 88)
      LN (+)

      (n = 38)
      PTotal

      (n = 101)
      LN (-)

      (n = 75)
      LN (+)

      (n = 26)
      P
      Sex0.0700.2570.309
       Male65 (23.2)38 (58.5)27 (41.5)27 (21.4)17 (63.0)10 (37.0)33 (32.7)23 (69.7)10 (30.3)
       Female215 (76.8)152 (70.7)63 (29.3)99 (78.6)71 (71.2)28 (28.3)68 (67.3)52 (76.5)16 (23.5)
      Age
      We divided the patients into two groups based on age using 55-year-old as cut-off value according to the 8th AJCC staging systems.
      0.0720.5260.138
       ≤ 55207 (73.9)135 (65.2)72 (34.8)85 (67.5)59 (69.4)26 (30.6)67 (66.3)47 (70.1)20 (29.9)
       >5573 (26.1)55 (75.3)18 (24.7)41 (32.5)29 (70.7)12 (29.3)34 (33.7)28 (82.4)6 (17.6)
      Tumor position (S-I)0.0000.0210.000
       Inferior109 (38.9)97 (89.0)12 (11.0)49 (38.9)39 (44.3)10 (26.3)50 (49.5)48 (64.0)2 (7.7)
       Middle89 (31.8)70 (78.7)19 (21.3)36 (28.6)27 (30.7)9 (23.7)26 (25.7)17 (22.7)9 (34.6)
       Superior82 (29.3)23 (28.0)59 (72.0)41 (32.5)22(25.0)19 (50.0)25 (24.8)10 (13.3)18 (57.7)
      ETE0.0000.0000.000
       Negative215 (76.8)179 (83.3)36 (16.7)97 (77.0)86 (88.7)11 (11.3)69 (68.3)59 (85.5)10 (14.5)
       Positive65 (23.2)11 (16.9)54 (83.1)29 (23.0)2 (6.9)27 (93.1)32 (31.7)16 (50.0)16 (50.0)
      LN, lymph node; ETE, extrathyroidal extension; AJCC, American Joint Committee on Cancer.
      # We divided the patients into two groups based on age using 55-year-old as cut-off value according to the 8th AJCC staging systems.
      Table 2Demographics of patients with PTC in the derivation cohort.
      Variable, N (%)Total

      (n = 280)
      LN (-)

      (n = 190)
      LN (+)

      (n = 90)
      P
      Tumor laterality0.443
       Left lobe136 (48.6)88 (64.7)48 (35.3)
       Right lobe130 (46.4)91 (70.0)39 (30.0)
       Isthmus14 (5.0)11 (78.6)3 (21.4)
      Tumor position (V-D)0.405
       Ventral115 (41.1)73 (63.5)42 (36.5)
       Middle128 (45.7)90 (70.3)38 (29.7)
       Dorsal37 (13.2)27 (73.0)10 (27.0)
      Tumor position (I-E)0.227
       Interior135 (48.2)85 (63.0)50 (37.0)
       Middle122 (43.6)89 (72.9)33 (27.1)
       Exterior23 (8.2)16 (69.5)7 (30.5)
      Calcification0.031
       Negative219 (78.2)175 (79.9)44 (20.1)
       Positive61 (21.8)15 (24.6)46 (75.4)
      Enhancement0.000
       Homogeneous176 (62.9)137 (77.8)39 (22.2)
       Heterogeneous104 (37.1)53 (51.0)51 (49.0)
      PTC, papillary thyroid carcinoma; LN, lymph node.
      Table 3DECT quantitative parameters in patients with PTC.
      VariableDiameter (cm)
      Mann-Whitney U test, median (IQR).
      IC IAP (mg/mL)
      Mann-Whitney U test, median (IQR).
      IC IVP (mg/mL)
      Mann-Whitney U test, median (IQR).
      Derivation cohort
       Total0.81 (0.60–1.30)2.9 (2.5–3.5)2.9 (2.4–3.3)
        LN (-)0.80 (0.57–1.20)2.7 (2.4–2.9)2.6 (2.3–2.9)
        LN (+)1.27 (0.80–1.80)2.6 (2.3–2.9)3.7 (3.2–4.2)
        P#< 0.0010.0000.000
      Internal Validation cohort
       Total0.93 (0.72–1.48)3.1 (2.6–3.6)2.9 (2.5–3.5)
        LN (-)0.76 (0.54–1.18)2.8 (2.5–3.2)2.7 (2.4–3.1)
        LN (+)1.15 (0.80–1.68)3.9 (3.4–4.7)3.9 (3.2–4.3)
        P#0.0000.0000.000
      External Validation cohort
       Total0.87 (0.68–1.39)3.2 (2.5–3.7)3.1 (2.4–3.9)
        LN (-)0.66 (0.61–1.15)2.4 (1.8–3.2)2.4 (1.9–2.8)
        LN (+)1.29 (0.84–1.42)4.1 (2.9–5.0)3.9 (2.6–4.4)
        P#0.0000.0000.000
      P#: the difference between the derivation cohort and validation cohort.
      DECT, dual-energy computed tomography; PTC, papillary thyroid carcinoma; IC, iodine concentration; IAP, in arterial phase; IVP, in venous phase; LN, lymph node; IQR, interquartile rang.
      * Mann-Whitney U test, median (IQR).

      3.2 Model development

      Univariate binary logistic analysis was performed for each variable in the derivation cohort. Age (OR, 0.967; 95% CI, 0.947–0.987; P 0.001), diameter (OR, 1.239; 95% CI, 1.097–1.398; P 0.001), IC in the arterial phase (OR, 8.655; 95% CI, 4.850–15.444; P 0.000), IC in the venous phase (OR, 9.997; 95% CI, 5.546–18.021; P 0.000), located in superior pole (OR, 4.994; 95% CI, 3.329–7.491; P 0.000), and ETE (OR, 4.752; 95% CI, 2.765–8.168; P 0.000) were statistically significantly associated with LLNM in patients with PTC (Table 4).
      Table 4Multivariate logistic regression analysis of risk factors associated with LLNM in PTC patients in the derivation cohort.
      LLNM, lateral cervical lymph node metastasis; PTC, papillary thyroid carcinoma; OR, odds ratio; CI, confidence interval; IC, iodine concentration; IAP, in the arterial phase; IVP, in the venous phase; ETE, extrathyroidal extension.
      Furthermore, a multivariate binary logistic analysis identified that IC in the arterial phase (OR 2.761, 95% CI 1.028–7.415, P 0.044), IC in venous phase (OR 3.820, 95% CI 1.430–10.209, P 0.008), located in superior pole (OR 4.181, 95% CI 2.645–6.609, P 0.000), and ETE (OR 4.392, 95% CI 2.142–9.004, P 0.000) were independent predictors of LLNM (Table 4). The variance inflation factor of each predictor was less than 10, and the corresponding tolerance was more than 0.1 (Appendix Table A.2). Therefore, no multicollinearity among these predictors was noted [
      • Dormann C.F.
      • Elith J.
      • Bacher S.
      • Buchmann C.
      • Carl G.
      • Carré G.
      • Marquéz J.R.G.
      • Gruber B.
      • Lafourcade B.
      • Leitão P.J.
      • Münkemüller T.
      • McClean C.
      • Osborne P.E.
      • Reineking B.
      • Schröder B.
      • Skidmore A.K.
      • Zurell D.
      • Lautenbach S.
      Collinearity: a review of methods to deal with it and a simulation study evaluating their performance.
      ]. A nomogram was produced by incorporating these four independent predictors (Fig. 2). The newly developed prediction model showed good discrimination with an AUC of 0.899 (95% CI, 0.857–0.931) (Table 5), and the calibration curve showed good agreement between the nomogram-estimated probability of LLNM and the actual LLNM rate in the derivation cohort, with the mean absolute error was 0.031 (Fig. 3A and 3D, Appendix Table A.3). The Hosmer-Lemeshow test demonstrated a P = 0.152, indicating no departure from a good fit.
      Figure thumbnail gr2
      Fig. 2The nomogram indicated the risk of LLNM in patients with PTC. The scores of each covariate were added. Any probability greater than 156 points (the corresponding cut-off value was 0.64) was compatible with LLNM. LLNM = lateral cervical lymph node metastasis, PTC = papillary thyroid carcinoma.
      Table 5The model performance in estimating the risk of LLNM in patients with PTC.
      ParametersDerivation ModelInternal Validation ModelExternal Validation Model
      Cut-off value0.64N/AN/A
      AUC0.899 (0.857, 0.931)0.905 (0.839, 0.950)0.912 (0.839, 0.959)
      Youden index0.59940.61180.6846
      Sensitivity (%)78.89 (69.0, 86.8)73.68 (56.9, 86.6)88.46 (69.8, 97.6)
      Specificity (%)81.05 (74.7, 86.4)87.50 (78.7, 93.6)80.00 (69.2, 88.4)
      PPV (%)66.4 (59.1, 72.9)71.8 (58.7, 82.0)60.5 (48.8, 71.1)
      NPV (%)89.0 (84.4, 92.4)88.5 (81.8, 93.0)95.2 (87.3, 98.3)
      P Value< 0.0001< 0.0001< 0.0001
      Note: Data in parentheses are 95% CIs.
      LLNM, lateral lymph nodes metastasis; PTC, papillary thyroid carcinoma; AUC, area under the curve; PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval.
      Figure thumbnail gr3
      Fig. 3A-C: ROC curves based on the nomogram for the probability of LLNM in the derivation cohort, internal validation cohort, and external validation cohort. D-F: Calibration curves of the nomogram in the derivation (D), internal (E), and external (F) validation cohorts. ROC, receiver operating characteristic curve; LLNM, lateral lymph node metastasis.

      3.3 Model internal and external validation

      The internal validation cohort showed good discrimination with an AUC of 0.905 (95% CI, 0.839, 0.950), and good discrimination with an AUC of 0.912 (95% CI, 0.839–0.959) was externally validated (Table 5). The good calibration was also confirmed in the validation cohorts, with the mean absolute errors were 0.046 and 0.035, respectively (Fig. 3B, C, E and F, Additional Table 3. Hosmer-Lemeshow test demonstrated a nonsignificant P = 0.470 and 0.685, respectively.
      In the decision curve, the x-axis was a measure of patient or physician preference, and the threshold probability indicated that the expected advantage of treatment was equal to the anticipated benefit of avoiding treatment [
      • Dormann C.F.
      • Elith J.
      • Bacher S.
      • Buchmann C.
      • Carl G.
      • Carré G.
      • Marquéz J.R.G.
      • Gruber B.
      • Lafourcade B.
      • Leitão P.J.
      • Münkemüller T.
      • McClean C.
      • Osborne P.E.
      • Reineking B.
      • Schröder B.
      • Skidmore A.K.
      • Zurell D.
      • Lautenbach S.
      Collinearity: a review of methods to deal with it and a simulation study evaluating their performance.
      ]. The decision curve revealed that if the threshold probability of a patient or physician were more significant than 3%, more advantages would be added by using the nomogram to estimate LLNM in patients with PTC (Fig. 4).
      Figure thumbnail gr4
      Fig. 4Decision curve analysis of the nomogram.
      In the derivation cohort, the cut-off value of 0.64 was selected to distinguish the presence of LLNM, and the cohort had a sensitivity of 78.89%, a specificity of 81.05%, a PPV of 66.4%, and an NPV of 89.0%. The internal validation cohort had a sensitivity of 73.68%, a specificity of 87.50%, a PPV of 71.8%, and an NPV of 88.5%. The external validation cohort had a sensitivity of 88.46%, a specificity of 80.00%, a PPV of 60.5%, and an NPV of 95.2% (Table 5).
      For clinical use, IC in the arterial phase was determined by drawing a line straight up to the point axis to establish the score associated with the IC in the arterial phase. Next, this process was repeated for the other three covariates (IC in the venous phase, Superior, and ETE). The scores of each covariate were added, and the total score was located on the total score points axis. Last, a line was drawn straight down to the risk of the LLNM axis to obtain the probability. Each DECT-derived measurement had corresponding value (points) that appear in upper toolbar as following: “IC IAP > 3.3 mg/mL” = 41 points, “IC IVP > 3.1 mg/mL” = 46 points, ETE = 51 points, Middle = 25 points, and Superior = 100 points. A summarized total was applied on the bottom scale to obtain the probability of LLNM. Any probability greater than 0.64 (about 156 points) was compatible with LLNM (Appendix Fig. A.2). Two sample cases of the diagnostic use of the nomogram were given in Fig. 5, Fig. 6.
      Figure thumbnail gr5
      Fig. 5An example of using the nomogram to predict the individual risk of LLNM by manually placing straight lines across the diagram. A-D, A 50-year-old male with PTC located on the dorsal side of the middle part of the right lobe of the thyroid (yellow arrow, A, B, axial and coronal images in the arterial phase; C, axial image in the venous phase). The density of the lesion was uneven, with calcification, and broke through the thyroid capsule. And IC in the arterial phase was 4.2 mg/ml, IC in the venous phase was 5.0 mg/ml. Vertical lines of each variable were drawn*. The values on the “Points” scale intersected by the lines were added to obtain total points (41 + 46 + 51 + 25 = 163). The graph revealed that the risk of LLNM was close to 70% by drawing a vertical line on the “Total points” scale. Postoperative pathology proved the positive LLNM in level Ⅳa. Image D showed that a metastatic lymph node in the right level Ⅳa in the iodine map. LLNM, lymph node metastasis; IC, iodine concentration; IAP, in the arterial phase; IVP, in the venous phase; ETE, extrathyroidal extension; SI, Superior-Middle-Inferior; PTC, papillary thyroid carcinoma. * IC IAP: blue line, IC IVP: orange line, ETE: green line, SI: purple line.
      Figure thumbnail gr6
      Fig. 6A-C, a 75-year-old female with PTC (located on the dorsal side of the middle of the right lobe of the thyroid (yellow arrow, axial, coronal, and sagittal images in the arterial phase; D, the axial image in the venous phase. The density of the lesion was uneven and broke through the thyroid capsule. And IC in the arterial phase was 3.7 mg/ml, IC in the venous phase was 2.5 mg/ml. Vertical lines of each variable were drawn*. The values on the “Points” scale intersected by the lines were added to obtain total points (41 + 0 + 51 + 25 = 117). The graph revealed that the risk of LLNM was<40% by drawing a vertical line on the “Total points” scale. Postoperative pathological examination validated no LLNM. PTC, papillary thyroid carcinoma; IC, iodine concentration; LLNM, lateral cervical lymph node metastasis; * IC IAP: blue line, IC IVP: orange line, ETE: green line, SI: purple line.

      4. Discussion

      Our study developed a prediction model based on DECT parameters and a nomogram to estimate the LLNM in patients with PTC. There were three significant findings. First, four parameters of the primary lesion from DECT, including IC in the arterial phase, IC in the venous phase, located in the superior pole, and ETE, were independent predictors of LLNM. Second, the new nomogram could facilitate the prediction of the risk of LLNM by using a cut-off value of 0.64 (about 156 points), with an AUC of 0.899. Third, the internal and external validations revealed the nomogram's high quality of discrimination and calibration abilities.
      Nomograms were the visualization of statistical models specifically developed to optimize individuals' predictive accuracy [
      • Thompson A.M.
      • Turner R.M.
      • Hayen A.
      • Aniss A.
      • Jalaty S.
      • Learoyd D.L.
      • Sidhu S.
      • Delbridge L.
      • Yeh M.W.
      • Clifton-Bligh R.
      • Sywak M.
      A preoperative nomogram for the prediction of ipsilateral central compartment lymph node metastases in papillary thyroid cancer.
      ]. Some scholars [
      • Lu W.
      • Zhong L.
      • Dong D.
      • Fang M.
      • Dai Q.
      • Leng S.
      • et al.
      Radiomic analysis for preoperative prediction of cervical lymph node metastasis in patients with papillary thyroid carcinoma.
      ,
      • Jiang M.
      • Li C.
      • Tang S.
      • Lv W.
      • Yi A.
      • Wang B.
      • Yu S.
      • Cui X.
      • Dietrich C.F.
      Nomogram based on shear-wave elastography radiomics can improve preoperative cervical lymph node staging for papillary thyroid carcinoma.
      ] had constructed a nomogram based on US radiomics to predict cervical LNM, and AUC was from 0.807 to 0.867. In our prediction model, AUC was 0.899 in the derivation cohort. Therefore, the prediction model was adequate for base clinical decisions. That is to say, preoperative nomograms estimating LLNM can aid clinicians in identifying patients who may derive more significant clinical benefit from more extensive surgery.
      To facilitate clinical application, all variables were converted to categorical variables. The risk of LLNM was always contributed to by these four predictors of the primary lesion from DECT. The individual risk of LLNM could be conveniently estimated by the nomogram after the DECT examination. For a low risk of LLNM (the total points ≤ 156 points), the nomogram indicated that central cervical lymph node dissection was adequate. Inversely, for a high risk of LLNM (the total points > 156 points), additional LLND might be needed. Consideration of specific cases may help illustrate these concepts, and two instances given in the result can support this. Both the two patients had central LNM, located in the middle of the thyroid, and with ETE, which at first glance, the difference between them seemed small. However, after DECT examination, it was found that IC in the arterial and venous phases was different. Considering that the occurrence probability of LLNM varied considerably according to our predictive nomogram, which was of great value for clinical preoperative decision-making.
      To our knowledge, the predictive nomogram was the first internal and externally validated model to predict LLNM in patients with PTC. It incorporated DECT imaging and had more advantages than the nomogram based on the traditional US.
      In the previous studies [
      • Liu X.
      • Ouyang D.
      • Li H.
      • Zhang R.
      • Lv Y.
      • Yang A.
      • et al.
      Papillary thyroid cancer: dual-energy spectral CT quantitative parameters for preoperative diagnosis of metastasis to the cervical lymph nodes.
      ,
      • Zhao Y.
      • Li X.
      • Li L.
      • Wang X.
      • Lin M.
      • Zhao X.
      • et al.
      Preliminary study on the diagnostic value of single-source dual-energy CT in diagnosing cervical lymph node metastasis of thyroid carcinoma.
      ], DECT had been used to quantitatively assess cervical LNM in patients with PTC because IC was a direct response to blood flow [
      • Tawfik A.M.
      • Michael Bucher A.
      • Vogl T.J.
      Dual-energy computed tomography applications for the evaluation of cervical lymphadenopathy.
      ]. A similar result was obtained in our study. IC in the arterial and venous phases were independent risk factors for LLNM in patients with PTC, with the cut-off value of 3.3 mg/mL and 3.1 mg/mL, respectively. Meanwhile, located in the superior pole and ETE were also independent predictors for LLNM, consistent with previous studies [
      • Sun R.H.
      • Li C.
      • Zhou Y.Q.
      • Cai Y.C.
      • Shui C.Y.
      • Liu W.
      • et al.
      Predictive role of intraoperative clinicopathological features of the central compartment in estimating lymph nodes metastasis status.
      ,
      • Heng Y.
      • Yang Z.
      • Zhou L.
      • Lin J.
      • Cai W.
      • Tao L.
      Risk stratification for lateral involvement in papillary thyroid carcinoma patients with central lymph node metastasis.
      ,
      • Back K.
      • Kim J.S.
      • Kim J.H.
      • Choe J.H.
      Superior located papillary thyroid microcarcinoma is a risk factor for lateral lymph node metastasis.
      ,
      • Siddiqui S.
      • White M.G.
      • Antic T.
      • Grogan R.H.
      • Angelos P.
      • Kaplan E.L.
      • et al.
      Clinical and pathologic predictors of lymph node metastasis and recurrence in papillary thyroid microcarcinoma.
      ,
      • Kim K.
      • Zheng X.
      • Kim J.K.
      • Lee C.R.
      • Kang S.W.
      • Lee J.
      • et al.
      The contributing factors for lateral neck lymph node metastasis in papillary thyroid microcarcinoma (PTMC).
      ].
      There were some limitations in the current study. First, because it was a retrospective study, it might result in a potential selection bias. Thus, a multicenter larger sample size prospective clinical research was required to confirm the model we developed. Second, to ensure model development accuracy and the independence of included parameters, we only included patients with a single lesion. Therefore, in the next study, we should enroll patients with multiple lesions.

      5. Conclusions

      The nomogram in this study is the first prediction model for LLNM based on DECT in patients with PTC, distinguishing PTC patients with a genuinely high risk of LLNM, and more tailored surgical interventions could be performed. This simple and easily applicable nomogram for the diagnosis of LLNM showed good discrimination and calibration abilities. These four primary lesion variables can be obtained noninvasively. Therefore, the prediction model can help clinicians quickly and confidently determine further clinical management plans and avoid unnecessary invasive procedures.

       CRediT authorship contribution statement

      Ying Zou: Conceptualization, Formal analysis, Writing – original draft, Writing – review & editing. Shuangyan Sun: Data curation. Qian Liu: Investigation. Jihua Liu: Visualization. Yan Shi: Validation. Fang Sun: Validation. Yan Gong: Data curation. Xiudi Lu: Data curation. Xuening Zhang: Conceptualization, Writing – review & editing. Shuang Xia: Conceptualization, Funding acquisition, Writing – review & editing.

      Declaration of Competing Interest

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      Acknowledgments

      The authors thank Jianhua Gu, MD, of Department of General Surgery, Tianjin First Central Hospital for patient recruitment and guidance of clinical work; Wen Shen, MD, of Department of Radiology, Tianjin First Central Hospital for image acquisition; Xi Zhao, Senior engineer, of Siemens for his support for dual-energy CT image post-processing.

      Appendix A. Supplementary material

      The following are the Supplementary data to this article:

      References

        • Carlson R.W.
        The NCCN 2019 annual conference: improving the quality, effectiveness, and efficiency of cancer care.
        J. Natl. Comprehensive Cancer Network: JNCCN. 2019; 17 (Epub 2019/05/23): 529-530https://doi.org/10.6004/jnccn.2019.5001
        • McLeod D.S.
        • Sawka A.M.
        • Cooper D.S.
        Controversies in primary treatment of low-risk papillary thyroid cancer.
        Lancet (London, England). 2013; 381 (Epub 2013/05/15): 1046-1057https://doi.org/10.1016/s0140-6736(12)62205-3
        • Haugen B.R.
        • Alexander E.K.
        • Bible K.C.
        • Doherty G.M.
        • Mandel S.J.
        • Nikiforov Y.E.
        • et al.
        2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer.
        Thyroid: Off. J. Am. Thyroid Assoc. 2016; 26 (Epub 2015/10/16): 1-133https://doi.org/10.1089/thy.2015.0020
        • Oh H.S.
        • Kwon H.
        • Song E.
        • Jeon M.J.
        • Song D.E.
        • Kim T.Y.
        • et al.
        Preoperative clinical and sonographic predictors for lateral cervical lymph node metastases in sporadic medullary thyroid carcinoma.
        Thyroid: Off. J. Am. Thyroid Assoc. 2018; 28 (Epub 2018/01/20): 362-368https://doi.org/10.1089/thy.2017.0514
        • Yeh M.W.
        • Bauer A.J.
        • Bernet V.A.
        • Ferris R.L.
        • Loevner L.A.
        • Mandel S.J.
        • et al.
        American Thyroid Association statement on preoperative imaging for thyroid cancer surgery.
        Thyroid: Off. J. Am. Thyroid Assoc. 2015; 25 (Epub 2014/09/05): 3-14https://doi.org/10.1089/thy.2014.0096
        • Han Z.
        • Lei Z.
        • Li M.
        • Luo D.
        • Ding J.
        Differential diagnosis value of the ultrasound gray scale ratio for papillary thyroid microcarcinomas and micronodular goiters.
        Quant. Imaging Med. Surgery. 2018; 8 (Epub 2018/07/28): 507-513https://doi.org/10.21037/qims.2018.06.04
        • Lee J.Y.
        • Na D.G.
        • Yoon S.J.
        • Gwon H.Y.
        • Paik W.
        • Kim T.
        • et al.
        Ultrasound malignancy risk stratification of thyroid nodules based on the degree of hypoechogenicity and echotexture.
        Eur. Radiol. 2020; 30 (Epub 2019/11/17): 1653-1663https://doi.org/10.1007/s00330-019-06527-8
        • Karkada M.
        • Costa A.F.
        • Imran S.A.
        • Hart R.D.
        • Bullock M.
        • Ilie G.
        • et al.
        Incomplete Thyroid Ultrasound Reports for Patients With Thyroid Nodules: Implications Regarding Risk Assessment and Management.
        AJR Am. J. Roentgenol. 2018; 211 (Epub 2018/10/18): 1348-1353https://doi.org/10.2214/AJR.18.20056
        • Khokhar M.T.
        • Day K.M.
        • Sangal R.B.
        • Ahmedli N.N.
        • Pisharodi L.R.
        • Beland M.D.
        • et al.
        Preoperative High-Resolution Ultrasound for the Assessment of Malignant Central__Compartment Lymph Nodes In Papillary Thyroid Cancer.
        Thyroid: Off. J. Am. Thyroid Assoc. 2015; 25: 1351-1354https://doi.org/10.1089/thy.2015.0176)
        • Liu X.
        • Ouyang D.
        • Li H.
        • Zhang R.
        • Lv Y.
        • Yang A.
        • et al.
        Papillary thyroid cancer: dual-energy spectral CT quantitative parameters for preoperative diagnosis of metastasis to the cervical lymph nodes.
        Radiology. 2015; 275 (Epub 2014/12/19): 167-176https://doi.org/10.1148/radiol.14140481
        • Eun N.L.
        • Son E.J.
        • Kim J.A.
        • Gweon H.M.
        • Kang J.H.
        • Youk J.H.
        Comparison of the diagnostic performances of ultrasonography, CT and fine needle aspiration cytology for the prediction of lymph node metastasis in patients with lymph node dissection of papillary thyroid carcinoma: A retrospective cohort study.
        Int. J. Surg. 2018; 51 (Epub 2018/01/18): 145-150https://doi.org/10.1016/j.ijsu.2017.12.036
        • Yeom J.A.
        • Roh J.
        • Jeong Y.J.
        • Lee J.C.
        • Kim H.Y.
        • Suh Y.J.
        • et al.
        Ultra-Low-Dose Neck CT With Low-Dose Contrast Material for Preoperative Staging of Thyroid Cancer: Image Quality and Diagnostic Performance.
        AJR Am. J. Roentgenol. 2019; 212 (Epub 2019/03/23): 748-754https://doi.org/10.2214/AJR.18.20334
        • Liu C.
        • Chen S.
        • Yang Y.
        • Shao D.
        • Peng W.
        • Wang Y.
        • et al.
        The value of the computer-aided diagnosis system for thyroid lesions based on computed tomography images.
        Quant. imaging Med. Surgery. 2019; 9: 642-653https://doi.org/10.21037/qims.2019.04.01
        • Kato H.
        • Kanematsu M.
        • Kato Z.
        • Teramoto T.
        • Mizuta K.
        • Aoki M.
        • et al.
        Necrotic cervical nodes: usefulness of diffusion-weighted MR imaging in the differentiation of suppurative lymphadenitis from malignancy.
        Eur. J. Radiol. 2013; 82 (Epub 2012/09/08): e28-e35https://doi.org/10.1016/j.ejrad.2012.08.014
        • Zhang H.
        • Zhang C.
        • Zheng Z.
        • Ye F.
        • Liu Y.
        • Zou S.
        • et al.
        Chemical shift effect predicting lymph node status in rectal cancer using high-resolution MR imaging with node-for-node matched histopathological validation.
        Eur. Radiol. 2017; 27 (Epub 2017/02/09): 3845-3855https://doi.org/10.1007/s00330-017-4738-7
        • Sun H.
        • Zhou J.
        • Liu K.
        • Shen T.
        • Wang X.
        • Wang X.
        Pancreatic neuroendocrine tumors: MR imaging features preoperatively predict lymph node metastasis.
        Abdominal Radiol. (New York). 2018; (Epub 2018/12/13)https://doi.org/10.1007/s00261-018-1863-y
        • Hoang J.K.
        • Vanka J.
        • Ludwig B.J.
        • Glastonbury C.M.
        Evaluation of cervical lymph nodes in head and neck cancer with CT and MRI: tips, traps, and a systematic approach.
        AJR Am. J. Roentgenol. 2013; 200 (Epub 2012/12/21): W17-W25https://doi.org/10.2214/AJR.12.8960
        • Noda Y.
        • Kanematsu M.
        • Goshima S.
        • Kondo H.
        • Watanabe H.
        • Kawada H.
        • et al.
        MRI of the thyroid for differential diagnosis of benign thyroid nodules and papillary carcinomas.
        AJR Am. J. Roentgenol. 2015; 204 (Epub 2015/02/26): W332-W335https://doi.org/10.2214/AJR.14.13344
        • Kim S.K.
        • So Y.
        • Chung H.W.
        • Yoo Y.B.
        • Park K.S.
        • Hwang T.S.
        • et al.
        Analysis of predictability of F-18 fluorodeoxyglucose-PET/CT in the recurrence of papillary thyroid carcinoma.
        Cancer Med. 2016; 5 (Epub 2016/10/30): 2756-2762https://doi.org/10.1002/cam4.867
        • Seo Y.L.
        • Yoon D.Y.
        • Baek S.
        • Ku Y.J.
        • Rho Y.S.
        • Chung E.J.
        • et al.
        Detection of neck recurrence in patients with differentiated thyroid cancer: comparison of ultrasound, contrast-enhanced CT and (18)F-FDG PET/CT using surgical pathology as a reference standard: (ultrasound vs. CT vs. (18)F-FDG PET/CT in recurrent thyroid cancer).
        Eur. Radiol. 2012; 22 (Epub 2012/05/09): 2246-2254https://doi.org/10.1007/s00330-012-2470-x
        • Lu W.
        • Zhong L.
        • Dong D.
        • Fang M.
        • Dai Q.
        • Leng S.
        • et al.
        Radiomic analysis for preoperative prediction of cervical lymph node metastasis in patients with papillary thyroid carcinoma.
        Eur. J. Radiol. 2019; 118 (Epub 2019/08/24): 231-238https://doi.org/10.1016/j.ejrad.2019.07.018
        • Collins G.S.
        • Reitsma J.B.
        • Altman D.G.
        • Moons K.G.M.
        • members of the Tg
        Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement.
        Eur. Urol. 2015; 67 (Epub 2015/01/13): 1142-1151https://doi.org/10.1016/j.eururo.2014.11.025
        • Muller D.C.
        • Johansson M.
        • Brennan P.
        Lung cancer risk prediction model incorporating lung function: development and validation in the UK biobank prospective cohort study.
        J. Clin. Oncol.: Off. J. Am. Soc. Clin. Oncol. 2017; 35 (Epub 2017/01/18): 861-869https://doi.org/10.1200/JCO.2016.69.2467
        • Recht A.
        Radiation-induced heart disease after breast cancer treatment: how big a problem, and how much can-and should-we try to reduce it?.
        J. Clin. Oncol.: Off. J. Am. Soc. Clin. Oncol. 2017; 35 (Epub 2017/01/18): 1146-1148https://doi.org/10.1200/JCO.2016.71.4113
        • Martinez-Zayas G.
        • Almeida F.A.
        • Simoff M.J.
        • Yarmus L.
        • Molina S.
        • Young B.
        • et al.
        A prediction model to help with oncologic mediastinal evaluation for radiation: HOMER.
        Am. J. Respir. Crit. Care Med. 2020; 201 (Epub 2019/10/02): 212-223https://doi.org/10.1164/rccm.201904-0831OC
        • Kim J.R.
        • Hwang J.Y.
        • Yoon H.M.
        • Jung A.Y.
        • Lee J.S.
        • Kim J.S.
        • et al.
        Risk estimation for biliary atresia in patients with neonatal cholestasis: development and validation of a risk score.
        Radiology. 2018; 288 (Epub 2018/04/11): 262-269https://doi.org/10.1148/radiol.2018172390
        • Dreizin D.
        • Bodanapally U.
        • Boscak A.
        • Tirada N.
        • Issa G.
        • Nascone J.W.
        • et al.
        CT prediction model for major arterial injury after blunt pelvic ring disruption.
        Radiology. 2018; 287 (Epub 2018/03/21): 1061-1069https://doi.org/10.1148/radiol.2018170997
        • Collins G.S.
        • Reitsma J.B.
        • Altman D.G.
        • Moons K.G.
        • Group T
        Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement The TRIPOD Group.
        Circulation. 2015; 131 (Epub 2015/01/07): 211-219https://doi.org/10.1161/CIRCULATIONAHA.114.014508
        • Kerr K.F.
        • Brown M.D.
        • Zhu K.
        • Janes H.
        Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use.
        J. Clin. Oncol.: Off. J. Am. Soc. Clin. Oncol. 2016; 34 (Epub 2016/06/02): 2534-2540https://doi.org/10.1200/JCO.2015.65.5654
        • Dormann C.F.
        • Elith J.
        • Bacher S.
        • Buchmann C.
        • Carl G.
        • Carré G.
        • Marquéz J.R.G.
        • Gruber B.
        • Lafourcade B.
        • Leitão P.J.
        • Münkemüller T.
        • McClean C.
        • Osborne P.E.
        • Reineking B.
        • Schröder B.
        • Skidmore A.K.
        • Zurell D.
        • Lautenbach S.
        Collinearity: a review of methods to deal with it and a simulation study evaluating their performance.
        Ecography. 2013; 36: 27-46https://doi.org/10.1111/j.1600-0587.2012.07348.x
        • Thompson A.M.
        • Turner R.M.
        • Hayen A.
        • Aniss A.
        • Jalaty S.
        • Learoyd D.L.
        • Sidhu S.
        • Delbridge L.
        • Yeh M.W.
        • Clifton-Bligh R.
        • Sywak M.
        A preoperative nomogram for the prediction of ipsilateral central compartment lymph node metastases in papillary thyroid cancer.
        Thyroid: Off. J. Am. Thyroid Assoc. 2014; 24: 675-682https://doi.org/10.1089/thy.2013.0224
        • Jiang M.
        • Li C.
        • Tang S.
        • Lv W.
        • Yi A.
        • Wang B.
        • Yu S.
        • Cui X.
        • Dietrich C.F.
        Nomogram based on shear-wave elastography radiomics can improve preoperative cervical lymph node staging for papillary thyroid carcinoma.
        Thyroid: Off. J. Am. Thyroid Assoc. 2020; 30: 885-897https://doi.org/10.1089/thy.2019.0780
        • Zhao Y.
        • Li X.
        • Li L.
        • Wang X.
        • Lin M.
        • Zhao X.
        • et al.
        Preliminary study on the diagnostic value of single-source dual-energy CT in diagnosing cervical lymph node metastasis of thyroid carcinoma.
        J. Thoracic Dis. 2017; 9 (Epub 2017/12/23): 4758-4766https://doi.org/10.21037/jtd.2017.09.151
        • Tawfik A.M.
        • Michael Bucher A.
        • Vogl T.J.
        Dual-energy computed tomography applications for the evaluation of cervical lymphadenopathy.
        Neuroimaging Clin. North America. 2017; 27 (Epub 2017/07/18): 461-468https://doi.org/10.1016/j.nic.2017.04.002
        • Sun R.H.
        • Li C.
        • Zhou Y.Q.
        • Cai Y.C.
        • Shui C.Y.
        • Liu W.
        • et al.
        Predictive role of intraoperative clinicopathological features of the central compartment in estimating lymph nodes metastasis status.
        Ann. Translat. Med. 2019; 7 (Epub 2019/11/09): 471https://doi.org/10.21037/atm.2019.08.01
        • Heng Y.
        • Yang Z.
        • Zhou L.
        • Lin J.
        • Cai W.
        • Tao L.
        Risk stratification for lateral involvement in papillary thyroid carcinoma patients with central lymph node metastasis.
        Endocrine. 2020; (Epub 2020/01/27)https://doi.org/10.1007/s12020-020-02194-8
        • Back K.
        • Kim J.S.
        • Kim J.H.
        • Choe J.H.
        Superior located papillary thyroid microcarcinoma is a risk factor for lateral lymph node metastasis.
        Ann. Surgical Oncol. 2019; 26 (Epub 2019/07/03): 3992-4001https://doi.org/10.1245/s10434-019-07587-2
        • Siddiqui S.
        • White M.G.
        • Antic T.
        • Grogan R.H.
        • Angelos P.
        • Kaplan E.L.
        • et al.
        Clinical and pathologic predictors of lymph node metastasis and recurrence in papillary thyroid microcarcinoma.
        Thyroid: Off. J. Am. Thyroid Assoc. 2016; 26 (Epub 2016/04/28): 807-815https://doi.org/10.1089/thy.2015.0429
        • Kim K.
        • Zheng X.
        • Kim J.K.
        • Lee C.R.
        • Kang S.W.
        • Lee J.
        • et al.
        The contributing factors for lateral neck lymph node metastasis in papillary thyroid microcarcinoma (PTMC).
        Endocrine. 2020; (Epub 2020/03/09)https://doi.org/10.1007/s12020-020-02251-2