Applicability and reproducibility of the CPAT-grading system for pancreas allograft thrombosis

Purpose: : Although pancreas allograft thrombosis (PAT) incidence has progressively decreased, it remains the most common cause of early graft failure. Currently, there is no consensus on documentation of PAT, which has resulted in a great variability in reporting. The Cambridge Pancreas Allograft Thrombosis (CPAT) grading system has recently been developed for classification of PAT. In this study we aimed to assess the applicability and validate the reproducibility of the CPAT grading system. Methods: : This study is a retrospective cohort study. Selected for this study were all 177 pancreas transplantations performed at our center between January 1 st, 2008 and September 1 st, 2018 were included. Results: : A total of 318 Computed Tomography (CT) images was reevaluated according the CPAT system by two local radiologists. Inter-rater agreement expressed in Cohen’s kappa was 0.403 for arterial and 0.537 for venous thrombosis. Inter-rater agreement, expressed in the Fleiss’ kappa, within clinically relevant thrombosis categories was 0.626 for Grade 2 and 0.781 for Grade 3 venous thrombosis. Conclusions: : Although not perfect, we believe that implementation of the CPAT system would improve current documentation on PAT. However, it is questionable whether identification of a small Grade 1 thrombosis would be relevant in clinical practice. Furthermore, a good quality CT scan, including adequate phasing, is essential to accurately identify potential thrombus and extend after pancreas transplantation.


Introduction
Pancreas transplantation has been proven to be a successful treatment for patients with insulin-dependent diabetes mellitus. Survival rates of pancreas transplantation have further improved over the past decades. A recent study showed 1-and 2-years patient survival rates of almost 100 % and 1-and 2-year graft survival rates over 80 % in deceased-after-brain-death (DBD) organs and over 90 % in deceasedafter-circulatory-death (DCD) organs [1]. Technical failure is the leading cause for early graft loss in pancreas transplantation [2][3][4][5][6]. Pancreas allograft thrombosis (PAT), which is the most frequent surgical complication and responsible for 29 % of all early graft loss, remains an unsolved problem and usually results in graft loss [7,8]. Although incidence has progressively decreased, pancreas allograft thrombosis is still reported to develop in 3-34 % of all transplanted patients [9][10][11][12]. Complete thrombosis is likely to result in graft loss, although cases of successful salvage of the graft by performing a thrombectomy have been reported [8,13]. In contrast to complete thrombosis, little is known about partial thrombosis. Partial thrombosis is presumably underreported and develops and occurs more frequently than complete thrombosis [13]. Different interventions and strategies for detection and treatment of partial thrombosis have been described in literature [8,13, Abbreviations: CIT, cold ischemia time; CPAT, Cambridge Pancreas Allograft Thrombosis; DBD, donor after brain death; DCD, donor after cardiac death; LMWH, low molecular weight heparin; PACS, picture archive and communication system; PAK, pancreas after kidney; PAT, pancreas allograft thrombosis; PTA, pancreas transplant alone; SA, splenic artery; SD, standard deviation; SMA, superior mesenteric artery; SMV, superior mesenteric vein; SV, splenic vein; SPK, simultaneous pancreas kidney. 14]. However, convincing evidence on the efficacy of these interventions is scarce. This lack of evidence may be explained by the limited knowledge about the extent of knowledge of partial thrombosis.
In 2017, Hakeem et al. [15] conducted a study on pancreas thrombosis. They proposed the Cambridge Pancreas Allograft Thrombosis (CPAT) grading system (Table 1.) for classification of pancreas allograft thrombosis. The system was developed to enhance reporting of thrombosis and to improve comparison of reports from different centers for a better understanding and management. The CPAT grading system differentiates 4 grades of thrombosis: no thrombosis (Grade 0), peripheral thrombosis (Grade 1), intermediate non-occlusive thrombosis (Grade 2) and central occlusive thrombosis (Grade 3).
Currently, there are no solid guidelines for documentation on pancreas allograft thrombosis, especially on partial allograft thrombosis. This has resulted in great variability in reporting of thrombosis. Verification of these reports led to the conclusion that the reports are not suitable for analysis or comparison. Thus, implementation of a grading system, such as the CPAT grading system would be eligible. However, the applicability of the CPAT grading system has not been described by others so far. This study aims to assess the applicability of the CPAT grading system and validate its reproducibility in a different cohort.

Study population and design
This study is a retrospective cohort study of all pancreas transplantations (SPK, pancreas transplant alone (PTA), pancreas after kidney (PAK) and retransplantations) performed at the Leiden University Medical Center (LUMC) between January 1st of 2008 and September 1st of 2018. Data regarding recipient and transplant characteristics and the original radiology report were retrieved from the patient charts. Inclusion criterion for the study was the availability of an abdominal Computed Tomography (CT)-scan, performed within the first 3 months after transplantation. Follow-up time was 3 months after transplantation.

Pancreas transplantation
In our center, arterial revascularization of the pancreas allograft is usually performed using the donor common, external and internal iliac arteries. The donor iliac arteries are converted into a Y-graft conduit and are anastomosed to the splenic artery (SA) and superior mesenteric artery (SMA) of the allograft. Less frequently, when the donor SMA and celiac trunk remained with the pancreas graft including a common aortic patch, this was used without arterial reconstruction. For systemic venous revascularization, the donor portal vein was anastomosed endto-side onto the recipients inferior caval vein. Current standard immunosuppressive protocol, consists of a alemtuzumab induction and prednisone, followed by maintenance therapy with prednisone, tacrolimus and mycophenolate mofetil. Prednisone was added to the maintenance therapy in 2016. All transplant patients receive standard anticoagulative prophylaxis after transplantation with 1 or 2 daily doses 2850 IE of low-molecular-weight-heparin (LMWH). In our center, therapeutic anticoagulative treatment of partial thrombosis consists of systemic anticoagulation, either intravenous heparin or subcutaneous LMWH, followed by administration of vitamin K antagonists (VKA) for at least 3 months [10].

CT imaging
Each pancreas recipient received a routine abdominal CT scan within the first week postoperatively as part of the local pancreas-transplant protocol. In case of impaired kidney function, imaging was performed later. All routine abdominal CT scans were selected for this study. In addition, CT scans performed within 3 months after transplantation with specific request for evaluation of the vascular status of the graft, were included. All CT scans were performed on a Toshiba Aquilion or Toshiba Aquilion One 320-slice CT scanner with the following parameters: routine section thickness 1.0 mm; section thickness after reconstruction, 1.0-5.0 mm; filtered back projection reconstruction method; 120 kV; Automatic Exposure Control. Standard post-transplant protocol in our institute constituted of a two-phase contrast-enhanced CT scan of the abdomen, including an early-arterial phase of the lower abdomen (including the pancreas transplant) and a parenchymal phase of the entire abdomen. The two phases were respectively derived with a sure start in the abdominal aorta and a 35 s delay. Non-ionic iobitrol 350 mg I/mL was infused with a dose of 1.4 mL/kg and a flow of 0.05 mL/kg/s for 30 s. All scans received a unique study number and were reconstructed anonymously in the picture archive and communication system (PACS).

CT analysis
Reevaluation of the 318 CT scans was performed independently by one radiologist and one senior resident specialized in abdominal imaging (JK and BK) with respectively 7 and 4 years of experience. Reviewers were blinded to each other's results as well as to the original report and the clinical patient characteristics. The CT findings were classified according to the CPAT grading system developed by Hakeem et al. [15]. Additional clarification on the classification was received from the research team which conducted the original study. Based on the additional clarification, our reviewers classified non-occlusive thrombosis in the portal vein and the Y-graft as a Grade 2 thrombosis. Original drawings demonstrating the limits of each specific grade of the CPAT system were provided (Supplement 1). Our reviewers practiced the use and implementation of the CPAT system together on a training set of approximately 40 CT scans from pancreas transplants performed in 2006 and 2007, directly prior to onset of the study. Arterial and venous blood supply of the pancreas graft were evaluated separately, the highest grade of thrombosis was reported, whether this concerned only one of both arteries of veins. The reviewers also reported their opinion on the quality of both the arterial and venous phase of the CT scan (good, moderate, poor). Quality of the scans was primarily defined by vascular enhancement. Reviewers also looked at the presence of artifacts.

Statistics
Descriptive analyses were performed on patient demographics and frequencies of CPAT scores. The applicability of the CPAT grading system was assessed by the inter-rater agreement. The inter-rater agreement was evaluated using the Cohen's kappa and the Fleiss' kappa. The strength of agreement was interpreted using guidelines established by Landis and Koch [16]. According to Landis and Koch, κ values < 0 have a poor strength of agreement, κ values of 0− 0.2 a slight strength of

Results
Between January 1st 2008 and September 1st 2018, 182 patients underwent a pancreas transplantation in our center. Eligible for inclusion were 173 transplant patients corresponding with 177 pancreas transplantations (four cases of re-transplantation) and a total of 318 CT scans. Five patients were excluded from the study because no CT scan was performed within the first three months after transplantation. The study population included 76 females and 101 males (42.9 % vs 57.1 %) with a mean age of 43.0 (SD = 8.23, range 23− 64) at time of transplantation. Patient characteristics are displayed in Table 2. In this cohort, 84.2 % (N = 149) received a SPK transplant, 14.1 % (N = 25) a PAK and 1.7 % (N = 3) a PTA. Four patients suffered from a graft loss and were retransplanted within the study's period (2.3 %). Arterial reconstruction, using a Y-graft, had been performed in 76.3 % (N = 135) of all transplantations. In 22.6 % (N = 40) no arterial reconstruction had been performed and the donor's SMA and celiac trunk on the aortic patch, were used. Two patients (1.1 %) had an alternative arterial reconstruction. In one of those patients, arterial reconstruction was performed by direct end-to-side anastomoses of the splenic artery to the SMA. In the other patient, only the donor external iliac artery was used for arterial reconstruction. The overall 5-year graft survival in this cohort was 86.4 % and the 5-year patient survival was 88.7 %.

Arterial thrombosis
The different grades of arterial thrombosis are demonstrated in Fig. 1. Both reviewers reported higher thrombosis incidences compared to original report. R1 detected arterial thrombosis in 80.5 % of all scans (256/318). Of the 318 scans, 187 scans were scored a Grade 1 arterial thrombosis (58.8 %), 69 scans a Grade 2 arterial thrombosis (21.7 %) and no case of complete arterial thrombosis was reported. R2 detected arterial thrombosis in 75.2 % of all scans (239/318). Of the 318 scans, 180 scans were scored a Grade 1 arterial thrombosis (56.6 %), 58 scans a Grade 2 arterial thrombosis (18.2 %) and one case of complete arterial thrombosis (0.3 %).

Inter-rater agreement
Both reviewers classified all 318 CT scans into 1 of 4 categories. All scans received an arterial thrombosis score and a venous thrombosis score. For arterial thrombosis, overall Cohen's kappa was 0.403 (SE = 0.046, 95 % CI 0.313;0.493). For venous thrombosis, overall Cohen's kappa was 0.537 (SE = 0.038, 95 % CI 0.463;0.611). The CT scans were also classified according the quality of arterial and venous phase scan and Cohen's kappa values were measured for different categories of quality (Table 3)  The difference in distribution between both reviewers was calculated using the Stuart Maxwell Test for marginal homogeneity. Difference in distribution was considered to be significant for arterial thrombosis (p = 0.025), whereas for venous thrombosis it was not (p = 0.233).

Clinical outcome
In this study population, 26 transplantectomies were performed in 177 transplantations (14.7 %). One patient lost both his first and second transplant and underwent two transplantectomies during the duration of this study. Nineteen grafts (10.7 %) were removed because of complete thrombosis which had led to necrosis of the pancreatic tissue. The other 7 grafts (4.0 %) were removed for other reasons including bleeding, leakage and one case of proven rejection. Sensitivity and specificity were calculated for evaluation of the accuracy of assessment of Grade 3 thrombosis. Sensitivity was 80.0 % and specificity 96.4 %.

Discussion
This study was primarily designed to assess the applicability and the reproducibility of the CPAT grading system on our pancreas transplant database. The CPAT grading system is the first classification scheme developed for pancreas allograft thrombosis. Although appearing promising, the applicability and reproducibility of the CPAT grading system has not been reported by others to date. To evaluate whether we can implement this grading system into our clinical practice, we assessed the inter-rater agreement of two local radiologists on a set of 318 CT scans from 177 pancreas transplant patients. Our results showed a moderate inter-rater agreement for both arterial and venous thrombosis. Cohen's kappa for overall arterial thrombosis was 0.403 and for overall venous thrombosis 0.537. According to the guidelines of Landis and Koch [16] these kappa values correspond with fair strength of agreement in arterial thrombosis and moderate strength of agreement in venous thrombosis.
For arterial thrombosis, CPAT scores from both reviewers differed significantly (p = 0.025). Analysis of our results suggest that there is a correlation between the quality of the scans and the extent of agreement. Not surprisingly, the agreement in scans, classified as poor quality, was lower in both arterial and venous thrombosis. Results in this category showed different CPAT scores between both reviewers (Supplement 2). Cases of disagreement between no thrombosis and Grade 2 (6/318, both arterial and venous thrombosis) and between no thrombosis and Grade 3 (3/318, venous thrombosis) thrombosis were documented. In clinical practice, these cases of disagreement would correspond with disagreement on administration of anticoagulants (Grade 2/Grade 3) or not (Grade 0). This depicts the importance of the quality of a CT scan for adequate evaluation of the vascularization. Acceptance of poor CT-scans could increase the risk of incorrect diagnosis in real clinical practice in which evaluation is only performed by a single reviewer. As was described in the introduction, results from the study by Hakeem et al. [15] showed that Grade 1 and 2 of arterial thrombosis and Grade 1 of venous thrombosis can be managed without anticoagulation. However, patients with Grade 2 venous thrombosis or complete arterial/venous thrombosis have shown to benefit from anticoagulative therapy. Hence, identification and consensus in these cases is paramount. Also, in clinical practice the radiologists are not blinded to the patient's clinical characteristics and it is likely that a more weighted decision can be made concerning the thrombosis grade, presumably after consultation of the surgeon.
Inter-rater agreement for grade subgroups was measured and expressed in the Fleiss' kappa. In the arterial thrombosis set, reviewers disagreed on one CT scan: whether it was proximal partial (Grade 2) or complete (Grade 3) thrombosis. Because of the small number of cases in this category (N = 1), a valid Fleiss' kappa value could not be measured for this category. Although outcomes on the overall inter-rater agreement are relatively low, agreement on thrombosis grades, in which treatment is considered to be effective, was found to be substantial.
In our experience, description of the CPAT grading system provided in the article by Hakeem et al. was not specific enough for accurate implementation. Even with the additional information provided by the research team which developed the CPAT grading system, our radiologists experienced difficulty with some parts of the classification. This difficulty was mostly experienced in differentiation between a Grade 0 and Grade 1 thrombosis, stated by the Cambridge research team as respectively the absence or presence of a visible thrombus at the transected margin, not extending into the main vessel. In our opinion, development of a thrombus at the transected margin is considered to be a physiological process in a blind ending vessel. For that reason, presence of a thrombus at the distal end of the vessel may be expected or even assumed. However, due to the small caliber of the vessels it is not always possible to follow the vessel until it's transected margin or to distinguish the contours of a distal thrombus on a CT scan. In this situation, the decision between presence or absence of a distal thrombus (Grade 0 vs Grade 1) will most likely be based on the reviewer's personal intuition.
Difficulty was also experienced in the differentiation between Grade 1 and Grade 2 thrombosis in the small branches of SA, as a result of the definition in the CPAT grading system. The CPAT system defines Grade 1 thrombosis as the presence of a thrombus in the very distal vessel at the transected margin but not reaching into the main vessel. The uncertainty for our reviewers was what the definition of 'the main vessel' is. According to the additional information and drawings (Supplement 1), the main vessel was defined as the SMA proximal to the branches to the pancreatic head (inferior pancreatico-duodenal artery, IPDA), but for the SA this was not as clearly stated. Our radiologists assumed that if the thrombus was not extending too far in the SA (1− 2 cm from the transected margin), classification as a Grade 1 would be most accurate. We question whether documentation of the presence of a Grade 1 thrombosis would be relevant in clinical practice. Results from the study by Hakeem et al. [15] showed that there is no benefit of anticoagulation and on top of that our results showed a low inter-rater agreement for this

group.
Another limitation of the CPAT grading system is that it does not mention guidelines for thrombosis in branches of the main vessels, for example the IPDA. Finally, the CPAT grading system defines Grade 2 thrombosis as partial non-occlusive thrombosis extending into the main trunk of SMA/SA or SMV/SV but not into the Y-graft or the portal vein. In our set, we encountered several cases in which there was complete occlusion of either the SMA/SA or SMV/SV, but without extent into the Y-graft or portal vein. We were uncertain whether these cases should be classified as a Grade 2 or a Grade 3 according to the CPAT grading system. We agreed to classify these scans as a Grade 2.
Despite of the difficulties, experienced with the initial implementation, the CPAT grading system was considered reproducible and applicable on our database. We believe that implementation of a grading system would improve identification and understanding of pancreas allograft thrombosis. Since the CPAT grading system is, to our best knowledge, the only available grading system, implementation of this system would be a good first step in systematically reviewing pancreas allograft CT scans. Also, classification according to the CPAT system would considerably enhance both communication between radiologist and surgeon and the quality of documentation, which will benefit both clinical and future scientific purposes.
It was a remarkable finding that on re-evaluation 80-90 % of all CTscans showed some form of thrombosis. We cannot conclude whether all the detected thromboses should be considered relevant in clinical practice, since the focus of this study was to evaluate the CPAT grading system. Further research to investigate the efficacy of anticoagulation in such cases is warranted. However, as previously described, Hakeem et al. [15] concluded that treatment of Grade 1 venous and Grade 1 and 2 arterial thrombosis was not considered to be effective, whereas treatment of Grade 2 venous thrombosis and Grade 3 arterial and venous thrombosis significantly contributes to a better graft survival.
One important limitation of this study is the absence of a goldstandard, different from the CT scan which was re-evaluated for this study, to compare the results of the re-evaluation to. The only available clinical outcome is whether a transplantectomy was performed due to   We acknowledge that there is a possibility that our reviewers did not classify all CT scans correctly during the re-evaluation. This type of error is called perceptual error and it is an acknowledged and unfortunately unconquered problem in radiology. Complete elimination of this type of error cannot be achieved since radiologic interpretation is a human enterprise which cannot be automated [17]. To minimize the risk of perceptual error in this study, all CT scans were anonymized, and reviewers were blinded to each other's evaluation. Furthermore, results were documented in structured and uniform reports. Previous studies have reported on the accuracy of CT imaging as a diagnostic tool for pancreas allograft thrombosis [18][19][20][21]. Studies report different statements on whether US Doppler or CT imaging should be used for the detection of thrombosis. Tolat et al. [18] described that both arterial and venous pancreatic allograft thrombosis are best displayed with volumetric high-spatial-resolution CT scanning. These statements were supported by O'Malley et al. [19] which reported that cross-sectional imaging plays a key role in the accurate assessment of vascular complications after pancreas transplantation. In our center, CT imaging is considered most accurate for assessment of post-operative complications including thrombosis. For that reason, a routine abdominal CT scan is performed in all pancreas transplant in the first week after transplantation.
Another limitation of our study is the assessment of the inter-rater agreement of the CPAT grading system for only two reviewers. Theoretically, involvement of a third reviewer would contribute to a higher reliability of our results. However, it is no guarantee that the third reviewer will score the CT scans more correctly or as correctly as the first two reviewers. Since there is no gold-standard for interpretation of pancreas allograft thrombosis, it is not possible to state that the two out of three reviewers with the highest agreement implemented the CPAT grading system most accurately.
Since this study is a retrospective analysis, results were based on CT scans performed in the past. For that reason, it was not possible to avoid the inclusion of CT scans which were found to be of poor quality (arterial: 12 scans, venous: 24 scans). Reason for the poor quality was predominantly a relatively hypovolemic state of the patient, resulting in a poor visibility of the venous vessels. However, all included scans were originally requested for evaluation of vascular status of the pancreas allograft and therefore the presence of poor scans in our set reflects the reality in clinical practice.

Conclusion
In conclusion, currently there are no solid guidelines for documentation on pancreas allograft thrombosis, which has resulted in variability within CT reports. The CPAT grading system is, to our best knowledge, the first and only available grading system for pancreas allograft thrombosis. Although some training and extra information was needed to precisely understand the classification and to correctly score the CT images, the reviewers considered the CPAT grading system reproducible and applicable. Implementation of the grading system will firstly enhance communication between radiologist and surgeon. Also, it will improve both the documentation and the understanding of pancreas allograft thrombosis. Therefore, we would recommend to implement the CPAT grading into clinical practice. It remains questionable whether identification of Grade 1 thrombosis would be relevant in clinical practice since this is considered to be a physiological process in a blind ending vessel. The agreement within this category has shown to be low and treatment is not considered to be effective in patients with arterial or venous Grade 1 thrombosis. Furthermore, this study depicts the importance of a good quality of CT-scan in diagnosis of pancreas allograft thrombosis. Poor CT scans showed diverging CPAT scores. Therefore, a CT scan, which meets the quality standard in regard to correct contrast phasing, should be demanded to ensure accuracy of the reviewer's judgment.