, Introduction
Acute kidney injury (AKI) is a common condition among critically ill patients. A recent multicenter study reported that the incidence of AKI at all stages was 57% while that of stage 3 AKI was 30%; the mortality rate increased as AKI severity increased in critically ill patients [1]. Although continuous renal replacement therapy (CRRT) has not been proven superior to intermittent renal replacement therapy (RRT) in critically ill patients [2,3], CRRT is the preferred and mainly used RRT in critically ill patients because of its hemodynamic stability and high capacity to remove solutes or fluids. Meanwhile, CRRT has disadvantages including the risk of catheter-related complications, risk of bleeding associated with continuous anticoagulation, increased length of intensive care unit (ICU) stay, delayed rehabilitation, and removal of undesired substances such as antibiotics [4].
Therefore, it is important to discontinue CRRT for patients who no longer require it. However, consensus is lacking regarding the optimal timing of CRRT discontinuation. Urine output is reportedly the most consistent and important predictor of successful discontinuation of CRRT [5,6]. Some studies presented regression equations of urine output, serum creatinine, or sequential organ failure assessment score and showed excellent discriminative ability for successful discontinuation of CRRT [5,7,8]. However, no externally validated prediction models are simple or easy to apply.
We previously developed a prediction model derived from four clinical parameters for successful discontinuation of CRRT that was simple and showed good performance [9]. This study aimed to validate a prediction model for successful discontinuation of CRRT using temporal and external cohorts.
Methods
Study setting and population
This retrospective cohort study consisted of one temporal cohort and four external cohorts. All cohorts included adults (>18 years) who received CRRT for at least 3 days and survived for at least 7 days after its discontinuation between January 1, 2018 and November 31, 2020. Excluding patients with preexisting end-stage kidney disease (n = 345) or patients in hopeless condition or with insufficient data (n = 29), 1,517 patients were included in the final analysis: 540 in the temporal cohort from Samsung Medical Center and 977 from Seoul St. Mary’s Hospital, Yeouido St. Mary’s Hospital, Bucheon St. Mary’s Hospital, and Myongji Hospital.
Ethics approval and consent to participate
The study complied with the Declaration of Helsinki and was approved by the Institutional Review Board of Samsung Medical Center (No. 2021-01-052, December 31, 2020; titled “Investigation of evidence-based optimal management strategies for continuous renal replacement therapy”), which waived the requirement for informed consent because of the minimal risk and retrospective nature of the collected data.
Variables
Data were extracted from the electronic medical records. Age, sex, primary diagnosis at admission, variables of the prediction model (urine output, blood pressure, blood urea nitrogen [BUN], and serum potassium), and reinitiation of RRT (CRRT or intermittent RRT) after CRRT discontinuation were obtained.
Prediction model
The prediction model was composed of four variables [9]: 24-hour urine output (mL/day) on the day before CRRT discontinuation (D–1), mean arterial blood pressure (MAP) (mmHg), BUN (mg/dL), and serum potassium (mmol/L) on the day of CRRT discontinuation (D0). Each variable was scored as follows: 4 points with a urine output on D–1 ≥300 mL/day; 1 point if MAP on D0 was 50 to 78 mmHg; 2 points with BUN on D0 <35 mg/dL; and 1 point if serum potassium <4.1 mmol/L. The total score, which was the sum of the scores, was 0 to 8 points. The patients were divided into three groups according to their total scores: low (0–2 points), intermediate (3–5 points), and high (6–8 points).
Validation of model performance
Discrimination performance was evaluated using the area under the receiver operating characteristic curve (AUROC) and calibration performance was evaluated by comparing the predicted probability of each score or group of scores to the observed probability.
Outcome
Successful discontinuation of CRRT was defined as the absence of RRT reinitiation within 7 days of CRRT discontinuation. Resumption of RRT with transition to intermittent hemodialysis (HD) within 7 days after CRRT discontinuation was considered as failure of CRRT discontinuation. Patients were divided into success and failure groups according to successful discontinuation of CRRT.
Statistical analysis
Descriptive summaries are presented as frequencies (percentages) for categorical variables and mean (standard deviation) or median (interquartile range) for continuous variables. The discrimination performance was assessed using the area under the AUROC, and the calibration performance was assessed by comparing the predicted probability of each score or score group to the observed probability. Data were analyzed using SAS version 9.4 (SAS Institute). The p-values of <0.05 were considered statistically significant.
Results
Patients’ characteristics
The median patient age was 65 years (interquartile range, 54–76 years), and 61.8% were male (Table 1). The primary diagnoses upon admission were cancer (57.2%), heart failure (27.2%), and others (15.6%). Approximately 45% of patients (677 of 1,517) successfully discontinued CRRT. A higher proportion of patients had heart failure in the success versus failure group. Urine output was larger, and BUN and serum potassium levels were lower in the success versus failure group. However, MAP was comparable between the two groups. Age, proportion of primary diagnoses, and values of major variables on the day of CRRT discontinuation differed among institutions (Table 2). The rate of successful discontinuation of CRRT was 40% to 51%.
Performance of prediction model
The predicted and observed probabilities of successful discontinuation of CRRT in the entire cohort matched well according to each score (Fig. 1A) and the low- (0–2 points), intermediate- (3–5 points), and high-score (6–8 points) groups (Fig. 1B). The overall differences between the observed and predicted incidence were 3.0% (17.7% observed and 16.9% predicted probabilities), 3.6% (35.2% and 34.8%), and 2.0% (69.3% and 70.3%) in the low-, intermediate-, and high-score groups, respectively. The overall AUROC of the prediction model was 0.74 (95% confidence interval [CI], 0.71–0.76) (Fig. 1C). We assessed the AUROC according to the primary diagnosis at admission. Regardless of primary diagnosis, the prediction model showed good performance (AUROC [95% CI]: 0.76 [0.68–0.83] in heart failure, 0.75 [0.69–0.81] in cancer, and 0.73 [0.67–0.76] in others) (Fig. 2).
Performance of prediction model at each hospital
We further assessed the performance of the prediction model according to the institution. One temporal cohort and three external cohorts showed good discriminative ability (AUROC [95% CI]: 0.77 [0.73–0.81], 0.73 [0.69–0.77], 0.72 [0.63–0.82], and 0.73 [0.63–0.84]), but one external cohort (Hospital C) showed poor discriminative ability (0.56 [0.45–0.67]) (Fig. 3). This cohort, with poor discriminative ability, showed little association between the scores and the CRRT discontinuation success rate. The predicted and observed probabilities of successful discontinuation of CRRT in each cohort according to the score groups and scores are presented in Fig. 4 and Supplementary Fig. 1 (available online), respectively.
We analyzed each variable separately to elucidate what variable in the model caused the difference in performance between cohorts. Hospital C (AUROC 0.539 for urine output) showed very low discriminative ability for urine output compared to other hospitals (AUROC 0.698, 0.715, 0.665, and 0.683 for urine output in each hospital) (Supplementary Fig. 2, available online). Hospital C also tended to show lower or similar discriminative ability for BUN and serum potassium, but higher discriminative ability for MAP compared to other hospitals (Supplementary Figs. 3–5, available online).
Differences in the strategies of continuous renal replacement therapy operation
We compared differences in the overall steps of CRRT operation to elucidate the cause of the differences in model performance between hospitals. Table 3 shows CRRT operation strategies regarding nephrology engagement and CRRT-dedicated nursing staff of each hospital. In Samsung Medical Center and the three hospitals where the performance of the prediction model was good, nephrologists determined all important steps of CRRT operation including initiation, prescription with monitoring the adequacy of CRRT, and screening the possibility or final decision of CRRT discontinuation by multidisciplinary care with critical care physicians. On the other hand, both prescription and discontinuation of CRRT were primarily controlled by attending physicians in the Hospital C although only initiation of CRRT was determined by nephrologists under on-demand consultation.
Discussion
Here we validated our simple prediction model for successful discontinuation of CRRT. The prediction model showed good discriminating ability for one temporal cohort and three external cohorts but showed poor performance for one external cohort. Our prediction model was based on urine output, MAP, BUN, and serum potassium levels. This is simple and easy to apply in clinical practice; however, as shown in one external cohort, its usefulness can be affected by the strategies of CRRT operation.
Consensus is currently lacking on CRRT discontinuation criteria. The prolonged maintenance of CRRT increases the length of ICU stay and overall costs, delays mobility, and exposes patients to CRRT-associated complications [4]. On the other hand, whether reinitiating RRT per se after CRRT discontinuation adversely affects mortality or renal outcomes has not been sufficiently studied. Previous observational studies demonstrated that 40%–50% of patients with AKI requiring RRT failed to discontinue it and had higher mortality rates and a lower chance of renal recovery than those who successfully discontinued RRT [6,10–12]. These differences may reflect the overall disease severity of patients rather than the adverse effects of RRT reinitiation. However, if kidney function does not sufficiently recover after CRRT is discontinued, the patient is at risk of recurrent volume overload, uremia, and electrolyte imbalance. In addition, approximately 50% of patients experience intradialytic hypotension when CRRT is switched to intermittent RRT, which can cause additional ischemic injury and aggravate organ dysfunction [13]. Therefore, the appropriate timing of CRRT discontinuation is important.
Urine output was the only common significant factor associated with successful discontinuation of CRRT across several previous studies and in our study [5–9,14]. Although most previous studies were small-scale, the largest multicenter study reported urine output, increased urine output, serum creatinine, history of chronic kidney disease, and CRRT duration as significant factors for successful discontinuation of CRRT [6]. Two previous studies presented calculation formulas using a regression model [5,7]. However, despite their high AUROC, they did not provide a simplified scoring system; thus, these models could be less practical for clinical decisions. One study presented a simple scoring system and a high AUROC with kinetic estimated glomerular filtration rate and urine output; however, kinetic estimated glomerular filtration rate itself requires serial serum creatinine values and a complex calculation formula [8]. In addition, the prediction models reported in previous studies have not been validated. Our prediction model, which is based on a simple scoring system with clinical parameters that can be easily obtained daily, has a relatively good AUROC value and good discriminative ability [9]. Furthermore, we validated the model in both temporal and external cohorts.
Four clinical variables were selected for our model and applied regardless of the patient’s medical condition or diagnosis. However, the relative importance of each variable may vary depending on the patient’s underlying disease or clinical setting. Although the performance of our simple model did not depend on primary diagnoses, more accurate and differential weighting of each variable according to the patient’s condition or clinical setting may improve the universality and optimality of the model. On the other hand, complex model may be less clinically useful. Further research is needed to improve the performance and universality of our model with simple clinical variables.
It is worth noting that the performance of our prediction model was good in the four cohorts but poor in one external cohort. This discrepancy can be caused by differences in CRRT application, operation policies, and patient characteristics. Only one institution demonstrated poor model performance, whereas the other two institutions with similar numbers of patients demonstrated good model performance. In addition, urine output, which has been reported as an important factor predicting CRRT discontinuation or kidney recovery in most previous studies [5–9,14], was not helpful in predicting successful discontinuation of CRRT in one institution with poor model performance. Therefore, poor performance of prediction model in one institution seems to be attributed to CRRT operation strategy rather than patient characteristics. In one institution showing poor performance of the model, the nephrology staffs or team played only a limited role in deciding the initiation of CRRT as consultants and did not actively engage to control and discontinue CRRT. Therefore, our study strongly supports the necessity of active intervention by nephrology team or specialized CRRT team for optimal CRRT management and good performance of our prediction model. On the other hand, our prediction model may also help clinicians make appropriate decisions regarding CRRT discontinuation even in institutions without active participation of nephrology staffs. Further prospective studies are needed to verify whether our prediction model can effectively assist the adequate discontinuation of CRRT in institutions with limited nephrological resources.
In South Korea, switching CRRT to intermittent HD is actively attempted in patients whose overall hemodynamic status has been relatively stabilized even during ICU stay, and this effort for the transition of the RRT modality has been driven by the Korean National Health Insurance Service. This policy makes a significant portion of CRRT discontinuation in South Korea aimed at transitioning to intermittent HD before sufficient renal recovery, rather than complete discontinuation of RRT. Therefore, our findings should be interpreted with caution in countries or institutions whose primary goal of CRRT discontinuation is to discontinue RRT completely. Nevertheless, our prediction model is still useful when attempting to use a scoring system to aid clinical decisions regarding transitioning from CRRT to intermittent HD or maintaining CRRT.
This study had several limitations. First, it was a retrospective cohort study. However, several cohorts were used to validate the prediction model, which may minimize or overcome potential bias. Second, our multicenter study included only Korean ICU patients covered by the Korean National Health Insurance Service. Therefore, our prediction model requires validation in other races or countries. Third, our prediction model showed poor performance in one external cohort, suggesting the necessity of active intervention by nephrology team supporting the entire process as a prerequisite for applying our prediction model. However, as mentioned above, if our prediction model is applied appropriately in an institution with limited involvement of nephrology team, it has the potential to help decisions regarding CRRT discontinuation. Fourth, detailed causes of AKI requiring RRT were not included in the analyses. The retrospective data collection inevitably limits the accurate capture of such data. To overcome this limitation, we conducted a subgroup analysis based on the primary diagnosis at admission and demonstrated that the model was effective for each group.
In conclusion, the good performance of our prediction model for successful discontinuation of CRRT was validated in all four cohorts, except for one cohort in which nephrologists did not actively engage in CRRT operation. This model can help discriminate between patients with a high or low probability of successful discontinuation of CRRT depending on the active engagement of nephrologists in CRRT operation. Further prospective multicenter studies with protocolized CRRT management considering the active engagement of nephrologists in CRRT operation are required to validate and improve our predictive model.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Supplement figure 1
Print
