External validation of the international immunoglobulin A nephropathy (IgAN) prediction tool in Korean children with IgAN
Article information
Abstract
Background
The International IgA Nephropathy (IgAN) Prediction Tool was recently updated to predict the risk of a 30% decline in estimated glomerular filtration rate (eGFR) or kidney failure in children with IgAN. We aimed to evaluate the clinical performance of this tool in a Korean cohort of children with IgAN.
Methods
We calculated the predicted risk for biopsy-proven IgAN children from 20 Korean centers. The primary outcome was a 30% decline in eGFR or kidney failure. Discrimination and calibration performances of two pre-developed models were first evaluated. Subsequently, we constructed an updated model for Korean children using clinically meaningful variables.
Results
The study included 472 children with a mean age of 11.4 years. During a median follow-up period of 47.5 months, 58 patients (14.0%) reached the primary outcome. The two prediction models from the International IgA Nephropathy Prediction Tool exhibited suboptimal prediction power, with an integrated area under the curve (AUC) level of 0.57 (model with race) and 0.55 (model without race), respectively. The updated model, incorporating additional coefficients (sex, body mass index, serum albumin, presenting symptoms), showed good agreement between predicted risk and observed outcomes for Korean children (integrated AUC level of 0.70), significantly better than the IgAN International tool. Various model performance assessments showed consistent results. External validation with 145 children also demonstrated a superior fit for our model.
Conclusion
The updated International IgA Nephropathy Prediction Tool for children had suboptimal prediction ability in Korean IgAN children whereas our proposed model showed acceptable prediction ability in this population.
Introduction
Immunoglobulin A (IgA) nephropathy (IgAN) is one of the most common primary glomerulonephritis in children globally, particularly in the East Asian population [1]. Children with IgAN have a diverse spectrum of clinical manifestations and prognoses, ranging from persistent microscopic hematuria to rapid kidney function deterioration. Individual risk factors such as proteinuria, pathological grade, and decreased estimated glomerular filtration rate (eGFR) at presentation have been suggested as prognostic indicators; however, variability persists even in the same risk groups [2–4]. In this context, the availability of a reliable tool to evaluate risk and categorize children with IgAN would facilitate timely recognition of patients at elevated risk. This, in turn, would assist in prioritizing treatment and minimizing unwarranted exposure to aggressive treatment. The International IgA Nephropathy Network developed a prediction tool for adults and externally validated two prediction models, one with and one without race/ethnicity [5]. These models were based on retrospective analyses of extensive, multinational IgAN datasets, utilizing Oxford classification and clinical variables such as age, blood pressure, eGFR, proteinuria, and the use of renin-angiotensin system (RAS) inhibitors and immunosuppressants to predict a 50% decrease in eGFR or the onset of kidney failure. Kidney Disease: Improving Global Outcomes (KDIGO) endorsed this tool as a reliable instrument for identifying the risk of disease progression and it has been externally validated across various races and ethnicities, including adult Korean IgAN patients [6,7].
Recently, the international prediction tool was updated for application to children by the same research team [8]. However, validation of this tool has not yet been extensively conducted in external populations, including the Korean population. Given IgAN’s higher prevalence and incidence in East Asia than in Western countries, and its more rapid progression to kidney failure in Pacific Asian patients compared to other ethnicities, we aim to validate this prediction tool in Korean children with IgAN [9–11].
Hence, the primary objective of this study was to perform an external validation of the International IgA Nephropathy Prediction Tool for children using a multicenter cohort from Korea, as Korean subjects were not included in the original study. Additionally, we aimed to establish a new model to predict kidney progression for Korean children to enhance the accuracy of prediction in this population.
Methods
Data source and patient selection
A total of 617 cases were included in this study (Fig. 1). Our study comprised two independent cohorts. The development cohort, aimed at validating the international IgAN prediction tool and developing a new model for Korean pediatric patients, consisted of 472 individuals whose characteristics have been described in a previously published paper [12]. The surveyed patients consisted of 1,154 individuals diagnosed with IgAN before the age of 18 years across 20 institutions from 1985 to 2015. From this pool, 472 patients with available Oxford classification and eGFR data at kidney biopsy, as well as over 1 year of eGFR follow-up data, were selected. The validation cohort used to validate the new tool comprised 145 patients not included in the development cohort. These patients were diagnosed and received over 1 year of follow-up between 2015 and 2022 at two institutions, Seoul National University Hospital and Seoul National University Bundang Hospital. Patients with secondary forms of IgAN, such as IgA vasculitis nephritis, were excluded [13].
Study flow chart.
eGFR, estimated glomerular filtration rate; IgAN, IgA nephropathy.
The Institutional Review Board of Seoul National University Hospital approved the study (No. 2306-217-1446), and the requirement for informed consent was waived due to the study’s retrospective design.
Exposure and outcomes
We collected explanatory variables for the International IgA Nephropathy Prediction Tool for children, namely age, eGFR, blood pressure, proteinuria, the Oxford MEST (mesangial hypercellularity, endocapillary hypercellularity, segmental glomerulosclerosis, and tubular atrophy/interstitial fibrosis) score, and the use of RAS inhibitors and immunosuppressants at the time of biopsy. As in the original paper, eGFR was calculated using the full-age spectrum eGFR; if it was unavailable, the Schwartz eGFR formula was used [14]. Mean arterial pressure was standardized according to the method of Wühl et al. [15], and proteinuria was normalized to a body surface area of 1.73 m2. Additionally, sex, body mass index (BMI), serum albumin, and presenting symptoms—categorized as microscopic hematuria with or without proteinuria, gross hematuria, or others such as isolated proteinuria or nephrotic syndrome—were gathered as potential explanatory variables for the new tool. Primary outcome was defined as a 30% decrease in eGFR or the development of kidney failure. The duration until this primary outcome occurred, or the censoring time, was documented to conduct survival analysis.
Statistical analysis and model development
Continuous variables with a normal distribution are presented as medians with interquartile ranges (IQRs) while categorical variables are presented as counts with percentages. We calculated the predicted risk of the primary outcome for each patient using two models: one with the “other race/ethnicity” category and one without race [8]. We then proposed a new model with risk factors determined based on clinical relevance for Korean children. The performance of these predictive models was assessed and compared based on predicted probability using the Kaplan-Meier method. Survival curves for risk groups were categorized by four percentiles (low, <15th; intermediate, 15th–50th; high, 51st–85th; and very high, ≥86th).
Prediction abilities of the models were evaluated using discrimination and calibration measures, namely the Akaike information criterion (AIC), Royston-Sauerbrei D statistic (R2D), and integrated area under the curve (iAUC). The R2D is a Cox model-based measure of explained variation on the log relative hazard scale, where an increase in R2D indicates a better fit [16]. The AIC is a widely used model fit measure for selecting a better-fitting model; a reduction in AIC indicates a better fit [17]. The iAUC is a weighted sum of time-dependent AUC values during follow-up in the presence of time-to-event outcomes [18]. We calculated iAUC with 95% confidence intervals (CIs) using the bootstrapping resampling method. Additionally, we compared the two models with our proposed model using iAUC differences. When the 95% CI of the iAUC difference does not include 0, it indicates a significant difference between the two models.
The final step involved applying our proposed model to an external sample (validation cohort) to verify its relevance. Two-tailed p-values less than 0.05 were considered statistically significant. All data were analyzed using R version 4.4.0 (R Core Team).
Results
Baseline characteristics
Table 1 summarizes the characteristics of our study participants and compares them with those of patients belonging to the original cohort. When compared with the original cohort, there were no significant differences in follow-up duration, male proportion, eGFR, or proteinuria level among the cohort participants in this study. However, individuals in the development cohort tended to be younger, while those in the validation cohort tended to be older. In terms of pathologic findings, individuals in the development cohort exhibited a lower proportion of E1 (endocapillary hypercellularity) lesions, and overall, our cohort showed a lower proportion of T1 lesions (interstitial fibrosis/tubular atrophy greater than 25%). Notably, the proportion of patients receiving RAS blocker (RASB) and immunosuppressants within our cohort was much higher than the original cohort, although some of the enrolled children’s detailed usage data were acquired within 1 year of biopsy.
Baseline characteristics of the current and original international cohorts
Evaluation of the international model and development of a new model
After a median follow-up of 47.5 months (IQR, 23.2–72.4 months), 58 children (14.0%) reached the primary endpoint (Supplementary Fig. 1, available online). Survival curves estimated using the Kaplan-Meier method showed significant separation between the very high-risk group and others based on the international model; however, separation between low and intermediate-risk groups was less clear. The model incorporating race underestimated primary outcome risk, as evident from the predicted survival curves deviating upwards compared to the observed data (Fig. 2A, B).
Survival curves comparing the primary outcome across risk groups defined by the linear predictor.
Dashed lines indicate predicted risks within each group, while solid lines show observed risks using the Kaplan-Meier method. (A) International model including race. (B) International model excluding race. (C) Our proposed model.
The fit of the international model, as shown in Table 2, was poor in our cohort, with R2D statistics of 0.017 and 0.005 for models with and without race, respectively. Discrimination, as measured by the iAUC, was also suboptimal: 0.565 (95% CI, 0.504–0.643) for the model with race and 0.547 (95% CI, 0.501–0.624) for the model without race.
Model performance in the development and validation cohorts
Given these limitations, we developed a new model using a Cox proportional hazards model with the additional variables of sex, systolic and diastolic blood pressure, BMI, serum albumin, and presenting symptoms as explanatory variables (Table 1; Supplementary Fig. 2, available online). Follow-up periods were similar across the added clinical factors, except that children presenting with gross hematuria had a significantly longer follow-up duration compared to those with microscopic hematuria (54.2 months vs. 37.4 months, respectively). This model demonstrated distinct separation between risk groups and alignment between predicted and actual survival (Fig. 2C). The iAUC improvement over the international model was significant, with a 0.132 increase (95% CI, 0.034–0.229) with race incorporated and 0.150 (95% CI, 0.049–0.246) increase without race incorporated (Supplementary Fig. 3).
External validation of the new model
The validation cohort’s characteristics were largely consistent with those of the development cohort, except for a higher occurrence of endocapillary lesions (Table 1; Supplementary Fig. 4, available online). Our new model showed better fit and discrimination than the international model, as evidenced by improved AIC and R2D values (AIC value, 132.73; R2D value, 0.725). The iAUC significantly exceeded that of the international model, with or without race, by 0.263 (95% CI, 0.124–0.387) and 0.283 (95% CI, 0.125–0.402), respectively (Table 2; Supplementary Fig. 5, available online).
Discussion
In this study, we assessed the performance of the International IgA Nephropathy Prediction Tool in a multicenter cohort comprising 472 Korean children. We found that both models had suboptimal discrimination power in low- and intermediate-risk groups. Further, models that incorporated race underestimated the risk for progression in Korean children. We showed that our updated model with additional explanatory variables predicted the risk with good calibration, which was further demonstrated in an additional 145 Korean children.
IgAN is the most prevalent glomerulonephritis and a common cause of chronic kidney disease, with about 30% of adult IgAN patients eventually requiring dialysis [19,20]. Therefore, understanding the risk of progression of IgAN is essential for treatment decisions and the risk-benefit balancing of medications such as immunosuppressants. Accordingly, an international prediction tool for adults utilizing about 4,000 patients was developed in 2019, and several countries not belonging to the original cohort, including Korea, have since performed external validation [5,7,21]. Based on the findings of these studies, the 2021 KDIGO guidelines recommended the use of this tool in clinical practice [6]. However, IgAN in children often presents with more active lesions on biopsy, such as endocapillary hypercellularity and crescents, but fewer chronic changes, thus a greater potential for improvement when treated with immunosuppressants compared to adults [3,22]. Consequently, this tool was updated in 2021 utilizing about 1,000 children using a similar approach as used to develop the original tool [8]. Notably, due to physical growth in children, the eGFR trajectory showed a nonlinear progression—an initial increase in eGFR, followed by a plateau and then an eventual linear decline—and a 30% decline in eGFR was set as the primary endpoint. Subsequently, two external validations were conducted in China, which represents the first external validation study targeting pediatric patients not initially included in the original model [23,24]. Both studies included pediatric IgAN patients from central China (n = 439) and Southwest China (n = 210). In both studies, it was suggested that discrimination and calibration for the full model with and without race were not highly applicable to children. Given the unsatisfactory results observed in China, which included race as present in the original pediatric international cohort, we speculated that risk prediction modification based on particular patient characteristics could be necessary for the effective application of the prediction tool.
Discrimination power was unsatisfactory when the international prediction tool was applied to Korean pediatric patients; in particular, while both models were able to distinguish the very high-risk group from other groups well, they failed to discriminate effectively between intermediate and low-risk groups. Interestingly, a validation study conducted in China also reported satisfactory discrimination only in the highest group [24]. Furthermore, prediction was suboptimal in our study with an iAUC value of less than 0.7.
The primary reason for the observed unsatisfactory outcomes is likely racial differences, particularly as Korean individuals were excluded from the original model. According to results from adult studies, the progression of IgAN varies with race and ethnicity, with individuals of Pacific Asian descent experiencing a faster progression of the disease than Caucasians [9]. Furthermore, when we incorporated racial coefficients for Japanese or Chinese individuals—who, like our study cohort, belong to the East Asian group—into the model, it still exhibited poor calibration. In contrast, the model without race yielded more accurate calibration, highlighting substantial differences in disease progression across East Asian races. This suggests that for the international model to be used effectively, it is imperative to consider specific racial subgroups.
In addition to racial differences, variations in patient characteristics or treatment patterns across countries may also account for these differences. For example, adjunctive treatments such as tonsillectomy or antithrombotic medication are predominantly administered in Japan, while Chinese herbs are commonly prescribed in China; however, these treatments are not widely used in Korea [25,26]. In our cohort, subjects were younger, had lower eGFR at the time of kidney biopsy, and showed a lower percentage of E1 and T1 findings in renal pathology compared to those in the initially reported derivation cohort. Moreover, as shown in Table 1, patients in our cohort were treated with RASB and/or immunosuppressants more frequently than those in the original cohort. In Korea, mass urine screening has been implemented since 1998, enabling the early detection and treatment of glomerulonephritis in asymptomatic children and adolescents presenting with only urinary abnormalities [27-29]. Younger age at diagnosis, less severe renal pathology findings, and a proactive treatment approach are characteristic features of the Korean cohort, which differ from the original cohort that did not implement mass urine screening. These differences in patient characteristics and treatment patterns likely influenced the disparities observed between the cohorts, suggesting that such variations may have impacted model performance. Generally, persistent albuminuria often prompts suspicion of glomerulonephritis, even in the absence of symptoms such as edema, hypertension, or gross hematuria. Clinicians may use RASB or immunosuppressants to achieve proteinuria remission and renoprotection, sometimes even before a kidney biopsy is performed—particularly in settings where immediate biopsy is unavailable. The proactive treatment approach in Korea reflects clinicians’ intent to address active inflammation in patients before significant disease progression occurs; this strategy may artificially have lowered proteinuria, thereby underestimating the primary outcome risk and resulting in a worse prognosis than predicted by clinical factors. However, in our study, treatment data included both pre-biopsy and within 1-year post-biopsy, making it challenging to directly compare treatment patterns between the two cohorts. Further validation is required to better understand regional variations in treatment patterns and their impact on outcomes.
Consequently, we developed a novel model by adding readily available clinical parameters such as BMI, serum albumin, and presenting symptoms to the existing model. Research has shown that high BMI is linked to adverse kidney pathology in IgAN, including tubular atrophy, interstitial fibrosis, and mesangial matrix expansion [8,30,31]. A meta-analysis revealed that IgAN patients with high BMI had a significantly lower eGFR than those with normal BMI [32]. Moreover, a study in Korean adults indicated that obesity increased the risk of kidney failure over a median follow-up of 93 months [33]. While fewer studies have focused on pediatric patients, there is some evidence that high BMI aggravates kidney involvement in IgA vasculitis patients [34,35]. Considering the rising rates of obesity among Korean youth, the impact of BMI on pediatric IgAN warrants further investigation.
Similarly, serum albumin levels have been associated with IgAN progression. In more detail, lower serum albumin was linked to poor outcomes in a long-term Norwegian study, whereas increased levels during follow-up were protective against renal function decline in Polish patients [36,37]. This suggests the potential of serum albumin as a marker for long-term outcomes in pediatric IgAN, particularly as it reflects proteinuria status.
We also considered presenting symptoms such as asymptomatic hematuria and gross hematuria as risk factors in our study. Isolated hematuria, often the initial symptom of IgAN, has been associated with disease progression to chronic kidney disease in pediatric patients in Korea [12]. Recent findings indicate that both hematuria and proteinuria are critical indicators of disease activity and prognosis [38]. Given the widespread use of mass urine screenings in Korea, which facilitate early detection, further research is needed to assess how these initial symptoms influence the long-term outcomes of pediatric IgAN patients.
This new model with several additional variables had an iAUC value of 0.7 and exhibited superior performance to the international prediction tool; despite utilizing fewer derivation and validation cohorts compared to typical model development practices, our model demonstrated satisfactory predictive capability. Surprisingly, the iAUC value in the validation cohort was even higher. From a clinical perspective, our model revealed a statistically significant interaction between the use of RAS inhibitors and proteinuria, demonstrating the well-established fact that the risk of disease progression is higher in patients whose proteinuria remains uncontrolled despite medication [19,29]. However, further research and external validation are necessary due to the small number of patients enrolled in the validation cohort.
This study has limitations, including its retrospective nature, which could have resulted in selection bias away from general Korean IgAN children. Moreover, development and validation of an appropriate model were challenging with a sample size of approximately 600 patients, especially since the number of events in the development model was small compared to typical model development, and barely approached an iAUC value of 0.7. However, incorporation of clinically relevant risk factors in our new model and assessment of model performance in Korean children, who were not included in the original model development study, are strengths of our study. Finally, another important point is that one of the key variables, RASB and immunosuppression, was measured based on information either prior to the biopsy or within 1 year of the biopsy. This makes it difficult to determine the exact treatment timeline, and the resulting mix of pre-treatment and post-treatment biopsy data is a significant limitation. Meanwhile, the recent IgAN prediction tool has been updated to utilize 1-year post-biopsy findings [39]. We expect that incorporating this in future studies will allow for more robust and reliable analyses.
In conclusion, the updated International IgA Nephropathy Prediction Tool for children demonstrated suboptimal prediction ability in Korean IgAN children. Our proposed model showed acceptable prediction ability in the Korean population and warrants external validation. For the international prediction tool in children to be truly incorporated into clinical practice guidelines, validation across various races is necessary, and our newly created tool requires further verification in a larger Korean population.
Supplementary Materials
Supplementary data are available at Kidney Research and Clinical Practice online (https://doi.org/10.23876/j.krcp.24.262).
Notes
Conflicts of interest
All authors have no conflicts of interest to declare.
Data sharing statement
The data presented in this study are available from the corresponding author upon reasonable request.
Authors’ contributions
Conceptualization: HGK, JSS, JHK, YHA
Data curation: PGP, JK
Formal analysis: PGP, JK
Writing–original draft: All authors
Writing–review & editing: All authors
All authors have read and approved the final manuscript.