Introduction
During the past three decades, the number of people undergoing maintenance dialysis globally has increased dramatically [1]. According to the Global Burden of Disease study, 697.5 million individuals worldwide had chronic kidney disease (CKD) in 2017, and 1.2 million of them died [2]. Korea is no exception; in 2019, the end-stage renal disease (ESRD) population was more than 100,000, doubling over a 10-year period [3].
CKD is not only a public health issue but also creates an economic burden. According to the 2014 National Health Insurance Statistical Yearbook (Korea), the number of hospital visits due to CKD increased from ~150,000 in 2013 to ~160,000 in 2014, and the corresponding medical expenses also increased [4]. CKD has the highest medical cost per person among chronic diseases, and hemodialysis (HD) and peritoneal dialysis (PD) cost 87 and 65 times more than hypertension treatment, respectively [5].
People with CKD and ESRD have a poorer health-related quality of life (HRQOL) than the general population [6,7]. In patients with ESRD, low HRQOL is associated with a higher risk of death and hospitalization, which affects morbidity and mortality in patients with ESRD [6–8]. However, little is known about the relationship between low HRQOL and adverse outcomes in patients with predialysis CKD.
Patients whose advanced CKD is approaching ESRD face complex medical decision-making regarding the type of medical therapy they wish to pursue. More than a decade ago, the Renal Physicians Association and the American Society of Nephrology recommended a shared approach to decision-making for all patients with ESRD [9], which is supported by evidence that shared decision-making can improve patient outcomes [10]. Despite shared decision-making emerging as a pillar of national and international quality of care standards and policies [11], evidence shows that people suffering from CKD still have limited involvement and participation in treatment decision-making [12,13].
Effective interventions to guide patients in decision-making include decision aids (DAs) and shared decision-making [14]. Patients exposed to DAs are more likely to be informed, have realistic expectations of option outcomes, participate actively in decision-making, and feel lower decisional conflict [15]. In this context, there is growing interest in developing, implementing, and further strengthening the quality of decision support provided to patients and families living with CKD. However, relatively little is known about how to achieve this goal.
The present research aims to develop self-assessment items for a DA by employing both qualitative and quantitative methods. We then seek to confirm the robustness of this study by examining the multifaceted validity of the newly developed scales for a DA.
Methods
Study 1: scale development
The aim of Study 1 was to develop DA scales for patients with CKD and to examine the content validity of the DA’s self-assessment items.
Study design
A flow chart of our study is shown in Fig. 1. First, we conducted five focus group interviews (FGIs) consisting of two groups undergoing HD, two groups undergoing PD, and one group of people facing a modality choice decision. Each group consisted of five patients, and interviewees volunteered to participate from two university medical centers; Ewha Womans University Medical Center and Severance Hospital. Based on the literature reviews of existing DA and communication expert consultations, 40 draft items were selected to undergo preliminary testing for further item selection and dimension exploration. Five nephrologists then reviewed the selected items, and two were excluded because they did not apply to the Korean population. Subsequently, items derived from the literature review and extracted from the FGIs were refined, and content validity was examined. Medically inappropriate items were deleted following content validity testing, and finally, 35 self-assessment items remained.
Procedure: item generation
A systematic literature review on international renal guidelines and other dialysis DA booklets was performed as the initial step in item procurement. Next, we solicited feedback from health communication experts regarding previously developed DA materials. Although there were many aspects that could be generally agreed upon from the review of existing DA materials, some elements did not fit the Korean context. Therefore, considering adequate self-assessment items for decision-making are culturally dependent, an in-depth qualitative study was planned to fulfill the objective of extracting specialized items for CKD patients in Korea. FGIs were conducted by a trained facilitator and a study team member using an interview guide developed by study team members with expertise in health communication. The interview questionnaire consisted of open questions not only relevant to health and lifestyle but also pertaining to other factors such as how patients think of the relative benefits and barriers of dialysis modality, dilemmas in undergoing the chosen treatment, satisfactory communication with attending doctors, and so on.
A qualitative interview study with five focus groups was performed from October 8, 2019 to October 31, 2019, at Ewha Womans University Medical Center and the dialysis center at Severance Hospital in Seoul. Interviews were conducted according to a semi-structured interview protocol that presented a series of questions designed to 1) identify interviewees’ demographic and cultural background, 2) articulate their illness perception and elicit ownership of their CKD management, and 3) explore the CKD patients’ communication traits and factors affecting doctor-patient communication. The interviews were recorded, and all interviewees gave consent to the analysis. The transcribed interviews were read repeatedly, and remarks that were determined to be relevant to each investigated phenomenon were recorded separately by each researcher. Through the interpretation and mutual exchange of opinions from the health communication researchers, the concept was refined, and the basis for deriving DA questions was formed.
Based on an extensive review of the DA materials in the field of CKD and the FGIs with patients, we generated an initial pool of 40 self-assessment items reflecting various aspects of personal orientation.
Study 2: scale evaluation
The aim of Study 2 was to evaluate the construct validity of the newly developed self-assessment DA items for patients with CKD. A sample of 330 participants was recruited from seven university hospitals to validate the properties of the items. Inclusion criteria were patients age 19 years or older who had undergone HD or PD for more than 3 months. Exclusion criteria were patients who were not able to participate in the survey due to cognitive impairment or psychological illness.
Procedure
An evaluation of the DA instrument was performed as follows. Patients with CKD undergoing dialysis at seven hospitals in Seoul were recruited from March 2020 to September 2020 to participate in preliminary testing of the DA instrument, containing 35 items. A total of 152 HD and 178 PD patients who met the eligibility criteria approved by institutional review board participated and completed the DA questionnaire.
Ethical considerations
This study was approved by each hospital’s Institutional Review Board (Ewha Womans University Seoul Hospital, No. 2020-03-018; Seoul National University Bundang Hospital, No. B-2004-604-307; Ulsan University Hospital, No. UUH 2020-06-009; Gil Medical Center, No. GFIRB2020-244; Seoul National University Boramae Medical Center, No. 06-2020-0036; Dongguk University Medical Center, No. DUIH 2020-05-011; and Severance Hospital, No. 4-2020-0422).
Statistical analysis
Statistical analyses were performed using IBM SPSS version 26.0 (IBM Corp., Armonk, NY, USA) and IBM SPSS AMOS version 21.0 (IBM Corp.). Exploratory factor analysis (EFA) was employed to identify the underlying relationships between measured variables (DA questions) and to extract their common dimensions. Based on the EFA results, confirmatory factor analysis (CFA) was performed to verify construct validity. Additional analysis was performed to examine the predictive validity of the newly developed scales using structural equation modeling (SEM).
Results
In total, 330 patients completed the instrument (DA questionnaire). Their mean age was 48.5 ± 12.8 years. The sociodemographic details of the study participants are summarized in Table 1.
Content validity
Content validity is usually qualitatively judged by experts; however, in this study, content validity was assessed by calculating the content validity index (CVI) for each item based on experts’ ratings of item relevance [16,17]. Here, CVI was assessed in terms of the item-level CVI (I-CVI), calculated as the proportion of experts assigning a rating of 3 (quite relevant) or 4 (highly relevant) among the total number of experts. In most cases, an I-CVI of 1.00 when there are five or fewer experts and an I-CVI greater than 0.78 otherwise are generally acceptable [16–18].
Out of 40 preliminary items, 37 were represented by I-CVI scores distributed between 0.80 and 1. Three items that did not meet the I-CVI criteria were deleted. The deleted items were “I am worried about the cost of dialysis,” “I am living with my elderly parents or in-laws,” and “I am so restless that I cannot lie still.” Two items that were not medically appropriate, “I want to be pregnant to be a mother” and “My sex life is important to me.” were also deleted. Through this refining process, 35 preliminary items were set up.
Exploratory factor analysis
Prior to conducting an EFA, we examined two indicators to determine whether the sample was appropriate for such an analysis. The Kaiser-Meyer-Olkin measure of sampling adequacy index was 0.62, and Bartlett’s test of sphericity was significant at χ2 (degree of freedom [df] = 595, n = 330) = 2,821.37 (p < 0.001), indicating that the sample was appropriate for the analysis [19]. We performed an EFA using the principal components analysis extraction method with a varimax rotation because varimax rotation simplifies factor loadings by removing the middle ground and more specifically identifies the factor on which data load [20]. The EFA showed 13 factors, explaining 65.3% of the variance in the data; however, the number of factors retained from the EFA seemed to be too large and inadequate as the smallest number of possible factors. Thus, EFA by principal component analysis was performed again based on three fixed factors. Each factor consisted of 10 to 12 items with Cronbach alpha (α) values ranging from 0.62 to 0.72. Next, an independent t-test was conducted to determine the difference between the two groups, according to the dialysis modality, through the average difference for each item. Table 2 lists all scale items and their properties.
Confirmatory factor analysis
CFA was performed on the 35 items based on the entire sample of 330 patients. Following the recommendations of Cole [21], Cuttance and Ecob [22], and Marsh and Bella [23], the DA scale’s goodness-of-fit was evaluated using multiple criteria: the goodness-of-fit index (GFI), the adjusted GFI, the root mean square error of approximation (RMSEA), the incremental fit index (IFI), the comparative fit index (CFI), and the Akaike information criterion [24]. Multiple criteria were used because each index has different strengths and weaknesses in assessing goodness-of-fit between a particular model and the observed data.
To assess the construct validity of the new scale, we estimated a series of CFA models using IBM SPSS AMOS [25]. The three-factor model identified via EFA initially consisted of three first-order latent variables, representing the following three scales; health-related values (12 indicators), lifestyle-related values (11 indicators), and dialysis environment-related values (12 indicators). In the first analysis of three first-order latent variables, model fit indices showed poor fitness; thus, we deleted some measured variables with low standardized path coefficients. Although the standardized estimate appeared low (>0.40), some measured variables were retained because they showed significance in the t-test, meaning that the item should be related to the decision-making choice for dialysis modality. This model was then refined to three scales with four indicators each.
Therefore, we performed CFA three times to obtain the best fit. A comparison of these alternate models revealed a significant difference in the fit indices, showing the best fit for the three-factor model, with a χ2 value (102.212, df = 51) significantly lower than those for the 13-factor (χ2 = 946.551, df = 417) and one-factor (χ2 = 2,255.458, df = 560) models. All the other fit indices (RMSEA = 0.05, GFI = 0.95, AGFI = 0.92, CFI = 0.91, IFI = 0.91, and χ2/df = 2.00) also showed good fit for the three-factor model based on the cut-off values recommended by Hu and Bentler [24], Kline [26], and Wheaton et al. [27]. The fitness changes of the alternative models, the best fit model, and the model fit of the HD and PD patient groups are shown in Table 3.
Convergent and discriminant validity
Construct validity was examined in terms of convergent and discriminant validity. Convergent validity is a subcategory of validity tests that can assist in establishing construct validity [28] and refers to the degree to which two measures of constructs, which theoretically should be related, are related. In contrast, discriminant validity tests whether concepts or measurements that are supposed to be unrelated are, in fact, unrelated [29].
The Fornell-Larcker [30] criterion has been commonly used to assess convergent and discriminant validity, and it can be assessed considering the average variance extracted (AVE) and composite reliability (CR). AVE measures the level of variance captured by a construct versus the level captured due to measurement error; values above 0.7 are considered very good, while a level of 0.5 is acceptable. CR is a less biased estimate of reliability than Cronbach α, and the acceptable values of CR are 0.7 and above. The AVE and CR can be calculated as follows:
Here, k is the number of items, and λi is the factor loading of item i.
As indicated in Fig. 1, four items have loaded onto each of the single latent factors, with standardized path coefficients ranging from 0.19 (item L2) to 0.90 (item H1) in the three-factor DA model. When evaluating the strength and adequacy of item loadings, some items and their expectedly related latent factors demonstrated poor convergent validity. For a more rigorous analysis of convergent and discriminant validity, the AVE and CR of each latent variable were calculated based on the formula given by Fornell and Larcker [30]. The AVE of health, lifestyle, and dialysis environment were found to be greater than 0.50, which is considered acceptable. The CR of each latent variable was also greater than 0.7, which can be said to signify convergent validity and internal consistency.
Discriminant validity was assessed by comparing the AVE and correlation coefficients. According to Fornell and Larker [30], when the AVE is greater than the square of the correlation coefficient between each factor, discriminant validity is secured. As a result of the verification, the AVE obtained between each factor was greater than the square of the correlation coefficient between each factor; thus, discriminant validity was secured (Table 4).
Predictive validity
A multigroup analysis of two patient groups was performed using SEM to determine whether the three decision support factors (health, lifestyle, and dialysis environment) could predict a patient’s choice of dialysis modality. Based on the literature on illness perception and dialysis modality decision-making, we hypothesized a causal relationship between decisional factors, CKD perception, and shared decision-making cognition [31,32]. More specifically, we forecasted differences in factors influencing decision-making between HD and PD patients, and we predicted these differences to be related to the perception of CKD as an illness and, ultimately, to affect the perception of shared decision-making processes.
As expected, a difference was found between the HD and PD patient groups. For the HD patient group, the dialysis decisional factor exhibited a positive association with illness perception (β = 0.60, p < 0.01) and illness perception was positively associated with perceived shared decision-making (β = 0.57, p < 0.05). However, the direct effect of decisional factors on perceived shared decision-making was not statistically significant. In the PD patient group, a positive relationship between the decisional factor (β = 0.50, p < 0.01), illness perception (β = 0.29, p < 0.05), and perceived shared decision-making (β = 0.50, p < 0.01) was found. Through the path difference between the two groups, which was statistically significant, the self-assessment items developed through this study can be said to have predictive validity. The results of the HD patient group analysis and the PD patient group analysis are shown in Fig. 2 and 3, respectively. The final items that have been validated are shown in Table 5.
Discussion
A recent review of DA validation demonstrated an increase in patients selecting options related to their values and less decision-making passivity, as well as lower decisional conflict and increased knowledge levels [33]. In other words, DAs increase participation in shared decision-making. As proposed by the Kidney Disease Improving Global Outcomes guidelines, dialysis modality should be chosen with timely and shared decision-making among the healthcare team, patients, and their caregivers [34]. In particular, the decision to undergo dialysis depends on many abstract aspects, such as knowledge of dialysis; personal beliefs; and feelings toward life, suffering, death, and other patient experiences [13,32,35,36]. Moreover, how this specific illness is viewed and understood in any given society is crucial to medical decision-making because patients’ social and cultural backgrounds highly influence their decision-making.
In this study, we developed Korea-specific DA scales for ESRD by integrating both qualitative and quantitative research methods. Some noteworthy findings from this study are as follows.
First, the content of the DA questionnaire was classified into three dimensions; health, lifestyle, and dialysis environment. Though the correlations between these three dimensions were statistically significant, they were not robust in effect size. These results imply that factors influencing decision-making are sometimes unrelated to each other, existing as independent factors in many cases. In other words, since there are various factors that influence decision-making, the communication process for shared decision-making should be set as specifically as possible and thoroughly include all possibilities regarding dialysis treatment.
Second, the difference between HD and PD patients in terms of health, lifestyle, and dialysis environmental factors was statistically significant. This implies that the importance of health and dialysis environmental factors is more emphasized in HD patients, while factors related to lifestyle may be considered more important to PD patients.
Third, in this study, we employed a multifaceted approach to testing the validity of the scales and examined the relationship between decisional factors, illness perception, and perceived shared decision-making. Interestingly, in HD patients, it was found that illness perception completely mediated the influence of decisional factors on shared decision-making. Meaning that the more the HD patient agrees with the decisional factor, the more negative the illness perception, and the more negative the illness perception, the more the patient recognizes shared decision-making. In the case of PD patients, illness perception partially mediated the relationship between decisional factors and perceived shared decision-making. Meaning that illness perception is not the only thing that affects perceived shared decision-making as is in HD patient group. Therefore, it is no doubt that patients’ decisional factors are crucial to such decision-making, but it can be equally said that decisional factors work somewhat differently between HD and PD patients.
Our study has several limitations. First, the design is that of a formative study, including both qualitative and quantitative methods. The qualitative study participants were all recruited from the university hospital medical center, while in-center dialysis patients were not included. Therefore, in a follow-up study, it is necessary to obtain cross validity of the newly developed DA items by examining whether there are any differences in decisional factors between patients taking HD at university medical centers versus other HD centers. Second, the DA items developed and validated in this study are indicators of a construct, which is an essential part of the complete DA. Therefore, further refinement and implementation will improve and contribute to shared decision-making using these DA items.
Nevertheless, this is the first attempt to develop and validate DA items, and the study results highlight a critical need for initiatives to encourage subsequent studies to improve shared decision-making.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Print
