Phys. Ther. Korea 2020; 27(3): 212-219
Published online August 20, 2020
© Korean Research Society of Physical Therapy
Department of Physical Therapy, College of Health and Welfare, Woosong University, Daejeon, Korea
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0)which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background: Although the original version of the health-related quality of life (HRQOL) questionnaires are found to be acceptable, the cross-culturally adapted versions may not be comparable to their original version.
Objects: To examine dimensionality and construct validity of two Korean versions of the brief version of the World Health Organization Quality of Life (WHOQOL-BREF) and EuroQOL-5 dimension (EQ-5D) questionnaires.
Methods: A total of 77 cancer survivors undergoing palliative rehabilitation programs from two rehabilitation institutes was recruited from April 16, 2018 to June 26, 2019. The WHOQOL- BREF and the EQ-5D were filled out by the various cancer survivors following a particular session of rehabilitation programs. The scores were analyzed with Winsteps Rasch analysis computer program using the rating scale model. Rasch fit statistics were used to determine the dimensionality and the item difficulty calibrations of WHOQOL-BREF and EQ-5D.
Results: All items except two, negative feeling, need treatment function and pain prevent activity (mean square [MnSq] = 2.42, 1.82 and 2.51, respectively), were found to be acceptable, while two items of the EQ-5D, anxiety/depression and self-care, were misfit (infit MnSq = 1.65 and 0.38, respectively). Item difficulty calibrations of WHOQOL-BREF match person ability measures (i.e., HRQOL) fairly well. However, the person ability distribution showed obvious ceiling effects for EQ-5D. All items of EQ-5D were appeared to be less challenged in comparison with those of WHOQOL-BREF.
Conclusion: Item-level analysis using the Rasch model supports the quality of culturally adapted items used to measure the HRQOL one exception; that is, whether or not to include misfit items as part of the HRQOL measurements. Additionally, cancer survivors undergoing palliative rehabilitation programs appear to have more of a tendency to view the EQ-5D items as being more challenging than the WHOQOL-BREF.
Keywords: Cancer survivors, Health related quality of life, Palliative medicine, Psychometrics
The latest worldwide cancer statistics estimated 9.6 million deaths in 2018 and reported an increasing burden of care resulting from various cancers. It is one of the most costly conditions of global population with the economic impact exceeding 1.16 trillion dollars in 2010 in the United States . According to the national cancer control institute in Korea, the average 5-year survival rate of individuals with various cancer conditions increased about 69.4% in 2017 . Despite potential improvements in the survivorship, these cancer survivors are generally susceptible to have some limitations in their health-related quality of life (HRQOL). Yet they are three times more likely to report their limitations during the course of survivorship .
In general, palliative rehabilitation program (PRP) controlling negative symptoms includes approaches primarily intended to enhance the HRQOL of the cancer survivors. This has been well evidenced in many peer-reviewed journals [3-7]. The primary goals of PRP are now acknowledged to include concept of the HRQOL addressing the palliative care of cancer survivors [8,9]. The HRQOL questionnaires often include global ratings of health status and multidimensional status of HRQOL. These measures often deal with a broad spectrum of health concepts and are intended to provide scores that are sensitive to all ranges of the concept being measured. By contrast, some measures are designed to assess the aspects of health status resulting from a particular pathology and to view the attribution of functional limitations to the specific condition [4,5].
Most commonly used HRQOL questionnaires, if not all, have often been noted for being psychometrically narrow, whereas the HRQOL is a multidimensional health-oriented concept encompassing aspects of health or health care, such as functioning and symptoms . Therefore, the outcome measures require underlying constructs reflecting particular goals of the types of rehabilitation programs, such as improving the HRQOL before death, controlling symptoms and supporting other factors . During the course of cancer survivorship, it is imperative to monitor how the programs impact the HRQOL and how optimally assess any variation of the HRQOL over time in suitable manner.
Of the most widely accepted HRQOL questionnaires, the brief version of the World Health Organization Quality of Life (WHOQOL-BREF) and EuroQOL-5 dimension descriptive system (EQ-5D) are considered to be optimal for cancer survivors in many peer reviewed journals in regards to construct validity and psychometric adequacy [7,11-13]. Cross-culturally adapted versions of them were validated in many peer-reviewed journals and proved to be reliable, also responsive [12,14,15]. Since these two questionnaires were simultaneously developed in many countries, such approaches offered intriguing a possibilities warranting the cross-cultural adaptation related validity of the measure. However, as demand for the measure grew in countries where no language version of the measure existed, it naturally developed many translated versions. Accordingly, in such cases, the most appropriate version has been used as a starting point for the adaptation and updated by many authors.
Numerous classical test theory (CTT)-based HRQOL measures for cancer survivors have been developed and cross-culturally adapted into other languages. With almost endless array of translated HRQOL measures attempted to maintain psychometrical properties after cross-cultural adaptation procedure, often there is little distinction between the versions of the measures [13-16]. While designed to assess on-going difficulties on the dimension of HRQOL, cross-culturally adapted versions of the measures often fail to maintain the dimensions referred to in the original version . An evidence was obtained for a multidimensional perspective of the HRQOL in regards to construct validity. That is, the measurement structure accounts for construct validity in item responses either due to factors underlying the data or mistakenly treated latent traits they are intended to measure .
In addition, despite the myriad existing HRQOL measures, most investigators are often faced with a prevailing challenge of selecting an optimal measure. Although most CTT-based questionnaires have adequate psychometric properties, they may be insensitive either to a broad range of HRQOL level or to the actual variations that result from particular cancer experiences. One of the reasons is that these questionnaires are commonly developed to target the average person, hence the questionnaires are more likely to be sensitive at center than at both extremes in the ability range . Consequently, these measures often demonstrate ceiling effects when these measures are administered to persons with low ability score and floor effects when administered to persons with high ability score [18-20].
The purpose of this study was to compare dimensionality and construct validity of two Korean version of WHOQOL-BREF and EQ-5D questionnaires.
A total of 77 cancer survivors undergoing palliative rehabilitation programs from an oriental medicine hospital in Busan and a rehabilitation hospital in Daejeon, Republic of Korea were recruited from April 16, 2018 to June 26, 2019. Potential subjects consisted of all appropriate clients undergoing the palliative rehabilitation program at the participating sites during the period. All participants received detailed information on the present study including informed consent and potential conflict of interests of which non-participation in the study may not influence whether the palliative rehabilitation programs terminated. Subjects were asked to sign an informed consent and the Korean version of the WHOQOL-BREF and the EQ-5D approved by the Institutional Review Board of College of Health and Welfare, Woosong University (approval No. 1041549-190114-SB-70). The Korean versions of the WHOQOL-BREF and the EQ-5D [21,22] were administered (i.e, WHOQOL-BREF was followed by EQ-5D) upon the last entry of the first bout of rehabilitation program into physical therapy service and was subsequently requested from the survivors at discharge following the completion of the program. Of the participants who completed two measures, more than 35 percent were male (n = 35) and less than 65 percent (n = 42) were female with an average age of 52.9 ranged from 35 through 77. In addition, forty-six percent of the participants were diagnosed with breast cancer and fifty-four percent were diagnosed with various cancers (i.e., cancers on colon, lung, stomach, pancreas, kidney, prostate, liver, ovary, lymphatic system related, tonsil, bile duct and brain).
Scores were analyzed with the Winsteps software program version 3.57.2 (Wisteps.com, Chicago, IL, USA) using a rating scale model [23,24]. Rasch fit statistics and item difficulty calibrations were examined to determine the dimensionality of the Korean version of the WHOQOL-BREF. The construct validity of the Korean version of the WHOQOL-BREF was visually examined using the person-item map. The Rasch model transforms raw scores into the estimate of person ability (i.e., the level of HRQOL) and item difficulty (i.e., more or less challenging items) in logits. All descriptive statistics were calculated using the SPSS software version 25.0 (IBM Co., Armonk, NY, USA).
The Winsteps software program provides goodness-of-fit statistics for each item and subject. These fit statistics were examined to detect items that did not fit the Rasch rating scale model criterion of dimensionality (or unidimensionality). The Infit and outfit mean square (MnSq) ≥ 1.4 and ≤ 0.6 and a standardized score greater than 2.0 were considered to be misfit, which is an indication that the particular item or survivor responded in unexpected ways . The unexpected response indicated by the fit statistics suggests that the particular item is measuring another construct rather than the HRQOL. The program also generates a log odds unit (i.e., logit) scale to present the survivor’s level of HRQOL and item difficulty. The logit scale is based on the probability of getting success over the probability of getting failure on a response category of each item. The logical fashion of the analysis places test items along a continuum of the most to the least in difficulty. The hierarchical order of item difficulty provides a means to examine the construct validity of the instrument .
To determine how well the EQ-5D and the WHOQOL-BREF represent the construct of HRQOL, data were analyzed by exploratory factor analysis. A correlation matrix in which inter-correlations between individual test items was used respectively. The dimensionality of the matrix can be reduced by inspecting the items highly correlating with a group of other items. However, the matrix may represent low correlations with items outside of that group . In a nutshell, these items with high correlations could measure the trait being measured well, which is called a “factor”. The obtained factor creates a new dimension visualizing classification axes along with test items can be plotted. This projection of the scores of the original test items on the factor leads to two concepts: factor score and loadings. Factor scores are the scores of a subject on a particular factor, while the loadings are the correlations of the original test items with a factor. The factor loadings are now especially useful in determining the substantive importance of a particular item to a factor . A rotated factor analysis was used to compute maximizing the loadings from the original factor matrix with eigenvalues greater than 1. In addition, greater than 0.40 as a significant loading was used.
Rasch fit statistics were examined to determine the dimensionality of the Korean version of WHOQOL-BREF and EQ-5D. Using the criterion of misfit as MnSq > 1.4 and ZSTD > 2.0, the Table 1 presents item measures, infit/outfit statistics for the 26 WHOQOL-BREF items. All items, except the negative feeling, need treatment function and pain prevent activity items, show acceptable fit statistics (Table 1). For the EQ-5D items, all items, except the anxiety/depression and self-care items being slightly misfit, present acceptable fit statistics (Table 2).
To further examine the dimensionality of the WHOQOL-BREF, exploratory factor analysis was applied to the WHOQOL-BREF questionnaires. Table 3 presents factor loadings of the WHOQOL-BREF with a total of 7 constructs addressing HRQOL domains. Furthermore, the factor analysis revealed that a total of 22 items loaded on factor 1 and other items loaded either one or two factors (Table 3). Unlike the underlying 4 constructs originally designed, the factor analysis did not support the 4 constructs for the WHOQOL-BREF.
The person-item match analysis presents the relationship between the survivors’ various levels of HRQOL and item difficulty measures in logits for the two questionnaires. For the WHOQOL-BREF questionnaire, item difficulty calibrations were fairly well matched to the survivors’ levels of HRQOL throughout the whole range with some ceiling and floor effects. However, the EQ-5D showed a serious gap which was unable to effectively measure most cancer survivors at the medium and high levels of HRQOL. The items of the EQ-5D were only able to measure the individuals at the lower levels of HRQOL (Figure 1).
Construct validity of two well-established HRQOL questionnaires was tested using the Rasch-fit statistics and exploratory factor analysis. Overall, the Korean version of the WHOQOL-BREF showed acceptable construct validity except for few erratic items (i.e., negative feeling, need treatment function and pain prevent activity items of the WHOQOL-BREF, Anxiety/depression and self-care items of the EQ-5D with slight high/low fit). The exploratory factor analysis revealed that a total of 24 items loaded on factor 1 and other items loaded either one or two factors. Consequently, the WHOQOL-BREF showed nearly a single underlying construct. Furthermore, item difficulty hierarchy of the WHOQOL-BREF was fairly well matched to the HRQOL levels of the survivors throughout the whole range despite some ceiling and floor effects. However, item difficulty hierarchy of the EQ-5D showed a serious gap which was unable to measure all cancer survivors at the medium and high QOL levels. The items of the EQ-5D were only able to measure the individuals at the lower levels of HRQOL.
Items fit the Rasch model based on the probability of getting a rating on each item. For example, a cancer survivor with high level of HRQOL would be expected to have more difficulty on challenging items (i.e., WHOQOL-BREF item 26 through item 2 in Table 1) than less challenging items (i.e., WHOQOL-BREF item 7 through item 22 in Table 1). Similarly, a cancer survivor with low level of HRQOL would be expected to have even more difficulty on those challenging items than the survivors with high level of HRQOL. For the EQ-5D questionnaire with 5 items, the pain/discomfort item is the most challenging, while self-care item is the least challenging. Likewise, a survivor who is having difficulty on the self-care item would be expected to have more difficulty on the pain/discomfort item. However, these logical fashions are not demonstrated by the self-care item of the EQ-5D questionnaire as well as the negative feeling and pain prevent activity items of the WHOQOL-BREF (i.e., misfit items).
The World Health Organization (WHO) defines QOL as “individuals’ perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns” . Although the multidimensional concept of QOL has increasingly become a focus in the fields of health care, it is not clearly defined. The WHOQOL-BREF, a brief version of QOL questionnaire developed by the WHO, provides insights into the impact of all aspects on QOL as well as on Health-Related QOL. In an effort to focus on the assessment of health and QOL, the term, health-related QOL is now widely accepted and used. The HRQOL, in general, can be impacted by the notion of which presence of particular disease does not inevitably mean poor QOL . The implication of this is that various cancer survivors with such an experience of cancer related conditions may perceive their HRQOL as better than healthy individuals as long as palliative rehabilitation program goes successfully. Thus, it is advisable for clinicians to continually monitor their status of the HRQOL.
The construct of HRQOL questionnaire commonly changes and is unable to maintain the dimensions referred to in the original measures when a questionnaire is cross-culturally adapted . Traditionally, investigators establish construct validity of a measure by correlating it with a number of other measures by inspecting the pattern of the correlations, in which the measure is associated with these variables in predictable ways . However, a previous study, in a review of dimensionality problems with versions of the WHOQOL-BREF questionnaire, encouraged using item level analysis in evaluating dimensionality issues for validation procedure for the instrument [6,24,28]. The authors indicated that the nature of sample dependence often causes the dimensionality change which was explained as a dynamic construct model. That is, when translating a questionnaire into other languages, the questionnaire is more likely to be subject to change over time. To overcome this psychometric property limitation, item level analysis using item response theory is recommended to be used by focusing on the items of a questionnaire instead of the questionnaire as a whole . This study focused on items of the two well-known HRQOL questionnaires and attempted to determine whether the latent structure accounted for a single construct purported to measure by the items can be confirmed.
The present study carries some inherent limitations because the Rasch analysis was applied to a possible multidimensional structure of the HRQOL and exploratory factor analysis was applied without optimal sample sizes. It is unknown whether the determination in the fit statistics and the factor loadings was optimal. In addition, the present study includes the misfit items for all analyses due to the items’ critical roles within the questionnaires. In addition, the study subjects with various stages of cancer related conditions were recruited from two institutions where different palliative rehabilitation programs were carried. These factors could make substantial differences towards the levels of HRQOL. Future research is needed to investigate the effects of excluding the problematic items to confirm a single construct of the HRQOL.
Item level analyses using the Rasch model (1-parameter item response theory model) supports some psychometric properties of culturally adapted versions of the WHOQOL-BREF and the EQ-5D except two erratic items for each questionnaire. Whether or not to include the misfit items as part of the two questionnaires, cancer survivors undergoing palliative rehabilitation programs appear to have more of a tendency to view the WHOQOL-BREF items as being more challenging than the EQ-5D.
This research is based on the support of 2020 Woosong University Academic Research Funding.
No potential conflict of interest relevant to this article was reported.