- Research article
- Open Access
Development and validation of a screener based on interRAI assessments to measure informal caregiver wellbeing in the community
BMC Geriatrics volume 18, Article number: 310 (2018)
Informal caregivers are invaluable partners of the health care system. However, their caring responsibilities often affect their psychological wellbeing and ability to continue in their role. It is of paramount importance to easily identify caregivers that would benefit from immediate assistance.
In this nonexperimental cohort study, a cross-sectional analysis was conducted among 362 informal caregivers (mean age 64.1 years, SD ± 13.1) caring for persons with high care needs (mean age 78.6 years, SD ± 15.0). Caregivers were interviewed using an interRAI-based self-reported survey with 82 items covering characteristics of caregivers including key aspects of wellbeing. A factor analysis identified items in the caregiver survey dealing with subjective wellbeing that were compared against other wellbeing measures. A screener, called Caregiver Wellbeing Index (CWBI), consisting of four items with response scores ranging from 0 to 2 was created. The CWBI was validated in a follow-up study in which 1020 screeners were completed by informal caregivers of home care clients. Clinical assessments of the care recipients (n = 262) and information on long-term care home (LTCH) admission (n = 176) were linked to the screener dataset. The association between the CWBI scores and caregiver and care recipient characteristics were assessed using logistic regression models and chi-square tests. The reliability of CWBI was also measured.
The CWBI scores ranging from zero to eight were split in four ‘wellbeing’ levels (excellent, good, fair, poor). In the validation study, fair/poor psychological wellbeing was strongly associated with caregiver reports of inability to continue in their role; conflict with family; or feelings of distress, anger, or depression (P < 0.0001). Caregivers caring for a care recipient that presented changes in behavior, cognition, and mood were more likely to present fair/poor wellbeing (P < 0.0001). Additionally, caregivers with high CWBI scores (poor wellbeing) were also more likely to provide care for someone who was admitted to a LTCH (OR 3.52, CI 1.32–9.34) after controlling for care recipient and caregiver characteristics. The Cronbach alpha value 0.89 indicated high reliability.
The CWBI is a valid screener that can easily identify caregivers that might benefit from further assessment and interventions.
Informal caregivers, defined here as family members or friends caring for an adult with a chronic or disabling condition, are an integral part of the home and community care system in many countries. The care provided by informal caregivers is paramount for supporting those in need to remain at home for as long as possible and to avoid institutional care . Additionally, the provision of unpaid care results in substantial cost savings for the health system by off-setting paid home care service and reducing long term care home admissions .
However, informal caregivers are often at risk of poor health outcomes such as depression and physical pain, especially when caring for persons with high care needs [3,4,5]. The ability of informal caregivers to continue in their role is often threatened when they face increasing demands to provide care for individuals with complex health issues. Indeed, the sustainability of the home care system relies on the capacity of informal caregivers to function effectively in their role as providers of support.
Psychological wellbeing, defined as ‘the state of equilibrium or balance that can be affected by life events or challenges’  is often negatively affected by physical and psychological challenges of caregiving [5, 7]. This is a worrisome issue because poor psychological wellbeing has been associated with poor health and shorter survival in older adults . On the other hand, caregiving may also bring positive experiences that may have a protective role in health maintenance [9,10,11].
A major challenge for the health system is to measure caregiver wellbeing to identify those at risk of poor quality of life (i.e., psychological, physical, social, material wellbeing)  or health declines in order to provide appropriate supports to address their needs. Most available instruments to screen caregivers are impractically long for initial assessment, lack cut-offs points for intervention, or lack validity and reliability [13,14,15,16]. Moreover, most caregiver assessments are not designed to be compatible with assessment tools already in widespread use for assessment of home care clients.
This study uses established interRAI assessment systems ; specifically, the RAI Home Care (RAI-HC), the interRAI Community Health Assessment (interRAI CHA), the interRAI/Kendal Corporation Collage “Wellness” Assessment, and the interRAI Quality of Life Self-report Surveys for Home Care/Community Living, Senior Housing, and Mental Health . These assessments were developed by interRAI, an international group of researchers and clinicians with a focus on improving care for vulnerable populations (e.g., child and adult mental health, older adults in the acute care setting, residents in long-term care, adults in acute and longer-term community care). The RAI Home Care Assessment is a validated multidimensional assessment for adults  and is currently used in a number of jurisdictions in North America (Canada and multiple states in the U.S.), Europe (Italy, Switzerland, Finland, Estonia, etc.), and Asia/Pacific Rim (Hong Kong, Singapore, Japan, Australia, New Zealand) . This assessment includes items related to various domains: health, function, social support, diagnosis, services use etc. describing around 300 characteristics [20, 21]. Risk scales are derived from calculations using items in the assessment (e.g., Cognitive Performance Scale [22, 23], Depression Rating Scale , Method for Assigning Priority Levels (MAPLe) [25, 26]).The interRAI CHA is based on the interRAI HC assessment but is intended to be used by community support organizations to assess potential clients that are looking for day activities, meals, and personal support . The interRAI CHA is a modular assessment system and includes a core assessment (for all clients) and four supplements that are used to evaluate specific problems in depth as needed (functional, mental health, assisted living and deafblind assessments) . While not as extensively used as the RAI-HC or its newer version, the interRAI HC assessment, the interRAI CHA has been used in research or mandated for use in various jurisdictions (e.g., Finland, Hong Kong, New Zealand, Australia, Israel, and Ontario, Canada). The interRAI CHA core items are similar to those in the RAI-HC and risk scales can also be derived from the interRAI CHA .
Both assessments highlight issues related to functioning and quality of life for community-residing individuals and have common items that were used to describe and investigate the characteristics of the care recipient. Health care providers (e.g., social workers, nurses) are trained to perform these assessments using information obtained from the client, their caregiver, and medical records. Using these well-established assessments in the development of a caregiver screener allows for more accurate and representative evaluation of associations between caregiver and care recipient characteristics and needs.
The interRAI/Kendal Corporation Collage “Wellness” Assessment is a self-report assessment that addresses issues related to a person’s psychosocial, cognitive, health status, and daily functioning based on items from interRAI community assessments. The interRAI Quality of Life Self-report Surveys for Home Care/Community Living, Senior Housing, and Mental Health and are self-report surveys that address a number of issues, including, for example, self-determination, mood, social participation and community involvement, social ties, and belonging. These self-report assessments provided the researchers with established interRAI-based items on which to develop a self-report survey for caregivers.
Studies reviewing caregiver assessments point to lack of 1) evidence on psychometric properties, 2) information on sensitivity to change for several caregiver measures [1,2,3], and 3) theoretical approach to explain the concept behind the development of the assessment [27,28,29]. Also, because most assessments include multiple domains , they are not suitable to be used for screening purposes and the sensitivity to change of the instrument may be low if the instrument mixes items that represents different domains (e.g., objective and subjective burden) that could be affected differently by interventions. In other words, specific measures should be used to identify changes within significant caregiver outcomes (e.g., depression, psychological wellbeing) . While the vast majority of screeners reported in caregiver research measure caregiver burden and other negative aspects of caregiving [31,32,33], this study develops and validates a measure of caregiver psychological wellbeing that is influenced by both negative and positive experiences. Caregiver psychological wellbeing is a valuable measure to be assessed before and after interventions as it reflects how caregivers are coping with caregiving challenges or stressful situations [34, 35].
A short screener that captures key aspects of a caregiver’s psychological wellbeing would help organizations identify caregivers that would benefit from immediate assistance and further detailed assessment. Such a screening and further assessment, if warranted, could identify areas of need (e.g., depression), and allow for timely implementation of interventions to support the caregiver and evaluate changes in the caregiver’s wellbeing following implementation of interventions for the caregiver. The objective of this study was to develop and validate a screener, compatible with established care recipient assessments (i.e., interRAI suite) [17, 36], to measure caregiver psychological wellbeing and identify caregivers at risk of adverse outcomes.
This study involved two stages. The first stage, the development of the screener, is represented by a nonexperimental longitudinal cohort study that includes a baseline family caregiver assessment at program admission and one follow-up assessment 6 months later. The second stage, the validation study, is also a nonexperimental longitudinal cohort study where assessment data was collected simultaneously for the caregiver and care recipient. Information on care recipient discharge from home care to long term care home was also obtained for this study.
As an initial step, a Caregiver Survey was developed using items from the interRAI/Kendal Corporation Collage “Wellness” Assessment and the interRAI Quality of Life Self-report Surveys for Home Care/Community Living, Senior Housing, and Mental Health. All items from the Wellness Assessment and the Quality of Life self-report surveys were considered for the interRAI Caregiver Survey, with or without modifications. Over the course of a number of revisions, a pilot interRAI Caregiver Survey was finalized and approved by interRAI with a total of 82 items covering a range of domains including demographic information, physical and emotional health, caregiving responsibilities, well-being, supports, and quality of life. The interRAI Caregiver Survey was intended to capture characteristics of a caregiver, with the goal of identifying those characteristics that were most pertinent to caregiver wellbeing.
The informal caregivers in this study were participants of the Caregiver Recharge Service (CRS), a respite program for informal caregivers offered through the Mississauga Halton Local Health Integration Network (MH LHIN), in Ontario, Canada. The CRS was established to provide respite to caregivers by offering in-home assistance with activities they usually perform with the goal of reducing caregiver strain, improving caregiver wellbeing and increasing their ability to continue caring activities.
In order to participate in the CRS program, two sources of information were used. The Caregiver Strain Index (CSI), an assessment to identify caregivers experiencing strain [37, 38], was completed by the caregiver. The other source of information was derived from the care recipient’s interRAI CHA or RAI-HC assessment [17, 23], specifically the Method of Assigning Priority Levels (MAPLe) score. MAPLe is a decision-support tool with scores ranging from 1 to 5 used to identify community care recipients who have the most urgent need for care . It includes items dealing with behaviour issues, cognitive and activity of daily living impairment, falls, and other types of health problems that have been associated with caregiver distress and long-term care home (LTCH) admission . Caregivers eligible for the program included those who scored 9 or higher on the CSI assessment (maximum score of 13), indicating high strain, and were caring for a care recipient with high needs as indicated by a MAPLe score of 4 or 5 or receiving palliative care.
Participants in the study were recruited from three health service providers (HSP) involved in the Caregiver Recharge program: Links2Care, Home Instead, and Able Living. Staff from these organizations introduced the study to 560 caregivers during their first visit by providing the caregiver with a letter from the researchers explaining the study and then obtaining written consent to participate in the study from those caregivers who agreed to do so. Once consent was obtained, the research team was provided with the signed consent form, the caregiver’s phone number, and best time to contact them. These organization also provided the research team with care recipient assessments (interRAI CHA or RAI-HC) and discharge summary reports.
Trained volunteers of the research team interviewed caregivers on the phone using the the interRAI Caregiver Survey within 1 week of receiving the consent. A total of 362 caregivers (65% of 560 that were approached) completed the caregiver survey on the phone at baseline (i.e., CRS admission) between September 2013 and September 2015. Initially, caregivers cited lack of available time to complete the survey. In order to increase response rate and minimize bias related to inclusion of ‘less busy’ caregivers, the study team members offered more flexible interview times (e.g., before 8 AM, after 6 PM).
A total of 169 caregivers completed the survey at least 6 months after the first assessment. Most caregivers that did not complete the follow-up survey were no longer participating in the CRS program because the respective care recipient died or was admitted to LTCH. Also, some caregivers were unavailable to complete the survey (e.g., did not answer the phone or reported to be busy). The data on the Caregiver Survey was entered by the interviewer in an excel spreadsheet after the phone call. The care recipient assessments (i.e., interRAI CHA or RAI-HC) were sent electronically to the research team using standard and approved methods for secure transmission of data.
The RAI-HC is used by the home care provider of the MH LHIN to assess home care clients for determining needs and home care services. The majority of the referrals to the Caregiver Recharge program come from this organization (i.e., 60%) and as such, the care recipient’s RAI-HC was used in the study. However, as part of routine practice, care recipients in the Caregiver Recharge program that did not have a RAI-HC assessment were assessed using the core and functional supplement of the interRAI CHA by a trained member of the agency staff. Thus, these assessments were also available to the research team.
Care recipient assessments performed by an organization more than 1 month before there was a referral for the caregiver respite program were excluded from the study. Thus, only 262 caregiver surveys that were completed close to the care recipient assessment were included in the analysis to ensure that both assessments reflected the caregiving background situation. All palliative clients were also excluded from the analysis due to the low number of available linked caregiver assessments (n = 20). Full ethics clearance for this study was granted by the Office of Research Ethics, University of Waterloo (#18982).
The aim of this study was to develop a brief caregiver screener to measure psychological wellbeing of caregivers based on indicators of psychological wellbeing derived from the pilot interRAI Caregiver Survey. For identifying the items that best represent psychological wellbeing, an exploratory factor analysis (EFA) was used to examine the association between the items representing different domains in the survey (e.g., physical and mental health, social supports) and an underlying construct representing caregiver subjective wellbeing.
For data preparation before the factor analysis, results of frequency analyses of 531 caregiver surveys (first assessment: 362; reassessment: 169) were examined and a total of 24 items were removed based on low variability (that is, at least 85% of the sample sharing the same response for the same question). For example, some of the variables removed due to lack of variability were: ‘caregivers did not receive help with bathing’, ‘caregiver diagnosis of dementia’, ‘caregiver diagnosis of Alzheimer’s disease’, ‘caregiver reported hip fracture’, ‘caregiver reported any fracture’, ‘caregivers taking care of an adult family member or friend’. In addition, 14 items were removed because they were related to demographic information (e.g., gender, age, living arrangement, primary language, full time/part time job) leaving 44 items for analysis. According to Costeloo and Osborne (2005) a subject to item ratio greater than 10:1 (i.e., 440 sample size) is considered appropriate for exploratory factor analysis .
The random distribution of missing values was evaluated through the missing data pattern yielded by the procedure “MI” in SAS. The use of an imputation method to replace missing values with the mean of the variable was performed after determining the random distribution of missing values. Lastly, the scoring of reverse-phrased items was inverted.
A Kaiser-Meyer-Okin (KMO) score of 0.86 indicated that a factor, or factors, could be extracted from the dataset . The Barlett’s Test of Sphericity was performed to ensure that the dataset was appropriate for extracting factors. The null hypothesis (no factors) was rejected (P < 0.0001) confirming that factors could be extracted.
A parallel analysis (PA), a method based on Monte Carlo simulation, was performed to obtain the number of factors to be extracted. This method overcomes some of the limitations of the Kaiser’s method, or mineigen greater than 1 criteria, such as misleading results due to sampling error . A SAS Macro for parallel analysis was used  including 100 iterations. Oblique rotation was chosen as the most appropriate type of rotation for this analysis because it allows for correlation among factors , aligning with evidence on the interactions between variables representing different aspects of caregiver’s life [44,45,46].
The factor analysis was performed after removing items with factor loadings lower than 0.35. After identifying the factor representing the psychological wellbeing construct, the second step of the analysis was to confirm whether items with loading higher than 0.35 that were identified as part of factor 3 (psychological wellbeing) were correlated with measures that one would expect to be associated with psychological wellbeing – i.e., measure of strain (scores of Caregiver Strain Index) and depression. The latter is an item in the caregiver survey which states: ‘caregiver is currently receiving treatment or being monitored for depression’. For this analysis, scores from 1 to 5 were assigned for each response of the seven items in ‘Factor 3’ with high loading (bolded in Table 2), with higher values corresponding to higher frequency of the expressed feeling (e.g., never/not in the last 3 days, but do have the problem/1 day/ 2 days; and every day in the past 3 days). Only the interRAI-based Caregiver Surveys completed at baseline with a CSI completed within a 90 days interval between these assessments were used for this analysis (average time between CSI and survey = 37 days; SD 20.2, n = 286) to ensure that the CSI assessment score reflected the status of the caregiver at the time of the survey.
In the third step, a series of chi-square tests was performed to evaluate the association of the sum of different combinations of the selected items scores from the previous step with 1) CSI scores converted to a binary variable and 2) the ‘depression item’.
The next section describes the evaluation of screener cut-offs and validation of the Caregiver Wellbeing Index (CWBI) screener using two datasets respectively: 1) derivation dataset and 2) validation study dataset.
Evaluation of screener cut-offs
Logistic regression models were used to evaluate if the CWBI screener cut-off points were associated with different likelihood of caregiver related outcomes, including: 1) physician visits; 2) self-reported health; 3) personal outlook; and 4) the caregiver reporting that the care recipient would be better off elsewhere. Personal outlook is defined here as “unique patterns of thinking, feeling and interacting with ourselves” , related but not similar to psychological wellbeing. These outcomes were chosen because they are related to different domains of caregiver’s lives that are often related to subjective wellbeing [8, 48].
The dependent variable responses from the interRAI Caregiver Survey were collapsed as follows: 1) appointment with doctor or nurse practitioner in the last 90 days (any vs. none); 2) self-reported health (fair/poor vs. excellent/good); 3) caregiver reports that ‘on the whole my life is good’ (never, rarely or sometimes vs. most of time or always); and 4) caregiver reports that they ‘believe that their care recipient would be better off elsewhere’ (yes vs. no). All surveys completed at intake were included in this analysis (n = 362). The association of the screener score levels with the caregiver outcomes was evaluated after controlling for age and gender, which were included as covariates in the models. Both covariates were chosen because of evidence showing that caregiver experiences and related outcomes may differ according to age [49,50,51] and gender [51,52,53,54,55,56,57].
In the last step, chi-square analyses examined whether CWBI levels were also associated with other variables related to caregiver subjective wellbeing such as ability to continue caring and loneliness . The reliability of the screener was examined using standardized Cronbach alpha value.
Screener validation (based on follow-up study)
A test of validity of the CWBI was performed using informal caregiver and respective care recipient information derived from home care clients in a subsequent pilot study. Predictive validity of the screener was examined by evaluating whether the CWBI scores predicted care recipient LTCH admissions.
A total of 1020 primary informal caregivers of long-stay home care clients in Hamilton Niagara Haldimand Brant Local Health Integration Network (HNHB LHIN) completed the CWBI screener as part of routine assessments between April 01st, 2016 and March 31st, 2017. Only the CWBI completed within 3 months of the RAI-HC assessment for the care recipients with MAPLe 3 to 5 living in a private home or apartment were included in the dataset.
The RAI-HC assessments are available to the researchers based on license agreements with interRAI. To guarantee the anonymity of the data, the dataset did not include any personal identifiers. Information on care recipient institutionalization was obtained from the Client Health and Related Information System (CHRIS), a web-based care recipient management system that collects information on home care client admission and discharge.
Validation of the screener was performed by evaluating the association (i.e., chi-square test) between the CWBI scores and the caregiver related information that is captured in the RAI-HC items completed during the care recipient assessment that provides information on areas of help and caregiver status. These items were selected for the analysis because of their relevance to the caregiving experience. The items on the ‘caregiver status’ section report information on 1) caregiver ability to continue in their role; 2) conflict with family; and 3) feelings of distress, anger, or depression. The information for these items is obtained from multiple sources (e.g., the care recipient, the caregiver, other care providers, the assessor’s observations). The recorded responses are based on the assessor’s clinical judgement that best summarizes all of the information about the presence or absence of caregiver distress. As such, the responses do not necessarily represent only the caregiver’s perception. This is a key difference to the caregiver screener which is a ‘self-reported’ item that specifically reflects the caregiver’s perception of their wellbeing.
Chi-square test, a method used to examine association between variables and test validity , was used to examine the association between CWBI scores and care recipient characteristics that affect the demand of care provided by caregivers often altering their caregiver psychological wellbeing [60,61,62]. For example, items contained in the RAI-HC and interRAI CHA assessments that describe cognitive impairment, bowel and urinary incontinence, and behavioural symptoms.
Logistic regression models were used to examine whether CWBI scores predict LTCH admission of the care recipient, after controlling for covariates such as caregiver and care recipient age and gender, caregiving dyad co-residence, and care recipient health care needs.
The reliability of the screener was also measured in this validation study using standardized Cronbach alpha value.
The following section provides the sample description and results of the analysis performed in the study for the development of the screener.
Caregiver and care recipient samples
Table 1 provides the main characteristics of the caregivers by age. The majority were female, 52% were over 64 years old, and most in this group were spouses of the care recipient. Younger caregivers were more likely to be involved in full or part time work and caring for children in addition to an older adult in the family compared to the older caregivers. A higher proportion of younger caregivers (< 65 years old) experienced financial challenges than older caregivers (65+).
More than half of care recipients were married (55%) and 35% were widowed. The mean age of care recipients in this study was 78 (n = 260, SD 15.1), 51% were female, 58% had a dementia diagnosis, 41% had a MAPLe score of 5, and 55% of the care recipients needed at least extensive assistance for completion of activities of daily living.
Screener development - results
As shown in Table 2, a factor representing psychological wellbeing (factor 3) was identified in addition to two factors representing other aspects of caregiver life: physical health (factor 1) and psychosocial support (factor 2). The correlation between factors ranged between 0.31 and 0.50, with highest values between psychosocial support and psychological wellbeing factors (0.50). The standardized Cronbach values were 0.77, 0.83 and 0.76 for factors 1, 2 and 3 respectively, indicating acceptable or good internal consistencies.
The results of the correlations between each of the items with high loading (bolded items of ‘Psychological wellbeing’ factor) and 1) strain measure (i.e., CSI) and 2) ‘depression item’ showed that only the following items were statistically correlated with both the CSI scores and the ‘depression items’(P < 0.05) that ask: ‘In the last three days, how often have you felt: 1) little or no pleasure in the things you normally enjoy, 2) anxious, restless, or uneasy, 3) sad, depressed or hopeless, and 4) overwhelmed by your relative/friend’s illness?’. All selected questions represent ‘caregiver feelings’ experienced in the 3 days prior the survey. The following five possible responses were: 1) never; 2) not in the last 3 days, but do have the problem; 3) on 1 day; 4) on 2 days; and 5) every day in the past 3 days.
The best distribution of the sum of the four item scores associated with CSI scores (9, 10, 11 vs. 12, 13) and the ‘depression item’ was obtained when assigning a score of zero for the responses 1) never and 2) not in the last 3 days, but do have the problem; a score of ‘1’ for responses 3) 1 day and 4) 2 days; and a score of ‘2’ for the response 5) every day in the past 3 days.
The resulting summary score of the four questions of the CWBI screener ranged from 0 to 8 with cut points zero, 1 to 3, 4 to 6, and 7 to 8 obtained after evaluating the frequency distribution of the CWBI screener scores and CSI.
The association between the CWBI and caregiver related variables was assessed using logistic regression models that included age and gender of the care recipient as covariates in the model.
Table 3 shows that caregivers with higher CWBI scores were more likely to have had doctor/nurse appointments in the last 90 days and poor or fair self-reported health. They were also more likely to report that their lives were never, rarely, or only sometimes good and that they believed the care recipient ‘would be better off elsewhere’. The CWBI predicted all outcomes after controlling for caregiver age and gender.
The standardized Cronbach value calculated for the CWBI was 0.75, indicating that this algorithm has an acceptable internal consistency .
As shown in Fig. 1, other caregiving issues were also related to caregiver poor wellbeing (higher CWBI scores). In contrast, Fig. 2 shows that positive characteristics were related to excellent/good self-reported caregiver wellbeing (all Chi square values had P < 0.0001).
Validation study - results
The age groups ‘younger than 45’, ‘45 to 64’, ‘65 to 74’ and ‘75 or older’ represented 5, 34, 21 and 39% of the informal caregivers in the follow-up study (n = 1020). More than half (54%) were spouses or partners and 36% were children of the care recipient. A total of 85% of all caregivers resided with their respective care recipient. More than half of care recipients were married (62%) and 32% were widowed. The mean age of care recipients in this study was 79 (n = 1020, SD 12.7), 54% were females, 37% had a dementia diagnosis, and 49% of the care recipients needed at least extensive assistance for completion of activities of daily living. Proportions of care recipient with MAPLe 3, 4 and 5 were 36, 38 and 26%, respectively.
There was a strong association between the CWBI scores with the following binary caregiver items in the RAI-HC: 1) caregiver is unable is continue caring activities, 2) primary caregiver is not satisfied with support provided by family and friends and 3) primary caregiver expresses feelings of distress, anger, or depression (Table 4). In addition, higher CWBI scores were associated with caregiver lack of willingness (with ability) to increase emotional support to the care recipient.
Higher CWBI scores were strongly associated with the presence of the following behavioural symptoms of the care recipient: wandering (P < 0.05), verbal abusive (P < 0.0001), and resists care (P < 0.0001). Care recipient diagnosis of dementia, poor ability to understand others, and bowel incontinence were also associated with higher CWBI scores for the respective caregivers (P < 0.0001). There was a higher proportion of caregivers with higher CWBI scores among those caring for older adults presenting recent (last 90 days) changes in cognition, behaviour and mood (Table 4).
Long-term care home admission of care recipients was predicted by caregiver CWBI scores in logistic regression models when only caregiver characteristics were included (model 1) and also after including only care recipient characteristics in the model (model 2) (Table 5). The goodness of fit of the models including caregiver characteristics only (model 1) or care recipient characteristics only (model 2) improved when both models were merged indicating that caregiver and care recipient characteristics are needed to better predict care recipient LTCH admission (model 3) (Table 5).
The standardized Cronbach’s alpha value for the CWBI was 0.89 (n = 1020) in the replication sample indicating that this tool presents good internal consistency.
The development of the CWBI represents an advance in caregiver research. This study showed not only that this screener is a valid and reliable measure of caregiver wellbeing but also that this measure is directly related to positive and negative aspects of a caregiver’s lives. On the one hand, high scores were linked with negative aspects such as financial concerns, inability to continue in their role, physical pain, and loneliness while on the other hand, lower scores were associated with positive aspects of life such as caregivers feeling valued and respected by others, capable of managing stresses, satisfied with life, and hopeful about their future. A highlight of the validation study was the association between poor caregiver wellbeing and care recipient long term care home admission, changes in their behaviour, and worsening of their mood, decision making, and communication.
The screener differentiates levels of wellbeing linked to positive and negative aspects of caregiver lives. This is in agreement with other studies  but it also has other important implications. Since wellbeing has a protective role in health maintenance and poor wellbeing is associated with impaired health , the screener may be used to identify caregivers that would benefit from target interventions or preventive measures, using only four items. According to Dodge et al. , wellbeing is related to the psychological social, and physical resources that someone has to address as a result of challenges on these aspects of their lives . Thus, caregivers can achieve wellbeing by receiving necessary resources (e.g., information, social support, respite) to maintain a sense of equilibrium.
The association between CBWI scores and caregiver related outcomes shows how psychological wellbeing is related to a caregiver’s perceived health and multiple doctor or nurse appointments. These findings are also consistent with other studies that report high prevalence of poor health among family caregivers  and higher health care utilization .
It is possible that a substantial number of caregivers, particularly with the higher scores on the CWBI, were experiencing or could be at risk of depression as they were more likely to report loneliness, hopelessness, and daily physical pain. This is concerning considering that depression affects quality of life and may lead to a caregiver contemplating suicide  and tragic outcomes, highlighting the need to monitor the mental health of caregivers.
Interestingly, caregivers with excellent/good subjective wellbeing were more likely to present a positive outlook (i.e., ‘feel good about myself’). It is important to note that caregiver life satisfaction has been linked with positive caregiving experiences even when they report high burden . In addition, the presence of a large number of friends and close relationships has been associated with overall life satisfaction of caregivers .
The results of the present research clearly demonstrate multidimensionality of the factors related with caregiver wellbeing (e.g., perceived physical health, loneliness, financial concerns, personal outlook). These findings are consistent with interactions between caregiver wellbeing and other aspects of their lives providing evidence on the validity of the screener [58, 64, 68,69,70].
The validation of the screener using information collected during a follow-up pilot study at a large home care organization emphasized the relationship between different aspects of a caregiver’s life such as their relationship with family (social resource), presence of distress, anger, and depression (psychological resources) and their wellbeing. Caregiver perceived burden or distress has been associated with wellbeing  and may affect the ability to continue in the caregiving role. In this study, caregivers with higher CBWI scores (poor wellbeing) were more likely to feel unable to continue. The inability to continue among caregivers with high CBWI scores was emphasized by the association found between high caregiver CWBI scores and the respective care recipient LTCH admission even after controlling for care recipient age and health care needs (i.e., MAPLe). This finding is consistent with studies showing the linkage between caregiver wellbeing and care recipient LTCH admission .
The CWBI has the advantage of being fully compatible with interRAI standards for assessments that are widely used in Canada and elsewhere . Linking caregiver information from the pilot survey with the most recent care recipient interRAI CHA or RAI-HC allows for the assessment of caregiver needs at the same time that care recipient health is examined. In this study, the linkage between the RAI-HC assessment of the care recipient and the CWBI indicated some patient characteristics that could be affecting caregiver wellbeing. For example, CWBI scores were not only associated with the health care needs of the care recipient (i.e., MAPLe) but also with care recipient worsening in decision making, communication, changes in behavioural symptoms, and mood decline at the time of the assessment compared to the status of 90 days ago. These associations may indicate the impact of recent changes in symptoms on caregiver psychological wellbeing [6, 73].
The result of standardized Cronbach alpha indicates that the screener is a reliable tool with items measuring a similar construct. The screener scores provide a measure of caregiver wellbeing that is not only driven by care recipient needs but also by caregiving experiences, available resources, and caregiver mental and physical health.
As an observational study, the present research does not allow for making causal inferences. Another limitation is that the screener was developed using information from caregivers that were already experiencing stress and therefore many were likely experiencing poor wellbeing, although the validations sample included caregivers experiencing different levels of wellbeing. The generalizability of the outcomes of this study is limited to caregivers of older adults receiving home care.
There are a number of advantages to using the CWBI. The fact that the CWBI was developed using items common to other interRAI assessments is an important advantage of this tool. The screener can be applied on its own or used in conjunction with an interRAI assessment of the care recipient to facilitate the design of an integrated care plan to address the needs of both persons in the dyad.
The CWBI identifies caregivers at risk of adverse outcomes with a small number of items that could be implemented in a relatively straightforward manner, especially by organizations that assess a high volume of care recipients and caregivers. The CWBI consists of only four questions, focusing in one concept (psychological wellbeing). The CWBI is different than other screeners used in caregiver research such as the risk appraisal measure (RAM), a screener with 16 questions  or the Tailored Caregiver Assessment and Referral (TCARE), a 32 question screen used to develop care plans for informal caregivers [75, 76].
The CWBI has been validated using various data sources. Scores of this short screener were associated with caregiver and care recipient characteristics and outcomes (long term care admission) and therefore could be used to flag the need for further assessment of the caregiver to target interventions that match their unique needs.
The CWBI scores were associated with negative and positive aspects of caregiving. Results from this study demonstrate the complexity involved in caregiving by reporting negative and positive aspects of their experiences, the latter often omitted in caregiving studies. This is a novelty compared with most instruments that focus on burden. The choice of using a psychological wellbeing measure acknowledges that caregiving may result in both positive and negative feelings, where unstable feelings affect the screener scores. Moreover, this measure seems a better predictor of self-reported health than other caregiver measures .
This screener has the potential to identify caregivers that would likely benefit from further assessment and intervention (such as respite care) with the goal of improving their quality of life. For example, clinical treatment may be appropriate for caregivers affected by depression or pain, whereas caregivers experiencing loneliness might benefit from participation in a support group. Further research to explore the outcomes of using the CWBI in other jurisdictions, coupled with information from the interRAI CHA or RAI-HC, has the potential to provide key information to be used by policy and decision makers when developing strategies and funding initiatives to better support caregivers in their role.
Canadian Institute for Health Information
Caregiver Recharge Services
Caregiver Strain Index
Caregiver Wellbeing Index
Hamilton Niagara Haldimand Brant
Local Health Integration Network
long term care home
Method of Assigning Priority Levels
Statistical Analysis System
Gaugler JE, Kane RL, Kane RA, Clay T, Newcomer R. Caregiving and institutionalization of cognitively impaired older people: utilizing dynamic predictors of change. Gerontologist. 2003;43:219–29.
Chari AV, Engberg J, Ray KN, Mehrotra A. The opportunity costs of informal elder-Care in the United States: new estimates from the American time use survey. Health Serv Res. 2015;50:871–82.
Vitaliano P. Spouses of people with incident dementia are at sixfold increased risk of developing dementia themselves. Evid Based Ment Health. 2010;13:101.
Vitaliano P, Zhang J, Scanlan JM. Is caregiving hazardous to One’s physical health? A meta-analysis. Psychol Bull. 2003;129:946.
Dassel KB, Carr DC, Vitaliano P. Does caring for a spouse with dementia accelerate cognitive decline? Findings from the health and retirement study. The Gerontologist. 2015;57(2):319–28.
Dodge R, Daly AP, Huyton J, Sanders LD. The challenge of defining wellbeing. Int Jof wellbeing. 2012;2(3):222–35.
Gaugler JE, Potter T, Pruinelli L. Partnering with caregivers. Clin Geriatr Med. 2014;30:493–515.
Steptoe A, Deaton A, Stone AA. Psychological wellbeing. Health Ageing Lancet. 2015;385:640.
Kruithof WJ, Visser-Meily JMA, Post MWM. Positive caregiving experiences are associated with life satisfaction in spouses of stroke survivors. J Stroke Cerebrovasc Dis. 2012;21:801–7.
Kruithof WJ, Post MW, Visser-Meily JM. Measuring negative and positive caregiving experiences: a psychometric analysis of the caregiver strain index expanded. Clin Rehab. 2015;29(12):1224–33.
Kramer BJ. Gain in the caregiving experience: where are we? What next? The Gerontologist. 1997;37:218–32.
Felce D, Perry J. Quality of life: its definition and measurement. Res Dev Disabil. 1995;16:51–74.
Schene AH, Tessler RC, Gamache GM. Instruments measuring family or caregiver burden in severe mental illness. Soc Psychiatry Psychiatr Epidemiol. 1994;29:228–40.
Vitaliano PP, Russo J, Young HM, Teri L, Maiuro RD. Predictors of burden in spouse caregivers of individuals with Alzheimer’s disease. Psychol Aging. 1991;6:392.
Schene AH, Tessler RC, Gamache GM. Instrument measuring family or caregiver burden in severe mental illness. Soc Psychiatry Psychiatr Epidemiol. 1994;29:228–40.
Mosquera I, Vergara I, Larrañaga I, Machón M, del Río M, Calderón C. Measuring the impact of informal elderly caregiving: a systematic review of tools. Qual Life Res. 2016;25:1059–92.
Hirdes JP, Ljunggren G, Morris JN, Frijters DH, Finne Soveri H, Gray L, et al. Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system. BMC Health Serv Res. 2008;8:277.
interRAI. http://www.interrai.org. Accessed 5 Apr 2017.
Morris JN, Fries BE, Steel K, Ikegami N, Bernabei R, Carpenter GI, et al. Comprehensive clinical assessment in community setting: applicability of the MDS-HC. J Am Geriatr Soc. 1997;45:1017–24.
Poss JW, Jutan NM, Hirdes JP, Fries BE, Morris JN, Teare GF, et al. A review of evidence on the reliability and validity of minimum data set data. In: Healthcare management forum: Elsevier; 2008. p. 33–9. http://www.sciencedirect.com/science/article/pii/S0840470410601275. Accessed 11 Oct 2016.
Morris JN, Fries BE, Mehr DR, Hawes C, Phillips C, Mor V, et al. MDS cognitive performance scale. J. Gerontol. 1994;49:M174–82.
Travers C, Byrne GJ, Pachana NA, Klein K, Gray L. Validation of the interRAI cognitive performance scale against independent clinical diagnosis and the mini-mental state examination in older hospitalized patients. J Nutr Health Aging. 2013;17:435–9.
Burrows AB, Morris JN, Simon SE, Hirdes JP, Phillips C. Development of a minimum data set-based depression rating scale for use in nursing homes. Age Ageing. 2000;29:165–72.
Hirdes JP, Poss JW, Curtin-Telegdi N. The method for assigning priority levels (MAPLe): a new decision-support system for allocating home care resources. BMC Med. 2008;6:9.
Hogeveen SE, Chen J, Hirdes JP. Evaluation of data quality of interRAI assessments in home and community care. BMC Med Inform Decis Mak. 2017;17:150.
Chou K-R, Chu H, Tseng C-L, Lu R-B. The measurement of caregiver burden. J Med Sci-TAIPEI. 2003;23:73–82.
Michels CT, Boulton M, Adams A, Wee B, Peters M. Psychometric properties of carer-reported outcome measures in palliative care: a systematic review. Palliat Med. 2016;30:23–44.
Vitaliano PP, Young HM, Russo J. Burden: a review of measures used among caregivers of individuals with dementia. The Gerontologist. 1991;31:67–75.
Schulz R, O’Brien A, Czaja S, Ory M, Norris R, Martire LM, et al. Dementia caregiver intervention research in search of clinical significance. The Gerontologist. 2002;42:589–602.
Bédard M, Molloy DW, Squire L, Dubois S, Lever JA, O’Donnell M. The Zarit burden interview a new short version and screening version. The Gerontologist. 2001;41:652–7.
Vitaliano PP, Russo J, Young HM, Becker J, Maiuro RD. The screen for caregiver burden. The Gerontologist. 1991;31:76–83.
Whalen KJ, Buchholz SW. The reliability, validity and feasibility of tools used to screen for caregiver burden: a systematic review. 2009. https://works.bepress.com/kimberly_whalen/7/. Accessed 10 Jun 2016.
Stephens MAP, Norris VK, Kinney JM, Ritchie SW, Grotz RC. Stressful situations in caregiving: relations between caregiver coping and well-being. Psychol Aging. 1988;3:208–9.
Van Durme T, Macq J, Jeanmart C, Gobert M. Tools for measuring the impact of informal caregiving of the elderly: a literature review. Int J Nurs Stud. 2012;49:490–504.
Fries BE, Hawes C, Morris JN, Phillips CD, Mor V, Park PS. Effect of the National Resident Assessment Instrument on selected health conditions and problems. J Am Geriatr Soc. 1997;45:994–1001.
Robinson BC. Validation of a caregiver strain index. J Gerontol. 1983;38:344–8.
Thornton M, Travis SS. Analysis of the reliability of the modified caregiver strain index. J Gerontol B Psychol Sci Soc Sci. 2003;58:S127–32.
Costello AB, Osborne JW. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10:1–9.
Beavers AS, Lounsbury JW, Richards JK, Huck SW, Skolits GJ, Esquivel SL. Practical considerations for using exploratory factor analysis in educational research. Pract Assess Res Eval. 2013;18:1–13.
Hayton JC, Allen DG, Scarpello V. Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organ Res Methods. 2004;7:191–205.
Kabacoff RI. Determining the dimensionality of data: A SAS® macro for parallel analysis. 2003. Seattle, WA: Proceedings of the 28th Annual Meeting of SAS Users Group International. Retrieved 5 March 2015, from http://www2.sas.com/proceedings/sugi28/090-28.pdf.
Hair J Jr, Anderson R, Tatham R, William C. Multivariate data analysis. 4th ed. New Jersey: Prentice-Hall; 1995.
Beach SR, Schulz R, Yee JL, Jackson S. Negative and positive health effects of caring for a disabled spouse: longitudinal findings from the caregiver health effects study. Psychol Aging. 2000;15:259.
Goode KT, Haley WE, Roth DL, Ford GR. Predicting longitudinal changes in caregiver physical and mental health: a stress process model. Health Psychol. 1998;17:190–8.
Schulz R, Martire LM. Family caregiving of persons with dementia: prevalence, health effects, and support strategies. Am J Geriatr Psychiatry. 2004;12:240–9.
Tindle H. Up: how positive outlook can transform our health and aging. Penguin; 2013.
Mittelman MS, Haley WE, Clay OJ, Roth DL. Improving caregiver well-being delays nursing home placement of patients with Alzheimer disease. Neurology. 2006;67:1592–9.
Andrén S, Elmståhl S. Relationships between income, subjective health and caregiver burden in caregivers of people with dementia in group living care: a cross-sectional community-based study. Int J Nurs Stud. 2007;44:435–46.
Horwitz AV, Reinhard SC. Ethnic differences in caregiving duties and burdens among parents and siblings of persons with severe mental illnesses. J Health Soc Behav. 1995;36:138.
Skarupski KA, McCann JJ, Bienias JL, Evans DA. Race differences in emotional adaptation of family caregivers. Aging Ment Health. 2009;13:715–24.
Gallicchio L, Siddiqi N, Langenberg P, Baumgarten M. Gender differences in burden and depression among informal caregivers of demented elders in the community. Int J Geriatr Psychiatry. 2002;17:154–63.
Kim H, Chang M, Rose K, Kim S. Predictors of caregiver burden in caregivers of individuals with dementia: predictors of caregiver burden. J Adv Nurs. 2012;68:846–55.
Mello J de A, Macq J, Durme TV, Cès S, Spruytte N, Audenhove CV, et al. The determinants of informal caregivers’ burden in the care of frail older persons: a dynamic and role-related perspective. Aging Ment Health. 2016;0:1–6.
Pinquart M, Sörensen S. Gender differences in caregiver stressors, social resources, and health: an updated meta-analysis. J Gerontol B Psychol Sci Soc Sci. 2006;61:P33–45.
Rose-Rego SK, Strauss ME, Smyth KA. Differences in the perceived well-being of wives and husbands caring for persons with Alzheimer’s disease. The Gerontologist. 1998;38:224–30.
Thompson RL, Lewis SL, Murphy MR, Hale JM, Blackwell PH, Acton GJ, et al. Are there sex differences in emotional and biological responses in spousal caregivers of patients with Alzheimer’s disease? Biol Res Nurs. 2004;5:319–30.
Lukaschek K, Vanajan A, Johar H, Weiland N, Ladwig K-H. “In the mood for ageing”: determinants of subjective well-being in older men and women of the population-based KORA-age study. BMC Geriatr. 2017;17:126.
Bertea P, Zait A. Methods for Testing Discriminant Validity. Craiova: Manag Mark; 2011. p. 217–24.
van der Lee J, Bakker TJEM, Duivenvoorden HJ, Dröes R-M. Multivariate models of subjective caregiver burden in dementia: a systematic review. Ageing Res Rev. 2014;15:76–93.
Bergvall N, Brinck P, Eek D, Gustavsson A, Wimo A, Winblad B, et al. Relative importance of patient disease indicators on informal care and caregiver burden in Alzheimer’s disease. Int Psychogeriatr. 2011;23:73–85.
Berg-Weger M, Rubio DM, Tebb SS. Living with and caring for older family members: issues related to caregiver well-being. J Gerontol Soc Work. 2000;33:47–62.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 16:297–334.
Yates ME, Tennstedt S, Chang B-H. Contributors to and mediators of psychological well-being for informal caregivers. J Gerontol B Psychol Sci Soc Sci. 1999;54:P12–22.
Schubert CC, Boustani M, Callahan CM, Perkins AJ, Hui S, Hendrie HC. Acute care utilization by dementia caregivers within urban primary care practices. J Gen Intern Med. 2008;23:1736–40.
O’Dwyer ST, Moyle W, Zimmer-Gembeck M, De Leo D. Suicidal ideation in family carers of people with dementia: a pilot study: suicidal ideation in dementia carers. Int J Geriatr Psychiatry. 2013;28:1182–8.
Haley WE, Levine EG, Brown SL, Bartolucci AA. Stress, appraisal, coping, and social support as predictors of adaptational outcome among dementia caregivers. Psychol Aging. 1987;2:323–30.
George LK, Gwyther LP. Caregiver weil-being: a multidimensional examination of family caregivers of demented adults. The Gerontologist. 1986;26:253–9.
Chen M-L. The growing costs and burden of family caregiving of older adults: a review of paid sick leave and family leave policies. Gerontologist. 2014. https://doi.org/10.1093/geront/gnu093.
Siegel K, Raveis VH, Houts P, Mor V. Caregiver burden and unmet patient needs. Cancer. 1991;68:1131–40.
McConaghy R, Caltabiano ML. Caring for a person with dementia: exploring relationships between perceived burden, depression, coping and well-being. Nurs Health Sci. 2005;7:81–91.
Carpenter I, Hirdes JP. Using interRAI assessment systems to measure and maintain quality of long-term care. OECD Health Policy Stud. 2013:93–139.
Hendry LB, Kloep M. Lifespan development: resources, challenges and risks. London, UK: Cengage Learning EMEA; 2002.
Czaja SJ, Gitlin LN, Schulz R, Zhang S, Burgio LD, Stevens AB, et al. Development of the risk appraisal measure: a brief screen to identify risk areas and guide interventions for dementia caregivers: risk appraisal measure for dementia caregivers. J Am Geriatr Soc. 2009;57:1064–72.
Montgomery RJV, Kwak J, Kosloski K, O’Connell Valuch K. Effects of the TCARE(R) intervention on caregiver burden and depressive symptoms: preliminary findings from a randomized controlled study. J Gerontol B Psychol Sci Soc Sci. 2011;66B:640–7.
Montgomery R, Kwak J. TCARE: tailored caregiver assessment and referral. AJN Am J Nurs. 2008;108(Supplement):54–7.
Savundranayagam MY, Montgomery RJV, Kosloski K. A dimensional analysis of caregiver burden among spouses and adult children. The Gerontologist. 2011;51:321–31.
The data used for the development of the CWBI were collected during the Caregiver Recharge Services evaluation financially supported by MH LHIN. We thank HNHB LHIN for the collaborative partnership with the University of Waterloo for the development and implementation of CWBI. We gratefully acknowledge the caregiver participants for their time to complete the survey.
This study was supported by MH LHIN. The funding organization did not participate in the design of the study and collection, analysis, and interpretation of data, and in the writing of the manuscript.
Availability of data and materials
The data that support the findings of this study are retained by Health Shared Services Ontario. These data were made available to the authors under the terms of an agreement with interRAI Canada. The data may be accessed by non-interRAI researchers through separate request to CIHI. The screener (formerly known as Caregiver Distress Index) is copyrighted by interRAI and a copy can be requested through firstname.lastname@example.org.
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Ethics clearance for the use of secondary data has been provided by the University of Waterloo Research Ethics Committee (ORE #1822). Ethics clearance for the use of primary data has been provided by the University of Waterloo Research Ethics Committee and written consents were obtained from all participants (ORE #18982).
Consent for publication
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Betini, R.S.D., Hirdes, J.P., Curtin-Telegdi, N. et al. Development and validation of a screener based on interRAI assessments to measure informal caregiver wellbeing in the community. BMC Geriatr 18, 310 (2018). https://doi.org/10.1186/s12877-018-0986-x
- Long-term care home
- Home care