Quality of care in European home care programs using the second generation interRAI Home Care Quality Indicators (HCQIs)

Abstract

Background

Evaluating the quality of care provided to older individuals is a key step to ensure that needs are being met and to target interventions to improve care. To this aim, interRAI’s second-generation home care quality indicators (HCQIs) were developed in 2013. This study assesses the quality of home care services in six European countries using these HCQIs as well as the two derived summary scales.

Methods

Data for this study were derived from the Aged in Home Care (AdHOC) study - a cohort study that examined different models of community care in European countries. The current study selected a sub-sample of the AdHOC cohort from six countries whose follow-up data were complete (Czech Republic, Denmark, Finland, Germany, Italy and the Netherlands). Data were collected from the interRAI Home Care instrument (RAI-HC) between 2000 and 2002. The 23 HCQIs of interest were determined according to previously established methodology, including risk adjustment. Two summary measures, the Clinical Balance Scale and Independence Quality Scale were also determined using established methodology.

Results

A total of 1,354 individuals from the AdHOC study were included in these analyses. Of the 23 HCQIs that were measured, the highest proportion of individuals experienced declines in Instrumental Activities of Daily Living (IADLs) (48.4 %). Of the clinical quality indicators, mood decline was the most prevalent (30.0 %), while no flu vaccination and being alone and distressed were the most prevalent procedural and social quality indicators, respectively (33.4 and 12.8 %). Scores on the two summary scales varied by country, but were concentrated around the median mark.

Conclusions

The interRAI HCQIs can be used to determine the quality of home care services in Europe and identify areas for improvement. Our results suggest functional declines may prove the most beneficial targets for interventions.

Background

As population aging continues around the world, many older individuals express a desire to maintain independence and remain at home as long as possible. However, a substantial portion of this population will require extensive health care services in later life [1]. Adequate service provision in the home setting can have a significant impact on the quality of life of older individuals. Further, optimal care can stave off undesirable outcomes including transitions to more intensive care settings such as long-term care facilities. Understanding how well home care services meet the needs of older individuals can help evaluate the quality of care and compare service provision across jurisdictions, either within or between countries.

There are different ways of assessing the quality of care. One measure is the Outcome Assessment and Information Set (OASIS), which is used in quality measurement and care planning for home health care in the United States [2]. However, concerns about the low to moderate validity and reliability for some OASIS items, as well as concerns over its applicability in outcome measure or outcome-based quality improvement have been raised [2]. interRAI, an international research consortium specializing in the development and application of standardized assessment instruments, released its first set of home care quality indicators (HCQIs) about a decade ago [3, 4]. Second-generation HCQIs, developed in 2013, introduced several refinements to the indicators, including more sophisticated risk adjustment strategies and additional indicator domains [5]. Widespread adoption of the interRAI Home Care (RAI-HC) instrument in several North American and European jurisdictions provided very large sample sizes, which could be used to develop a more comprehensive set of risk adjusters and to introduce a two-step adjustment model involving both individual level covariates, population level stratification, and temporal adjustments [5, 6]. With a more advanced risk adjustment approach, variations in the newer HCQI scores are likely a more accurate reflection of the impact of services provided and the overall effectiveness of home care services. The ability to include data from many nations also provided better evidence of cross-national applicability than was possible with the first generation HCQIs. The advantages of interRAI’s HCQIs include more standardized items included in the assessment, a more comprehensive set of indicators and the ability to provide an aggregated measure of different HCQIs rather than a simple measure of individual HCQIs. This could be helpful in providing a more complete evaluation of the quality of care. Also, interRAI assessments are used in more than 30 countries worldwide, allowing HCQIs to be obtained from a wider geographic area.

An important contribution to interRAI’s efforts to refine the HCQIs has been the use of European data from the Aged in Home Care (ADHoC) project for both generations of indicators. The background and key findings from this work have been well described [7, 8]. The ADHoC home care data have been analyzed by Bos and colleagues [9] using the original HCQIs, confirming significant cross-country differences. However, this earlier work only included the subset of 16 prevalence HCQIs (outcome based HCQIs could not be derived based on the data available at that time) and did not have well-developed summary scales that can be generated from the second-generation HCQIs [9]. Thus, this work did not provide the comprehensive picture of quality of home care services provided in the respective countries. The purpose of this study was to apply and refine the second-generation HCQI methodology to a large European sample of individuals receiving home care and examine the quality of services. A secondary aim was to determine summary measures of services across these countries as a more comprehensive measure of home care quality.

Methods

Data source

Data for this study were a subset of those collected from the AdHOC Study. Methods and sample description from AdHOC have been previously published. The sample included 3785 individuals receiving home care services in 11 European countries [7]. Data were collected using the RAI-HC, a comprehensive assessment tool with more than 300 items that has been well-established as a standardized and reliable assessment instrument [10, 11]. Data were collected in each country by specially trained assessors, usually nurses, who verify information collected with sources including direct interviews of home care clients and family members, as well as review of physician reports medical records. Data were collected at baseline and at 6 and 12 month follow-ups between 2000 and 2002. Additional data about service structures and delivery were collected using a separate form and published previously [7]. The AdHOC study was funded by the Fifth Framework Programme of the European Union and ethical approval was obtained in accordance with protocols in place in all participating countries. The ethical approval from the Comitato Etico Universita Cattolica del Sacro Cuore - Policlinico A. Gemelli, Rome, Italy covered the use of anonymized data in the current study.

Measures

Study sample

This study involved secondary analysis of data collected from the AdHOC study, utilizing multi-national, standardized RAI-HC data. The current study included a subset from the AdHOC sample (n = 3785) since necessary data to calculate HCQIs were not collected in all countries and not all individuals were followed up at 6 months. The final sample included clients from the Czech Republic, Denmark, Finland, Germany, Italy, and the Netherlands who were still living at home (n = 1354). In all countries except Finland, participants were invited to take part in the study and were free to decline participation. Written consent was obtained from clients of home services, their relatives, or legal guardians, with the assurance of data confidentiality. In Finland, The National Institute for Health and Welfare (formerly Stakes), holds a permission to collect data using the RAI-HC and maintain a national register based on this information. According to this permission (which is valid until 2025) no informed consent wass needed.

Descriptive characteristics

The full AdHOC study sample has been described previously [7], but the sub-sample included here was described using socio-demographic characteristics, geriatric conditions, disease diagnoses, and medication use using items from the RAI-HC instrument. Functional impairment was determined using interRAI’s activities of daily living (ADL) hierarchy scale, with scores derived from items on the RAI-HC [12]. Mild impairment was considered to be present in individuals with scores between 0 and 1, limited to extensive impairment was considered present in individuals with scores between 2 and 4, and those with scores of 5 or more were considered to be dependent in ADLs. Cognitive impairment was determined using the Cognitive Performance Scale (CPS), with scores of 0 to 1 representing intact cognition to borderline impairment, scores of 2 to 4 representing mild to moderate impairment and scores above 5 representing severe impairment [13]. Urinary incontinence was considered to be present if individuals were not always continent using the relevant item on the RAI-HC. Depression was determined using the Depression Rating Scale (DRS), with scores of 3 or higher being considered to represent the presence of probable depressive symptoms [14]. A binary measure for any behaviour was created using presence of any of the following: wandering, verbally or physically abusive behaviours, socially inappropriate behaviours or resisting care. The disease diagnosis section of the RAI-HC, which has been shown to collect accurate information about conditions compared to administrative databases [15], was used to determine the presence of hypertension, arthritis, dementia (including Alzheimer’s and non-Alzheimer’s types), coronary artery disease (CAD), diabetes and heart failure. The number of falls and number of medications used in the previous seven days were measured with stand-alone items on the RAI-HC.

Calculation of HCQIs

All 23 of the second-generation HCQIs were calculated in the current study [5]. These included eight functional indicators (decline in instrumental activities of daily living [IADLs], IADL improvement, ADL decline, ADL improvement, communication decline, communication improvement, cognitive decline, and cognitive improvement), ten clinical indicators (weight loss, injuries, falls, daily severe pain, pain not controlled, pain improvement, mood improvement, mood decline, bladder function decline, and bladder function improvement) and five social and treatment indicators (cessation of leaving home; alone and distressed; continued caregiver distress; use of hospital, emergency department [ED] or emergent care; and no influenza vaccination).

The development of these HCQIs has been described in detail elsewhere and the same methodology was used to calculate the HCQIs in this study [5]. Adjustment was an important component of the refinement of previous HCQIs and adjustment of means for each of the 23 HCQIs was carried out to account for differences in client profiles between the six countries. In this study, all covariates used in previous risk adjustment were explored and only those found to be significantly associated with the HCQIs were retained. Refer to Appendix for descriptions of each HCQI, including stratification and adjustment variables. Two summary scales were developed with the new generation HCQIs [5]. These are the interRAI Independence Quality Scale (assessing functional independence and engagement), and the interRAI Home Care Clinical Balance Quality Scale (assessing improvements in function, cognition, and psychosocial indicators). The interRAI Independence Quality Scale incorporated the following HCQIs: ADL decline, IADL decline, cognitive decline, communication decline, not going out, falls, injuries, hospitalizations/ED visits, mood decline, bladder decline, and pain not controlled. For the interRAI Home Care Clinical Balance Quality Scale, the following HCQIs were included: ADL improvement, IADL improvement, cognitive improvement, communication improvement, bladder improvement, mood improvement, pain improvement, caregiver not distressed, and not alone and distressed. Both scales range from 0 to 10 with 0 representing the worst score and 10 representing the best.

Statistical analyses

Descriptive statistics were calculated and HCQI development was done using SAS (version 8.2, SAS Inc., Cary NC). Risk adjustment for each HCQI was done initially using bivariate regression models and then multivariate logistic regression modelling, with the significance level set to p < 0.05. Summary scale development was completed using SPSS (version 18).

Results

The general characteristics of the sample are described in Table 1. The majority of individuals receiving home care were female and between 75 and 84 years of age, with the exception of Danish participants, who were older (46.3 % over age 85). Both functional and cognitive impairment were common, with particularly high rates of both types of impairment in the German and Italian samples (16.0 and 34.2 % for ADL and 14.8 and 20.6 % for CPS, respectively). Incontinence was present in more than 40 % of individuals from each country. The prevalence of behavioural symptoms was low, while rates of depression ranged from 7.0 % in Finland to 31.9 % in the Netherlands. Both pain and falls were common among participants. Comorbidities were common with the exception of dementia, which was present in up to 20 % of the sample. Polypharmacy was also common, with 54.1 % of the total sample using more than six medications.

Table 1 General characteristics of European home care clients at Baseline (N = 1354)

Figure 1 presents the raw and adjusted functional HCQIs for the sample. The highest proportion of individuals experienced decline in ADLs (48.3 %), while both improvements and declines in communication skills were least common (15.8 and 14.6 %, respectively). Among the clinical HCQIs (presented in Fig. 2), it can be seen that weight loss and injuries were less common (9.9 and 10.7 %, respectively), whereas pain improvement and mood decline were more common (29.0 and 30.0 %, respectively). From Fig. 3, no influenza vaccination was the most prevalent service QI (33.4 %), and alone and distressed was the most commonly observed social QI (12.8 %).

Fig. 1

Functional Quality Indicators - average proportions of individuals declining or improving, European home care clients, N = 1354. Abbreviations: ADL = activities of daily living; IADL = instrumental activities of daily living; QI = quality indicator

Fig. 2

Clinical Quality Indicators – average proportions of individuals declining or improving, European home care clients, N = 1354. Abbreviations: QI = quality indicator

Fig. 3

Social and Service Clinical Quality Indicators – average proportions of individuals declining or improving at 6-month follow-up, European home care clients, N = 1354. Abbreviations: ER = emergency room; QI = quality indicator

Figure 4 presents the interRAI summary scale scores by country and in the overall sample. For both the home care Clinical Balance Scale and the home care Independence Quality Scale, possible scores range from 0 (representing the worst performance) to 10 (representing the best performance). The best overall scores for the Independence Quality Scale were achieved by Finland, followed by Denmark and Germany; Italy achieved the lowest score. For the Clinical Balance Quality Scale, services in the Czech Republic performed the best, followed by those in Denmark and the Netherlands, while Finnish and German services performed the worst.

Fig. 4

Scores by Country, interRAI Home Care Clinical Balance Scale and interRAI Home Care Independence Quality Scale, N = 1354

Discussion

This study used the second-generation interRAI HCQIs to examine the quality of home care services in samples from 6 European countries. This work has shown that some indicators describe better performance of the home care service, whereas others, in particular, functional decline measures, signify areas of potentially lower quality service provision indicating areas for improvement. interRAI’s standardized data collection approach allows such comparisons of quality between countries, representing a major added value for such instruments. These instruments can be used to support care planning, intervention development and outcome measurement at the individual level and to allow performance measurement of health systems using population level data [16–18].

There was notable variance by country in the HCQIs presented in this study. The sophisticated adjustments involving many covariates to control for potentially differing client profiles in the different home care programs allowed for more accurate description of true differences in quality of home care. Noteworthy are indicators of good outcomes including IADL improvement (though there is still room for much improvement) and improvement in pain. Previous work done by the AdHOC study group found pain to be a highly prevalent problem in the overall cohort, with nearly 60 % of clients reporting it [19]. Thus, although the current results are encouraging, pain likely remains an area for continued improvement in community based individuals. More problematic outcomes were also observed, such as high rates of IADL decline. The overall rate of IADL decline observed in this study was higher than that observed in earlier work [5], though more similar rates in other functional indicators were observed. It should be noted that direct comparisons of HCQIs between countries was beyond the scope of this paper, as detailed information about policy differences between countries was not available.

The current study also builds upon earlier HCQI research that utilized ADHoC data [9]. By using the second-generation HCQIs, this study captures both prevalence and incidence based quality indicators, providing a more comprehensive understanding of the quality of home care services. Perhaps due to a more rigorous adjustment process, the estimates reported here are similar, but generally lower than those reported earlier [9]. Finally, the inclusion of two summary scales allows a high-level method of comparing between countries. The rankings generated by Bos and colleagues are similar to those of the Independence Quality Scale [9]. However, countries that performed better by this measure, such as Finland and Germany fared worse on the Clinical Balance Scale. Compared to the developmental work done previously by Morris and colleagues, this study observed similar rates of service and treatment indicators, higher rates of falls and injuries and lower rates of mood improvement [5]. The summary scale scores indicate that none of the programs in participating countries performed at the lowest level, but all have room for improvement. Such summary scales are useful in providing a high-level overview of the performance of home care programs. They can be used to provide relatively straightforward representations of complex sets of inter-related indicators in a manner that is relatively accessible to non-researchers. Policy makers, advocacy groups, managers, and the general population often wish to have one or two indicators providing a global rating of the quality of services. These two scales represent a composite of 20 separate measures of quality, providing a comprehensive but simplified representation of home care performance.

Overall this work shows that the HCQIs make it feasible to assess performance using the same items across many countries. As such, in Europe, countries can look to other examples to identify best practices and improve care. At a patient or agency level, such indicators are helpful in improving care planning, while at regional or national levels, these measures can help with benchmarking initiatives. Questions arise, however, such as how responsible a particular home care agency is for the quality of care. Such HCQIs allow the conversation about accountability in both performance and outcomes to begin. Although achieving perfect HCQI scores is unrealistic, motivating countries to improve by learning from care practices abroad is valuable.

Briefly, some limitations of the current work should be noted. First, the sample was not randomly selected, but rather was a convenience sample from AdHOC data. As the purpose of this study was to explore whether the second-generation HCQIs could be used in a European context, these data were adequate although somewhat dated. Nonetheless, it is important to note that these samples are considered to be representative of urban areas in the respective countries, but are not necessarily nationally representative. Thus, the results found here may not reflect overall quality of home care in each of the countries examined. It is also possible that some of the HCQIs are under-reported due to the study setting. For example, severe injuries may trigger transitions to long-term care and would not be captured here. Next, the HCQIs have been adjusted for a number of clinical covariates, but program characteristics and service frequency were not available. Further, it was not possible to explore potential policy differences between the countries on home care service provision, which also may have influenced the HCQIs reported here. This makes direct comparisons between countries somewhat difficult. However, all facilities were public facilities, with the exception of those in the Netherlands which were non-profit, but non-public. All facilities had public payment or compulsory insurance [7]. These similarities may mean that facility characteristics and not financial structures may explain more of the observed differences. These HCQIs have been developed in Western, developed countries and may not necessary be applicable in other countries with different policy and cultural contexts. Nonetheless, this work has demonstrated the utility of standardized assessment information from RAI-HC derived quality measures in a six country sample, demonstrating practical applications of such data to help improve quality of care. Since the AdHOC study, several new European regions (including Finland, Belgium, Italy and Ireland) have adopted the RAI-HC and these HCQIs can be useful in creating cross-national and cross-continental benchmarking for the quality of services. Finally, since the HCQIs are not routinely used and are obtained from routinely collected items within the RAI-HC itself, it is not likely that the implementation of the HCQIs would have impacted actual performance in any of the study sites. Further, it was not the primary aim of the AdHOC study to determine HCQIs, further reducing the likelihood that participation in the study altered performance.

Conclusions

The second-generation HCQIs assess different domains of quality of home care services and are applicable in different countries. Ultimately, this work could provide a model on which to base quality measures in other care settings using standardized interRAI assessment information. These would be powerful tools in improving care delivery to vulnerable older populations in the community who are at risk of transfer to more intensive care settings. Future work could also use our results as a first standard measure of performance to which more recent performance could be compared. Matching these second-generation HCQIs to cost-effectiveness of services would also be powerful extensions of the current research and would make arguments to policy-makers more salient.

Appendix

Table 2 Description of Home Care Quality Indicators including Stratification and Adjustment Covariates