Validation of a one year fracture prediction tool for absolute hip fracture risk in long term care residents

Background Frail older adults living in long term care (LTC) homes have a high fracture risk, which can result in reduced quality of life, pain and death. The Fracture Risk Scale (FRS) was designed for fracture risk assessment in LTC, to optimize targeting of services in those at highest risk. This study aims to examine the construct validity and discriminative properties of the FRS in three Canadian provinces at 1-year follow up. Methods LTC residents were included if they were: 1) Adults admitted to LTC homes in Ontario (ON), British Columbia (BC) and Manitoba (MB) Canada; and 2) Received a Resident Assessment Instrument Minimum Data Set Version 2.0. After admission to LTC, one-year hip fracture risk was evaluated for all the included residents using the FRS (an eight-level risk scale, level 8 represents the highest fracture risk). Multiple logistic regressions were used to determine the differences in incident hip or all clinical fractures across the provinces and FRS risk levels. We examined the differences in incident hip or all clinical fracture for each FRS level across the three provinces (adjusted for age, BMI, gender, fallers and previous fractures). We used the C-statistic to assess the discriminative properties of the FRS for each province. Results Descriptive statistics on the LTC populations in ON (n = 29,848), BC (n = 3129), and MB (n = 2293) are: mean (SD) age 82 (10), 83 (10), and 84 (9), gender (female %) 66, 64, and 70% respectively. The incident hip fractures and all clinical fractures for FRS risk level were similar among the three provinces and ranged from 0.5 to 19.2% and 1 to 19.2% respectively. The overall discriminative properties of the FRS were similar between ON (C-statistic = 0.673), BC (C-statistic = 0.644) and MB (C-statistic = 0.649) samples. Conclusion FRS is a valid tool for identifying LTC residents at different risk levels for hip or all clinical fractures in three provinces. Having a fracture risk assessment tool that is tailored to the LTC context and embedded within the routine clinical assessment may have significant implications for policy, service delivery and care planning, and may improve care for LTC residents across Canada.


Background
It has been estimated that 1 in 4 Canadians will be 65 years or older by 2036 [1]. As the population ages, a greater number of older adults will need residential support such as long-term care (LTC). LTC Residents are often frail, since their multiple physical and cognitive deficits place them at high risk of falls, disability, and death [2,3]. Hip fractures are the most common type of fracture in LTC (49% of all fractures) [4]. They are more common in older adults living in LTC (49%) than in the community (29%) [4,5], and lead to more hospitalizations [6] and worsening health-related quality of life [7]. In Canada, 45% of LTC residents with hip fracture die within 12 months [8] and of the survivors, 48% are no longer ambulatory [8].
Hip fracture prediction and prevention in LTC residents receive little attention due to the multiple comorbidities and medical complexity of LTC residents [9,10] and the challenges of predicting fracture in this population. It is difficult to identify LTC residents with high fracture risk, as the commonly used fracture risk assessment tools in Canada, including the Canadian Fracture Risk Assessment Tool (FRAX) and the Canadian Association of Radiologists and Osteoporosis Canada tool (CAROC) [11][12][13][14], are not valid or generalizable for residents of LTC [15,16]. FRAX and CAROC typically provide a 10-year fracture risk assessment timeframe, which is too long, given the mean 2.4-year life expectancy of LTC residents [17]. A recent study showed that FRAX (with bone mineral density) may predict incident hip fracture at one year [18]. Bone mineral density is heavily weighted in current fracture risk assessment protocols, but bone mineral density is not feasible to obtain in LTC. In addition, FRAX is not tailored to frail, institutionalized LTC residents. Thus, fracture prediction outputs of FRAX-Canada and CAROC may not be suitable for decision making and care planning among frail LTC residents [19,20].
Recently, our team developed the Fracture Risk Scale (FRS) [21], a standardized outcome scale for identifying LTC residents at risk for fracture within one year. The FRS can be obtained from the Resident Assessment Instrument Minimum Data Set (RAI-MDS 2.0), which is a comprehensive, standardized assessment that is used upon admission and on a quarterly basis thereafter, to gather information on a wide range of socio-demographic and clinical characteristics [22][23][24]. The FRS was developed using Ontario residents' data from the RAI-MDS 2.0, the Discharge Abstract Database (DAD) and the National Ambulatory Care Reporting System (NACRS). However, the FRS was not externally validated (in a population other than the tool development sample), nor was its validity tested in other Canadian provinces. As a predictive model, the FRS' reproducibility (performing sufficiently accurate across new samples from the same target population) and transportability (performing well across samples from different but related source populations) [25] need to be tested prior widespread adoption. Therefore, we conducted this validity study to examine the FRS performance across a new sample and in different but related population of LTC residents in other provinces [21].
This study aims to: 1) assess the discriminative properties of the FRS using C-statistics and examine the construct validity of the FRS by comparing incident hip fractures and all clinical fractures (includes hip, spine, humerus, forearm, pelvis fractures) for each fracture risk levels in LTC residents across three Canadian provinces at 1-year follow up; 2) compare incident hip and all clinical fractures in LTC residents across three Canadian provinces; and 3) compare incident hip and all clinical fractures between the FRS risk levels. To examine the construct validity study of the FRS [26], we hypothesize that incident hip fractures for each fracture risk levels in LTC residents in the three Canadian provinces are not statistically different when type 1 error is ≤0.05.

Study design and population
This retrospective cohort study uses data from the RAI-MDS 2.0. In Canada, LTC facilities provide living accommodation for people who need 24-h professional health services, personal care and services, including meals, laundry, housekeeping, frequent assistance with activities of daily living, on-site supervision or monitoring to ensure safety or well-being [27,28]. LTC residents were included if they: 1) were adults admitted to LTC homes in Canada from April 1st, 2006 to March 31st, 2010; and 2) were assessed with the RAI-MDS 2.0. LTC residents were excluded if they: 1) had multiple admissions; 2) reported on the RAI-MDS 2.0 to have end stage disease, were comatose, received hospice, or respite care; 3) were expected to have a short stay; 4) had the admission assessment completed more than 14 days after the date of admission (to exclude any assessment for existing residents); or 5) had no reassessments during the one-year follow-up. The project received ethics approval from the University of Waterloo Office of Research Ethics (ORE no 17045).

Fracture rating scale
The FRS is different from existing fracture risk assessment tools in that it does not use bone mineral density and includes fracture risk factors that are relevant to the long-term care (LTC) population [21]. Moreover, to ensure that it was valid for LTC residents and easily scalable, the FRS was designed and validated using large population-based datasets that include routinely collected data from LTC residents. The FRS was developed using decision tree analysis and the included items are walking in corridor (independent, supervision to extensive assistance or total assistance/walking did not occur), wandering (no wandering to infrequent wandering, less than daily wandering, or daily wandering), falling status within the past 30 days (no/yes), cognitive performance scale (intact cognition, borderline intact or mild impairment, or moderate to very severe impairment), transfer status (how resident moves between surfaces to and from; bed, chair, wheelchair, standing position), (independent to extensive assistance, or total assistance/transfer did not occur), age greater than 85 years (no/yes), body mass index (BMI) (< 18, 18-30, or > 30 kg/m 2 ), and previous fractures in past 180 days (no/yes) [21]. The FRS includes eight fracture risk level categories (level 1 represents the lowest and level 8 represents the highest fracture risk). Within the FRS, assessment of risk continues through the decision tree until a terminal risk level is identified. For example, individuals who can walk in corridor independently with BMI > 30 are defined as fracture risk level 3. However, if BMI is between 18 and 30, risk level also depends on history of falls, fracture and cognitive performance.

Incident fractures
Over the course of one-year follow-up period, residents were classified as to the presence or absence of an incident fracture. To capture incident fractures, we accessed in-patient hospital records and emergency department records [23,24]. DAD is received directly from acute care facilities or their respective health/regional authority or ministry/department of health and it contains demographic, administrative and clinical data of all Canadian provinces except Quebec [29]. NACRS contains data for all hospital-based and community-based ambulatory care including day surgery, outpatient and community-based clinics and emergency departments [30]. NACRS data is received directly from participating facilities or from regional health authorities or ministries of health. Data collection methods may vary by facility. We were able to link DAD and NACRS data to RAI-MDS data for this analysis.

Statistical analysis
The study population demographics, prior falls and fractures, and fracture incidence estimates are expressed as mean (SD) for continuous data and counts and percentages for categorical data. Percent of incident hip fracture and all clinical fractures (hip, spine, humerus, forearm, pelvis fractures) for each FRS risk level in the three provinces were calculated. Multivariable logistic regression models were used to determine if incident hip fractures are different across the provinces and to calculate the odds ratio of incident hip fractures in each FRS risk level. In this logistic regression, the incident hip fractures were the dependent variable (DV) and provinces (Ontario was used as the reference group) and FRS risk levels (FRS risk level 1 was used as the reference level) were the independent variables (IV). We tested the significance of the interaction term of FRS and provinces in the logistic regression model to determine if the incident hip fracture in each FRS risk level is different across the provinces. The logistic regression analyses were adjusted for age, BMI, gender, fallers in the last 180 days, previous fracture, and the size of the residential home (small, medium or large). All the analyses were repeated using all incident fractures as DV. To assess the discriminative properties of the FRS for each province, we used the C-statistic. All the statistical analyses were conducted using SAS V.9.4 (SAS Institute).

Results
The final study sample includes 35,270 participants (ON = 29,848, BC = 3129 and MB = 2293) and is displayed in Fig. 1. Of the eligible residents, 703 (ON = 447, BC = 169 and MB = 87) were not included in the analysis due to data entry errors. Table 1 demonstrates the characteristics of LTC residents in the three provinces. Table 2 describes FRS items in LTC residents of the three provinces. Of the included residents, 3561 (10.1%) residents died before the end of the one-year follow-up period. The characteristics of the excluded and included resident are comparable. Of the excluded residents 48.8% aged 85 years or older, and 67.9% are female. The age, gender distribution and comorbidities are similar across the provinces; with less than 1% of LTC residents having had a hospital admission within 3 months from LTC admission. Falls, incident fractures, and BMI were similar across the provinces as well. All of the Manitoba homes in the study were from one urban centre, as the remainder of the province does not use RAI-MDS.
Incident hip fractures and all clinical fractures in all risk levels ranged from 0.5 to 19.2% and 1 to 19.2% respectively (Figs. 2 and 3, respectively). Table 3 (Table 4). When adjusting for the provinces, the odds of incident hip fractures and all clinical fractures in all the FRS risk levels were significantly different compared to level 1 with consistently increasing odds of fractures with higher FRS risk levels, as shown in Table 4.
All the interaction terms associated with FRS risk levels and provinces were not statistically significant. This indicates the similarity of incident hip fractures and all clinical fractures for each FRS risk levels in the three provinces. The overall discriminative properties of the FRS were similar between ON (c-statistic = 0.67), BC (c-statistic = 0.64) and MB (c-statistic = 0.65) samples. The proportion of hip fractures that occurred in residents with a FRS level of 1-3 ranged from 4.9-7% across provinces, whereas the proportion of hip fractures that occurred in residents with a FRS of 6-8 was higher (> 26%).

Discussion
Our study demonstrates the validity of the FRS for predicting hip and all clinical fractures in LTC residents living in several Canadian provinces. As recommended by the Cosmin initiative [26], an international initiative that aims to improve the selection of health measurement instruments; hypothesis-testing construct validity is a critical component of evaluating outcome measures' psychometric properties. This study confirmed our hypothesis and showed that incident hip and all clinical fractures for each FRS risk levels in LTC residents in the three Canadian provinces were similar, confirming that the FRS is reproducible and transportable for LTC residents living in different geographical areas [25]. Therefore, our results confirmed that FRS can be used to identify LTC residents with high risk of hip or all fractures across different Canadian provinces.
We have previously shown that the FRS is able to identify LTC residents at highest risk of fracture in the initial FRS development study [21]. The FRS is an adequate reflection of the outcome of interest (one-year incident fracture). Our study builds on this work by examining FRS performance in LTC residents in three geographical areas. A clinical prediction tool performance is determined by assessing a model's calibration and discrimination [32]. Calibration is the agreement between prediction from the model (FRS model) and observed outcomes (incident hip and all clinical fractures) and indicates a model's predictive accuracy [33], which was shown in Table 4. Discrimination is the ability of the prediction model (FRS model) to set apart participants with and without the outcome of interest (incident hip and all clinical fractures); those with the outcome of interest should have a higher predicted risk compared to those who do not have it [33] which was also shown in Table 4.
Unlike commonly used fracture risk assessment tools (FRAX and CAROC), FRS is a unique one-year fracture prediction tool that is composed of risk factors specific to LTC residents. FRS is embedded in the RAI-MDS 2.0, which is completed within 14 days of a resident entering   wandering, falling, cognitive performance scale, transfer status, age, BMI, and previous fractures.
Recommendations for preventing fracture in LTC have been developed to provide non-pharmacologic and pharmacologic strategies for fracture prevention in frail older adults living in LTC [37]. However, there is a current gap in osteoporosis treatment and fracture prevention in LTC residents [38][39][40][41][42][43]. One of the barriers to implement fracture prevention guideline is the lack of information about fracture risk assessment [44]. The easily implemented and validated FRS tool may overcome this barrier. To help LTC clinicians prescribing the appropriate intervention to residents who are identified as "at risk", our team will develop and implement an electronic Clinical Assessment Protocol (CAP) in LTCs. The Fracture Risk CAP will automatically produce recommendations for residents based on their FRS fracture risk level and will inform clinical decision making as part of the person-centered care planning process to fill the gap of fracture prevention in LTC. Other CAPs have been developed to draw the attention of the healthcare provider to a matter (such as Activities of daily living, delirium and Cardiorespiratory) that can be improved and should be considered in LTC residents' care plan [45].
Strengths of this study include the use of a large, representative sample of LTC residents with RAI-MDS 2.0 data linked with DAD and NACRS. As it is recommended to externally validate the model in samples from related source populations (LTC residents in other provinces) [25], we examined the performance of the FRS outside of the development sample. Assessing the relation between the development and validation samples allowed us to explain the FRS (clinical) transportability and (statistical) reproducibility [25].
We acknowledge that our study has limitations. We excluded LTC residents with short life expectancy based

Availability of data and materials
The data that support the findings of this study will not be shared. Our access to the de-identified record-level data is provided through the Canadian Institute for Health Information (CIHI), under license from interRAI, the  international non-profit organization that holds the copyright for the assessment itself. Other interested researchers could also submit requests to CIHI if they wish to access the study data.