Skip to main content
  • Research Article
  • Open access
  • Published:

Development and validation of the FRAGIRE tool for assessment an older person’s risk for frailty

An Erratum to this article was published on 09 December 2016

Abstract

Background

Frailty is highly prevalent in elderly people. While significant progress has been made to understand its pathogenesis process, few validated questionnaire exist to assess the multidimensional concept of frailty and to detect people frail or at risk to become frail. The objectives of this study were to construct and validate a new frailty-screening instrument named Frailty Groupe Iso-Ressource Evaluation (FRAGIRE) that accurately predicts the risk for frailty in older adults.

Methods

A prospective multicenter recruitment of the elderly patients was undertaken in France. The subjects were classified into financially-helped group (FH, with financial assistance) and non-financially helped group (NFH, without any financial assistance), considering FH subjects are more frail than the NFH group and thus representing an acceptable surrogate population for frailty. Psychometric properties of the FRAGIRE grid were assessed including discrimination between the FH and NFH groups. Items reduction was made according to statistical analyses and experts’ point of view. The association between items response and tests with “help requested status” was assessed in univariate and multivariate unconditional logistic regression analyses and a prognostic score to become frail was finally proposed for each subject.

Results

Between May 2013 and July 2013, 385 subjects were included: 338 (88%) in the FH group and 47 (12%) in the NFH group. The initial FRAGIRE grid included 65 items. After conducting the item selection, the final grid of the FRAGIRE was reduced to 19 items. The final grid showed fair discrimination ability to predict frailty (area under the curve (AUC) = 0.85) and good calibration (Hosmer-Lemeshow P-value = 0.580), reflecting a good agreement between the prediction by the final model and actual observation. The Cronbach's alpha for the developed tool scored as high as 0.69 (95% Confidence Interval: 0.64 to 0.74). The final prognostic score was excellent, with an AUC of 0.756. Moreover, it facilitated significant separation of patients into individuals requesting for help from others (P-value < 0.0001), with sensitivity of 81%, specificity of 61%, positive predictive value of 93%, negative predictive value of 34%, and a global predictive value of 78%.

Conclusions

The FRAGIRE seems to have considerable potential as a reliable and effective tool for identifying frail elderly individuals by a public health social worker without medical training.

Peer Review reports

Background

Frailty, a core geriatric concept, is considered highly prevalent and heterogeneous in its level of expression [1]. Most people aged 65 years or over lead independent live. However, as people age, progressively they are more likely to live with frailty. Twenty-five to 50% of elderly subjects older than 85 years old could be considered frail in the North American [1, 2] and European [3] countries. In the Survey of Health, Aging and Retirement in Europe (SHARE), the prevalence of frailty is estimated at 17% in Europe and 15% in France for people older than 65 years. Frailty represents therefore an important clinical and public health problem.

Significant progress has been made to understand its pathogenesis process and several definitions of this concept have been proposed. Despite a recent large interest on the subject, and various models, definitions, and criteria [4], frailty is still an evolving concept [5, 6]. Nevertheless, frailty has been acknowledged consensually as a multidimensional geriatric concept combining both health status and environmental components (including sociability, accommodation and transport accessibility), but also increased vulnerability and loss of adaptability to stress [4, 7]. Frailty has been demonstrated in various populations as a predictor of negative health outcomes, such as falls, hip fractures, worsening mobility, activities of daily living disability, need for long-term care, hospitalization, and mortality. Therefore, identification of older individuals who are frail or at risk of becoming frail with appropriate subsequent tailored evaluation and intervention constitutes an important goal of geriatric medicine [8]. Properly assessed frailty indicators could prevent the dependency and thereby could provide a better quality of life to this population and have large benefits for families and society [9]. Age-related functional decline is usually a slow process including a phase during which individuals at risk for frailty can be identified and referred for preventive interventions [10].

Currently, there are only few or not adequate tools to measure frailty or risk for frailty in the elderly people. In France, the Short Emergency Geriatric Assessment (SEGAm) seems to be the most interesting instrument, but it mainly detects frailty in elderly emergency conditions and it is not fully appropriate for geriatric assessment and in turns the risk of frailty [11]. Outside the emergency context, a widely used definition of frailty proposed by Fried et al. [1] considers frailty as similar to disability, comorbidity, and other characteristics and defines it as a clinical syndrome in which three or more of the following criteria are present: unintentional weight loss, self-reported exhaustion, reduction of grip strength, slow walking speed, and low physical activity. Fried’s phenotype model could provide important information but fails to provide a complete assessment and to predict the occurrence of frailty in the general elderly population who are not yet frail [6, 12]. The frailty index, defined by a cumulative deficit approach, has emerged as a promising concept in gerontology research [13]. Rockwood deficits accumulation model is based on the idea that the frailty is measured by the number of health problems associated with age, regardless of their nature and severity. This approach is a well-recognized tool and could be described as an overall indicator of health condition of the elderly people. Nevertheless, frailty index does not refer to a clearly defined conceptual model. It is also not an equivalent method of a comprehensive geriatric assessment as practiced in medico-social situations that is structured, standardized and focused on the identification of needs for assistance and care. A recent study provides a short review of the multidimensional frailty assessments that are currently available and concluded that Comprehensive Model of Frailty should ideally be a multidimensional and multidisciplinary construct including physical, cognitive, functional, psychosocial/family, environmental, and economic factors [14].

In this context, two French institutions for the elderly people, the National Old-Age Insurance Fund (The Caisse Nationale d'Assurance Vieillesse; CNAV) and the Central Fund of Social Agricultural Mutual (The Caisse Centrale de la Mutualité Sociale Agricole; CCMSA), have been stepping up efforts to assess a new multidimensional screening tool for frailty prediction in a specific population of older subjects autonomous in their daily life (Groupe Iso-Ressource (GIR) 5 and 6 [15, 16] that can be administered by social and other healthcare workers. The GIR 5 and 6 French populations are not a systematically helped population by public health funders, thus the identification of people at risk to become frail (i.e. to become a GIR 4 or lower elderly subject after some years) in this group of elderly could allow the prevention of the frailty with an adapted support of the institutions. A recently reported postal questionnaire in the INTER-FRAIL study [17] is one such tool, however this one focuses only on two domains: autonomy and activities of daily living (derived from the Katz’s index) [18]. The Fried’s frailty criteria, strongly centered on the physical and mobility dimensions, are also by definition not adapted for the GIR 5 and 6 population.

This article describes the development and validation of the Frailty GIR Evaluation (FRAGIRE), a new frailty-screening instrument to predict the risk of frailty in a specific GIR French elderly population not yet frail that can be administrated by a public health social worker without medical training. The FRAGIRE grid construction involves conventional factors (physical, cognitive, functional, psychosocial/family, and environmental) and other dimensions unexplored potentially interesting for contemporary frailty prediction in this population (cultural, sexual, and nutritional).

Methods

Participants

A prospective multicenter recruitment of older people (>60 years old) was undertaken between May 2013 and July 2013 in Bourgogne-Franche Comté, France. Patients belonged to the GIR 5 (people need occasional help with bathing, meal preparation and housekeeping) and 6 (people still autonomous for the main activities of daily life) groups of dependency (Additional file 1). Elderly subject in states GIR 5 and 6 cannot benefit from a systematic personal autonomy allowance from French institutions, but in particular situations they may receive a financial help of 3500 euros/year (pension additional plan [PAP]) for the following benefits: home care including cleaning, laundry, help with shopping and meal preparation; meal deliveries; little assistance with using the toilet, or home installation improvement. To be eligible for the PAP attribution elderly need to detail the motivation for such request. Whatever the amount of the retirement pension received, the elderly people could be eligible for the financial help weighted according to the pension received.

Patients selection was based on a hypothesis that the elderly in GIR 5 and 6 populations who claim the PAP, contrary to those who do not (the groups matched by age and gender), are probably more at risk to become frail and thus represent an acceptable surrogate population for frailty prediction.in GIR 5 and 6 population who are not yet frail. Based on this hypothesis, the subjects were classified into one of two groups: financially helped (FH, with financial assistance) group and non-financially helped (NFH, without any financial assistance) group.

The inclusion and exclusion criteria for each population are described in Additional file 2. Written consent was obtained from all subjects and the protocol was approved by the local ethics committee.

Study design

The FRAGIRE grid was developed and validated in four phases with a cross-sectional cohort of elderly subjects (Fig. 1).

Fig. 1
figure 1

Study design: analysis and adaptation of the FRAGIRE model

The first step, phases 0 and 1, was intended to provide the FRAGIRE pre-grid for an overall assessment of frailty including all potentially relevant items. This step was performed to ensure that all the frailty dimensions are captured and that data are collected for the second step. In the phase 0, a pluridisciplinary panel of expert committee was constituted. It consisted of a geriatrician, a psychiatrist, a demographer, a methodologist, an epidemiologist, a data manager, and the social support professionals. In the phase 1 (face validity), based on the experts’ knowledge about frailty and on a comprehensive literature review the FRAGIRE pre-grid with selected items was constructed. In order to cover a priori all-important fields of frailty and to warrant face and content validity of the pre-grid, number of items in the first step was not restricted.

The second analytic step, phases 2 and 3, aimed to assess the psychometrics properties of the FRAGIRE pre-grid, to reduce the number of items, to generate a frailty prognostic score to predict the probability of needing assistance from the French retirement aide system and thus by analogy the frailty based on the final FRAGIRE grid. In this step, criterion validity was also assessed by exploring the degree of concordance between the results from the final FRAGIRE grid and those of gold standards including the Medical Outcome Study Short Form-36 (SF-36) [19] and the Mini Mental State Examination (MMSE) [20]. The choice of items retained and construction of prognostic score was based on both psychometric properties analyses and experts’ recommendations. The following validation psychometrics parameters were assessed: construct validity of the general structure, dimensionality of the frailty variables with principal component analysis (PCA), convergent validity with the MMSE and SF-36 tools, discriminant validity (comparison of items response between the helped and the non-helped group), reliability including internal consistency (factorial analyses and Cronbach alpha coefficient calculations [21]), and repeatability/reproducibility (test-retest method).

Data collection procedures and instruments

For each included subject, socio-demographic parameters were collected including age, gender, and job category in the pre-retirement period.

The FRAGIRE pre-grid was administered at inclusion (day 0). Items reproducibility was measured between two administrations of the pre-grid 3 days (maximum) apart. Majority of items were rated according to a 4-point Likert scale: 1) “not at all”, 2) “a little”, 3) “quite a bit”, and 4) “very much”.

In addition, participants were asked to fill out the SF-36 and MMSE questionnaires. The SF-36 is a 36-item well validated generic instrument measuring: physical functioning, role-physical, bodily pain, general health, vitality, social functioning, emotional role, and mental health. One score was generated per dimension on a 0–100 scale [19] with a high score reflects a high health-related quality of life level. The MMSE is a 30-item questionnaire evaluating various dimensions of cognition. The MMSE global score was generated as an index of global cognitive performance ranging from 0 to 30 (worst to best) [20]. Falls risks were assessed by the specific questionnaire, as per the recommendation of the French National Center of the Organization of Health Examination Centers (Centre Technique d'Appui et de Formation des Centre d'Examen de Santé [CETAF]). Questions were clearly enunciated to the elderly people and completed by a social worker according to the given responses (i.e. hetero-assessment). When an answer was not available in the item scale proposed, the social worker received the instruction to report a missing data.

In addition to the SF-36 and the MMSE, three other instruments were used. The Memory Impairment Screen (MIS) is a very brief 4-item screening tools for dementia. Patients score between 0 and 8 points, and a score of 5–8 is used to show no cognitive impairment while a score of less than 5 is used to show possible cognitive impairment [22]. The Isaacs Set Test (IST), consisting of generating a list of words (10 maximum) belonging to semantic categories in 15 s, evaluates verbal fluency abilities and speed of verbal production. Four semantic categories were successively used (cities, fruits, animals, and colors). A single score was generated ranges from 0 to 40, with higher score indicating better cognitive status [23]. The clock-drawing test (CDT) is a fast screening tool for cognitive impairment and dementia and can be used as a measure of spatial dysfunction and neglect [24].

Finally, the FRAGIRE pre-grid was reviewed with regard to clearness of the language, ambiguities, and ability of subject to understand the questionnaire without assistance.

Sample size

The primary endpoint for questionnaire validation was reproducibility/repeatability using intraclass correlation coefficient (ICC) of the final score. Considering a priori introduced dimensions and a posteriori estimated ICC, the null hypothesis H0 of none agreement between two measurements was rejected if estimated ICC was 0.5 to and the alternative hypothesis H1 of reproducibility was accepted if the ICC of was at least 0.65. The type I error rate was fixed to 0.001 (Bonferroni correction, bilateral situation) and a statistical power to 80%. It was required to include at least 338 subjects. Test-retest reliability of the FRAGIRE global score was finally evaluated by ICC at an alpha type I error rate fixed at 0.05. For all other analyses, P < .05 was considered statistically significant.

Statistical analysis

Mean (standard deviation) or median (range) values and frequencies (percentages) were provided for the description of continuous and categorical variables, respectively. The two groups were compared for means, medians, and proportions using Student’s t-test, non-parametric Mann–Whitney test, and chi-square test (or Fisher’s exact-test, if appropriate), respectively. The main psychometrics properties of the FRAGIRE pre-grid were evaluated using both classical tests and item response theory (IRT). Acceptability and feasibility were assessed regarding response rates and missing values. The construct validity and dimensional structure of the questionnaire were assessed using both PCA and IRT. Items of low clinical added value to dimension information were eliminated during the reduction phase, examining correlations between the item scores and dimension. A partial credit model by dimension derived from IRT model [25] will be reported elsewhere. Item-discriminant ability between the FH and the NFH group was assessed using Mann–Whitney test by comparing item response categories between groups. If a significant difference between items distribution among populations was observed, the item discrimination ability was supported. The PCA correlation circle also exhibited the items discrimination ability (contribution to the PC axes) and allowed us to visualize how they mutually interact (correlation). Reliability was evaluated by investigating both internal consistency and repeatability of the FRAGIRE measure using Cronbach’s alpha coefficients, which were computed across items to estimate the global internal consistency reliability and the internal consistency of each dimension. An alpha coefficient of 0.70 or higher was considered as acceptable [21, 26]. Uncertainties around Cronbach’s alpha coefficients were measured with a bootstrapping with calculation of a 95% confidence interval (95% CI). Repeatability was assessed by investigating changes in items response categories from day 0 to day 3 using Wilcoxon non-parametric test. An item was excluded if it demonstrated: missing value exceeding 10% (suggesting that subject had difficulty responding to the item); no discrimination ability, no added value in PCA, two items presenting quasi-complete positive or negative correlation (opposed on the PCA) induce the deletion of one item, and/or limited role in PCA correlation circle. Items were selected into the final grid based on the following criteria: high discrimination ability, large or acceptable contribution to PCA correlation circle, or clinically relevant items based on the choice of the expert group. The psychometrics properties of the final FRAGIRE grid were assessed after the item reduction phase.

For the phase 3, a global scoring system based on the selected items of the final FRAGIRE grid was developed, with items and tests as continuous variables. The association between items response and tests with “help requested status” was assessed in univariate and multivariate unconditional logistic regression analyses.

The predictive value and the discrimination ability [27] of the final model was evaluated with area under the curve (AUC) index, while calibration and goodness of fit of the model were assessed using Hosmer-Lemeshow test (i.e. the ability to provide unbiased predictions in groups of similar people). A high P-value (>0.1) was considered as an indicator for acceptable calibration. Bootstrapping [28] was used for internal validation of the model.

A score to predict help requested status was constructed and weighted with beta coefficients estimations from the final multivariate regression model. The possible changes in parameters were taken into account when the expert group suggests it. A prognostic score between 0 and 100 to predict the probability of needing assistance from the French retirement aide system and thus by analogy the frailty based on the final full model was calculated for each individual The FRAGIRE prognostic score, calculated for each subject, was normalized on a 0 to 100 scale with the highest score representing the most frail. A receiver operating characteristic (ROC) curve was constructed, with calculation of the AUC, to check discriminant capability of the score. The Youden index was used to identify the optimal threshold value [29]. Repeatability of prognostic score was also assessed by ICCs [30] Linear regression and Pearson’s coefficient correlation between the prognostic score at day 0 and day 3 were also computed. All analyses were performed using SAS version 9.3 (SAS Institute) and R software version 2.15.2 (R Development Core Team).

Results

The characteristics of the two population groups (FH and NFH) are presented in Table 1. Overall, 385 retired elderly subjects, 338 (88%) in the FH group and 47 (12%) in the NFH group, were included.

Table 1 Baseline characteristics comparison between the two groups of patients (N = 385)

The FRAGIRE pre-grid

For the phase 1, 65 items (Q1–Q65) describing 10 dimensions were identified (see Additional file 3): overall health status (4 items), emotional dimension (15 items), cognitive impairment (2 items plus 5 tests), environmental (9 items), cultural (2 items), sexual (4 items), burden of help (3 items), nutritional (8 items), neurosensory (6 items), mobility (9 items with 1 test), and proxy assessment of frailty by the social worker (3 items). This step resulted in a 65-item and 3-test grid (tests related to cognitive dimension: MIS, IST, and CDT) that administration lasted approximately 45 min. Tables 2 and 3 display the items of the FRAGIRE pre-grid and the distribution of responses rates. Most items have a large majority of responses. The maximal missing-item rates were 18% on day 0 and 21% on day 3. The items Q18, Q23, and Q39 were unanswered on day 0 by 16, 16, and 18% of subjects, respectively (Tables 2 and 3).

Table 2 The FRAGIRE pre-specified grid for dimension scores on day 0
Table 3 FRAGIRE pre-grid items distributions between the two measurements (on day 0 and day 3)

The comparison of scores in each item, considering the FH and NFH groups, showed discrimination power (Mann–Whitney test P < 0.05) between the two groups with respect to 26 items (Q1, Q2, Q5, Q6, Q8, Q16, Q18, Q24, Q29, Q30, Q31, Q33, Q37, Q38, Q44, and Q55 to Q65).

Overall health, emotional, and separate examiner dimensions showed good internal consistency, with Cronbach’s alpha coefficient of > 0.80 (Table 4). Low internal consistency, with a coefficient of < 0.50, was associated with cognitive, cultural, sexual, and neurosensory dimensions. Internal consistency was high for the whole scale (a coefficient of 0.87). Wilcoxon non-parametric test of the FRAGIRE pre-grid showed good reproducibility between the two evaluations (on day 0 and on day 3) except for three items: Q9 (P = .011), Q14 (P = .013), and Q17 (P = .045).

Table 4 Cronbach alpha coefficient estimation before and after the-items-selection procedure

Given the scoring heterogeneity (items scored as either 2 or 8 according to examiner) of the CDT and its poor observed compliance (53% and 58% of data available on day 0 and day 3, respectively), this test was no longer considered in the study.

A first stage of items selection process was based on completion rates and the extend of missing data on day 0 (Table 2). Eight items (12%; Q18, Q23, Q39, Q52, Q53, Q58, Q60, and Q62) were excluded at this stage. Five of those (Q52, Q53, Q58, Q60, and Q62) demonstrated a high rate of missing data due to the inter-item correlation therefore too difficult to handle in a scoring system. At a second stage of an elimination process (based on the item distribution comparison between the two groups (Table 2) and the PCA analysis of all dimensions made of at least two items [data not shown]), a total of 37 items were deleted due to: lack of discrimination ability (Q20, Q21, Q22, Q48, and Q50), lack of discrimination ability and no particular interest to PCA (Q26, Q27, Q28, and Q47), and lack of discrimination ability and presence of quasi-complete positive or negative correlation (Q7, Q9, Q11, Q12, Q13, Q14, Q15, Q17, Q25, Q35, Q41, Q42, Q45, Q46, Q49, and Q51). Moreover, eight items (Q2, Q6, Q10, Q29, Q33, Q43, Q59, and Q64) with almost complete correlation or rated as not relevant by a panel of experts were excluded despite their discrimination power. The final four items (Q3, Q57, Q61, and Q65) were removed due to their limiting role in PCA correlation circle. Two items, Q37 and Q38, composing “burden in help” dimension were combined in one single item in order to synthetize and simplify information from both items. The final set of items excluded were discussed and validated by a panel of experts.

The final FRAGIRE grid

The selection process resulted in the final FRAGIRE grid composed of 19 items describing 9 dimensions (with examiner section) and 2 tests (see Additional file 4). Of 19 items, 11 (58%) had high discrimination ability and contribution in PCA correlation circle (Q1, Q5, Q8, Q16, Q24, Q30, Q31, Q44, Q55, Q56, and Q63), four (Q4, Q34, Q40, and Q54) had only an acceptable contribution in PCA correlation circle, and three (Q19, Q32, and Q36) were chosen by the expert panel independently of the statistical results. The choice of the 19 items kept in the final FRAGIRE grid was confirmed by IRT analysis (data not shown). The final 19 items of the final FRAGIRE grid demonstrated an excellent reproducibility with no statistically significant distribution of changes between day 0 and day 3 (Table 3). The structure of the final grid was supported by PCA (Fig. 2). Cronbach’s alpha was 0.69 (95%CI: 0.64–0.74), satisfying the consistency reliability (Table 4).

Fig. 2
figure 2

Principal component analysis with the items retained in the final FRAGIRE grid: Panel a shows the projection of individuals on the two principal component and Panel b shows the correlation circle providing the item’s interaction and contribution to the axes for component 1 and 2 on which the individual projection is made in the Panel a. Each axis can be considered as a linear combination of items weighted by their importance. Abbreviations: PC: Principal component; FH: financially helped group; NFH: Non-financially helped group

Elaboration of a prognostic score

Of the final 19 FRAGIRE items, 16 were used for the prognostic score construction (For a detailed description see Additional file 5). Two items, Q34 and Q36, describing sexual dimension, were included in the construct with a view to future analysis, and one item, Q19 describing suicide dimension, given its non-neglected positive response rate was kept with public health screening in mind.

The “Set Test d’Isaacs” (STI) and the “Score de mémoire avec Indicage” (SMI) tests were maintained to assess the cognitive dimension (not included in prognostic score) and to provide complementary data for frailty evaluation (Additional files 6 and 7).

PCA, Cronbach alpha coefficient, and IRT results ensured an acceptable context for the prognostic score construction. PCAs conducted on the initial and final grids (Fig. 2) showed that the major part of the variance in data was explained by a first principal component (axis), which justified a unidimensional approach for the construction of frailty prognostic score. In fact, 18% and 6% of the variance in the 65-item grid was accounted for by the first two principal components, reflecting the importance of the first principal component.

In the final multivariate 19-item model (N = 339), six independent factors (Q5, Q24, Q30, Q31, Q32, and Q44) were found to be independently associated with “request help status” (P < .1) (Table 5). The model exhibited excellent discrimination ability (AUC = 0.85) and good calibration (Hosmer-Lemeshow P = 0.5800), reflecting an optimal agreement between prediction by the final model and actual observation. Bootstrapping results for internal validation reflected the robustness of the final model, especially for parameters significantly associated with “help requested status” (Table 5). The FRAGIRE prognostic score was normally distributed with a mean score of 55.7 (±10.5). In the FH group, the average score was significantly higher than in the NFH group (57.1 [±9.5] vs 46.4 [±12.1]; P < .0001). The score exhibited excellent discrimination ability (AUC 0.756) (Fig. 3). A score of 49.5 allowed efficiently and significantly discriminate individuals requesting for help from others (P < .0001), with sensitivity of 81%, specificity of 61%, positive predictive value of 93%, negative predictive value of 34%, and a global predictive value of 78%. When the elderly population is to be divided in three groups of interest (low, intermediate, and high probability of request help), FRAGIRE score tertiles (P33 = 52; P66 = 63) and the ROC curves discriminated between the groups with thresholds of 50 and 60.

Table 5 Univariate and multivariate unconditional logistic analyses on determinant and status of help beneficiary
Fig. 3
figure 3

Receiver Operating Characteristic Curve for the prognostic score (AUC = 0.756)

Linear regression and Pearson correlation analysis of the FRAGIRE prognostic scores between day 0 and day 3 (N = 293) showed an excellent correlation between the two measurements (R2 = 0.74, P < 0.0001 and R2 = 0.86, P < 0.0001, respectively, Fig. 4). Intraclass correlation coefficient scores were also excellent allowing a rejection of H0 (ICC > 0.86 for all methods, Table 6).

Fig. 4
figure 4

Linear regression between the individual prognostic score on day 0 and 3 (N = 293, R2 = 0.74, P < .0001)

Table 6 Intraclass correlations for inter-rater reliability

The FRAGIRE prognostic score significantly (P < .05) and negatively correlated with the MMSE global score and all dimensions of the SF-36, reflecting a satisfactory convergent validity (Table 7).

Table 7 Prognostic score correlation with the Mini Mental State Examination score and the SF-36 dimensions

Discussion

This paper describes the development and validation of a new frailty-specific instrument, the Frailty GIR Evaluation (FRAGIRE) consisting of 19 clinically relevant health or environmental items based on literature review and expert recommendations. The instrument showed good discriminative capability, sensitivity and specificity as reflected by the AUC analysis, good reliability with the Hosmer Lemeshow assessment of the calibration,, and excellent construct convergent validity with the strong correlation between the score and MMSE and SF-36 results. The Cronbach's alpha for the developed tool scored as high as 0.69. with a 95% bootstrap confidence interval equal to (0.64–0.73,) was considered as an acceptable result for this analysis as the 0.7 value was included in the confidence interval. This analysis demonstrated that the FRAGIRE instrument is clinically sensible and discriminates between groups of elderly.

The originality of our research was to provide a multidimensional tool to measure frailty and produce new simple prognostic score based on selected items and dimensions to identify high-risk frail older subjects. The great advantage of the tool is its easy implementation by a public health social worker without formal training in geriatric care. Noticeably, the final FRAGIRE tool showed an agreement for all selected items recorded on day 0 and day 3, highlighting an excellent reproducibility of these items.

Di Bari et al. recently developed and tested a 10-item screening questionnaire to intercept frailty in large cohort of older community-dwelling individuals.5 Compared with this Italian model, the 19-item FRAGIRE grid has advantages because it includes emotional and environmental aspects in addition to functional status, and seems to present a better discriminatory ability, has been rigorously tested for repeatability and convergent validity, and assesses multiple domains.

Each item in the final FRAGIRE tool was included as clinically necessary and relevant. Self-assessment of frailty by the individuals themselves (in the global health status dimension), a measure that provides an idea of its positioning compared to non-frail people of similar age, appeared to be a good component of initial assessment with good discrimination ability and an acceptable contribution to principal components in the PCA analysis. Hospitalization, the deciding factor in the functional ability of the frail elderly [31], likewise showed these properties. Three items in the psychological dimension, general well-being, happiness, and tiredness, were also retained in the final tool due to their clinical relevance that is close association with frailty [32]. We considered that these items would prompt the dynamism of the structure. Our a priori choice strategy was confirmed by statistical analyses showing that this structure had good discrimination ability and an acceptable contribution for all those items. In the environmental dimension, feeling of loneliness and/or abandonment and financial situation level were kept in the final FRAGIRE grip as these appeared the most relevant in terms of discrimination ability. These social factors, including isolation and financial situation, have been shown to be involved in the vulnerability process [33]. Despite a low internal consistency (Cronbach’s coefficient of < 0.50), two items in the socio-cultural dimension, use of Internet and participation to group activities, were maintained in the final grid due to their high discrimination abilities and contribution to PCA and due to clinical relevance recognized by the expert group, respectively. The structure incorporating these characteristics may be more successful in targeting social isolation and adaptability in older people. Four other variables, responsibility towards relatives (burden of help dimension), the number of falls within the last 6 months, physical difficulties, and walking speed (mobility dimension) were also retained as relevant in the final FRAGIRE tool as these attest to the dynamism, the non-sedentary and the non-social isolation of assessed persons [23], or showed high discrimination ability and contribution in PCA correlation. The three mobility items were shown to be strongly associated with frailty.1

Although some items were not included in the final score, these were retained due to their importance from a public health perspective. For instance, the FRAGIRE scale contains a suicide item that can be highly relevant in the assessment of the elderly. Suicide is specifically of concern in older adults as suicide rates increase with advanced age. However despite its potential as risk factor, suicide in the elderly people still receives little focus in terms of specific preventive strategies or research. Our analysis showed that suicide ideas were more frequent in our population (8%) than in the general population according to the 2010 Health Barometer in France (3.9%) [34], which emphasizes the importance of detection of the suicide risk in the elderly population. Even if our data do not show statistically significant correlation with frailty, we believe that the collection of this information for suicide prevention policies is of interest. Along the same line of though, the cognitive dimension with MIS-IST pairing was retained in the final model. The MIS-IST pairing is quick and simple to score and the efficacy of the MIS and IST combination in predicting short-term development of dementia in a group of people with questionable dementia has been previously reported.20 Although positive results cannot be used to definitely diagnose dementia, it can be considered a useful screening procedure for all types of dementia and can be a good way of directing the elderly people towards specialized consultation. We hope that this approach in the FRAGIRE grid will help to develop specific detection and prevention strategies.

Our study has some limitations that should be noted. First, our study did not consider socioeconomic status parameter that could provide important information about health status including frailty. Indeed, we hypothesized that the elderly from GIR 5 and 6 population who claim PAP will be potentially more at risk to become frail than those who do not. Whatever the amount of the retirement pension received, the elderly people could be eligible for the financial help weighted according to the pension received. By definition, all socioeconomic status measures can be found in each group, but we cannot guarantee their balance between the two populations.

The FRAGIRE grid was developed to be enunciated to the elderly population (corresponding to a hetero-assessment). While this method seems to be more adapted to elderly population than a self-reported questionnaire regarding the targeted population and to the tests included in the grid, it can raise the issue of the inter-rater reliability for the examiner dimension. The inter-rater reliability of examiners’ judgement however could not be assessed in our study because the assessment was made by only one social worker per elderly.

Another potential limitations of our study are the difficulty encountered for NFH enrollment and that we did not compare the FRAGIRE grid with frailty measures such as the Fried and Rockwood methods. In order to prevent excessive burden in data collection by social and other healthcare workers such very time-consuming and laborious process was considered unessential at this time of the development process of the FRAGIRE tool. However, future studies could potentially address this issue.

Further, this study involves a cross-sectional design. Our findings suggest that the FRAGIRE grid should now be validated prospectively to ensure that the score could predict frailty and thus help to make decision on resources allocation. The FRAGIRE tool is currently in use in France and is being tested in a prospective external validation cohort for sensitivity to change, for reproducibility to improve the proposed prognostic score, and for more accurate determination the cutoff threshold of the FRAGIRE score. The primary objective of the external validation is to assess the discriminative ability of the FRAGIRE grid for predicting the loss of autonomy; an indicator of frailty, i.e. the tilting of the elderly people to a GIR of 4 or lower from GIR 5 and 6 elderly subjects. Thus, the conduct of elderly frailty assessment will be performed in an accurate and objective way without taking into account hypothesis of the NFH and FH groups‘ frailty surrogacy. Secondary objective that include, the assessment of the status FH and NFH groups frailty surrogacy to validate the hypothesis involved in the present study. However, the internal-validation ensures a reliable estimate of performance for subjects similar to those of the present development sample. Another limitation is that the FRAGIRE score can only be estimated if all items and tests are answered. It would be important to perform a missing data sensitivity analysis on the prospective validation cohort with the items selected in the final FRAGIRE grid to assess their potential association with frailty status observed and to propose, if an association is highlighted, an alternative in the determination of the prognostic score.

Conclusion

In summary, the FRAGIRE grid and derived instruments have been constructed in response to a lack of any validated tool for frailty screening in the GIR 5 and 6 French population. It appears to be a potential reliable and effective tool for identifying elderly individuals at risk to become frail by a public health social worker without formal training in geriatric care and for providing a simple prognostic score for frailty prediction.

Abbreviations

AUC:

Area under the curve

CCMSA:

Caisse Centrale de la Mutualité Sociale Agricole

CDT:

Clock-drawing test

CETAF:

Centre Technique d'Appui et de Formation des Centre d'Examen de Santé

CI:

Confidence interval

CNAV:

Caisse Nationale d'Assurance Vieillesse

FH:

Financially-helped

FRAGIRE:

Frailty Groupe Iso-Ressource Evaluation

GIR:

Groupe Iso-Ressource

ICC:

Intraclass correlation coefficient

IRT:

Item response theory

IST:

Isaacs Set Test

MIS:

Memory Impairment Screen

MMSE:

Mini Mental State Examination

NFH:

Non-financially helped group

PAP:

Pension additional plan

PCA:

Principal component analysis

ROC:

Receiver operating characteristic

SEGAm:

Short Emergency Geriatric Assessment

SF-36:

Medical Outcome Study Short Form-36

SHARE:

Survey of Health, Aging and Retirement in Europe

References

  1. Fried LP, Tangen CM, Walston J, Newman AB, Hirsch C, Gottdiener J, Seeman T, Tracy R, Kop WJ, Burke G, McBurnie MA, Cardiovascular Health Study Collaborative Research Group. Frailty in older adults: evidence for a phenotype. J Gerontol A Biol Sci Med Sci. 2001;56:M146–56.

    Article  CAS  PubMed  Google Scholar 

  2. Song X, Mitnitski A, Rockwood K. Prevalence and 10-year outcomes of frailty in older adults in relation to deficit accumulation. J Am Geriatr Soc. 2010;58:681–7.

    Article  PubMed  Google Scholar 

  3. Santos-Eggimann B, Cuénoud P, Spagnoli J, Junod J. Prevalence of frailty in middle-aged and older community-dwelling Europeans living in 10 countries. J Gerontol A Biol Sci Med Sci. 2009;64:675–81.

    Article  PubMed  Google Scholar 

  4. Hogan DB, MacKnight C, Bergman H, Steering Committee, Canadian Initiative on Frailty and Aging. Models, definitions, and criteria of frailty. Aging Clin Exp Res. 2003;15(3 Suppl):1–29.

    PubMed  Google Scholar 

  5. Rockwood K, Fox RA, Stolee P, Robertson D, Beattie BL. Frailty in elderly people: an evolving concept. CMAJ Can Med Assoc J J Assoc Medicale Can. 1994;150:489–95.

    CAS  Google Scholar 

  6. Gillick M. Pinning down frailty. J Gerontol A Biol Sci Med Sci. 2001;56:M134–5.

    Article  CAS  PubMed  Google Scholar 

  7. Fried LP, Ferrucci L, Darer J, Williamson JD, Anderson G. Untangling the concepts of disability, frailty, and comorbidity: implications for improved targeting and care. J Gerontol A Biol Sci Med Sci. 2004;59:255–63.

    Article  PubMed  Google Scholar 

  8. Clegg A, Young J, Iliffe S, Rikkert MO, Rockwood K. Frailty in elderly people. Lancet Lond Engl. 2013;381:752–62.

    Article  Google Scholar 

  9. Sirven N, Rapp T. The cost of frailty in France. Eur J Health Econ HEPAC Health Econ Prev Care. 2016.

  10. Ferrucci L, Guralnik JM, Simonsick E, Salive ME, Corti C, Langlois J. Progressive versus catastrophic disability: a longitudinal view of the disablement process. J Gerontol A Biol Sci Med Sci. 1996;51:M123–30.

    Article  CAS  PubMed  Google Scholar 

  11. Schoevaerdts D, Biettlot S, Malhomme B, Rezette C, Gillet J-B, Vanpee D, Cornette P, Swine C. Identification précoce du profil gériatrique en salle d’urgences : présentation de la grille SEGA. Rev Gériatrie. 2004;29:169–78.

    Google Scholar 

  12. Rothman MD, Leo-Summers L, Gill TM. Prognostic significance of potential frailty criteria. J Am Geriatr Soc. 2008;56:2211–6.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Rockwood K, Mitnitski A. Frailty in relation to the accumulation of deficits. J Gerontol A Biol Sci Med Sci. 2007;62:722–7.

    Article  PubMed  Google Scholar 

  14. Kwan JSK, Lau BHP, Cheung KSL. Toward a comprehensive model of frailty: an emerging concept from the Hong Kong centenarian study. J Am Med Dir Assoc. 2015;16:536.e1–7.

    Article  Google Scholar 

  15. Vetel JM, Leroux R, Ducoudray JM. AGGIR. Practical use. Geriatric Autonomy Group resources needs. Soins Gérontologie. 1998;13:23–27.

    Google Scholar 

  16. Aguilova L, Sauzéon H, Balland É, Consel C, N’Kaoua B. AGGIR scale: a contribution to specifying the needs of disabled elders. Rev Neurol (Paris). 2014;170:216–21.

    Article  CAS  Google Scholar 

  17. Di Bari M, Profili F, Bandinelli S, Salvioni A, Mossello E, Corridori C, Razzanelli M, Di Fiandra T, Francesconi P. Screening for frailty in older adults using a postal questionnaire: rationale, methods, and instruments validation of the INTER-FRAIL study. J Am Geriatr Soc. 2014;62:1933–7.

    Article  PubMed  Google Scholar 

  18. Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. The index of ADL: a standardized measure of biological and psychosocial function. JAMA. 1963;185:914–9.

    Article  CAS  PubMed  Google Scholar 

  19. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30:473–83.

    Article  PubMed  Google Scholar 

  20. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–98.

    Article  CAS  PubMed  Google Scholar 

  21. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.

    Article  Google Scholar 

  22. Buschke H, Kuslansky G, Katz M, Stewart WF, Sliwinski MJ, Eckholdt HM, Lipton RB. Screening for dementia with the memory impairment screen. Neurology. 1999;52:231–8.

    Article  CAS  PubMed  Google Scholar 

  23. Isaacs B, Kennie AT. The Set test as an aid to the detection of dementia in old people. Br J Psychiatry J Ment Sci. 1973;123:467–70.

    Article  CAS  Google Scholar 

  24. Sunderland T, Hill JL, Mellow AM, Lawlor BA, Gundersheimer J, Newhouse PA, Grafman JH. Clock drawing in Alzheimer’s disease. A novel measure of dementia severity. J Am Geriatr Soc. 1989;37:725–9.

    Article  CAS  PubMed  Google Scholar 

  25. Masters GN. A rasch model for partial credit scoring. Psychometrika. 1982;47:149–74.

    Article  Google Scholar 

  26. Cohen J. Statistical Power Analysis for the Behavioral Sciences. United-Kingdom: Routledge; 2013.

  27. Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16:965–80.

    Article  CAS  PubMed  Google Scholar 

  28. Efron B. Bootstrap methods: another look at the Jackknife. Ann Stat. 1979;7:1–26.

    Article  Google Scholar 

  29. Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3:32–5.

    Article  CAS  PubMed  Google Scholar 

  30. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.

    Article  CAS  PubMed  Google Scholar 

  31. Lang P-O, Michel J-P, Zekry D. Frailty syndrome: a transitional state in a dynamic process. Gerontology. 2009;55:539–49.

    Article  PubMed  Google Scholar 

  32. St John PD, Tyas SL, Montgomery PR. Life satisfaction and frailty in community-based older adults: cross-sectional and prospective analyses. Int Psychogeriatr IPA. 2013;25:1709–16.

    Article  Google Scholar 

  33. Boulos C, Salameh P, Barberger-Gateau P. Malnutrition and frailty in community dwelling older adults living in a rural setting. Clin Nutr Edinb Scotl. 2015.

  34. Husky MM, Guignard R, Beck F, Michel G. Risk behaviors, suicidal ideation and suicide attempts in a nationally representative French sample. J Affect Disord. 2013;151:1059–65.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank all the retired elderly subjects and the public health social workers that were involved in this study.

Funding

None.

Availability of data and materials

Data are unsuitable for public deposition due to ethical and legal restrictions and are therefore available upon request with the signature of a data privacy form. To request the data, the readers may contact Prof. Franck Bonnetain (email: franck.bonnetain@univfcomte.fr).

Authors’ contributions

DV, AA, FB conceived and designed the experiments, analyzed the data, contributed reagents/materials/analysis tools and wrote the paper. PV, PM, conceived and designed the experiments, performed the experiments, contributed reagents/materials/analysis tools and wrote the paper. AP, SPB analyzed the data, contributed reagents/materials/analysis tools and wrote the paper. AF analyzed the data, contributed reagents/materials/analysis tools. MD, conceived and designed the experiments, performed the experiments, contributed reagents/materials/analysis tools. VB conceived and designed the experiments, performed the experiments, contributed reagents/materials/analysis tools. MB performed the experiments, contributed reagents/materials/analysis tools and wrote the paper. MBE wrote the paper. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Written consent was obtained from all subjects and the protocol was approved by the local ethics committee in Besancon (CPP : Comité de Protection des Personnes) and by CCTIRS and CNIL french institutions (CCTIRS : Comité Consultatif sur le Traitement de l’Information en matière de Recherche dans le domaine de la Santé ; CNIL : Commission Nationale de l’Informatique et des Libertés/ Favorable opinions received on April 24 and May 28 2013, respectively).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Franck Bonnetain.

Additional information

The original version of this article was revised: The author name Manckoundia had been incorrectly spelt as Mankoundia, but has now been corrected.

An erratum to this article is available at http://dx.doi.org/10.1186/s12877-016-0385-0.

Additional files

Additional file 1:

English and French version of the AGGIR grid. (DOCX 112 kb)

Additional file 2:

Definition of the eligibility criteria in the financially and non-financially helped group of subjects enrolled in the study. (DOCX 13 kb)

Additional file 3:

The FRAGIRE “pre-grid” (A) English version and (B) Original version in French. (DOCX 65 kb)

Additional file 4:

The final FRAGIRE grid (A) English version and (B) Original version in French. (DOCX 45 kb)

Additional file 5:

Details on the construction of the prognostic score. (DOCX 46 kb)

Additional file 6: Table S1.

Description of the Memory Impairment Screen results in the overall population and according to the financially helped status at day 0 and day 3. (DOCX 15 kb)

Additional file 7: Table S2.

Description of the Isaacs Set Test results in the overall population and according to the financially helped status. (DOCX 15 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vernerey, D., Anota, A., Vandel, P. et al. Development and validation of the FRAGIRE tool for assessment an older person’s risk for frailty. BMC Geriatr 16, 187 (2016). https://doi.org/10.1186/s12877-016-0360-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12877-016-0360-9

Keywords