- Research article
- Open Access
- Open Peer Review
Comparison of alternate scoring of variables on the performance of the frailty index
BMC Geriatrics volume 14, Article number: 25 (2014)
The frailty index (FI) is used to measure the health status of ageing individuals. An FI is constructed as the proportion of deficits present in an individual out of the total number of age-related health variables considered. The purpose of this study was to systematically assess whether dichotomizing deficits included in an FI affects the information value of the whole index.
Secondary analysis of three population-based longitudinal studies of community dwelling individuals: Nova Scotia Health Survey (NSHS, n = 3227 aged 18+), Survey of Health, Ageing and Retirement in Europe (SHARE, n = 37546 aged 50+), and Yale Precipitating Events Project (Yale-PEP, n = 754 aged 70+). For each dataset, we constructed two FIs from baseline data using the deficit accumulation approach. In each dataset, both FIs included the same variables (23 in NSHS, 70 in SHARE, 33 in Yale-PEP). One FI was constructed with only dichotomous values (marking presence or absence of a deficit); in the other FI, as many variables as possible were coded as ordinal (graded severity of a deficit). Participants in each study were followed for different durations (NSHS: 10 years, SHARE: 5 years, Yale PEP: 12 years).
Within each dataset, the difference in mean scores between the ordinal and dichotomous-only FIs ranged from 0 to 1.5 deficits. Their ability to predict mortality was identical; their absolute difference in area under the ROC curve ranged from 0.00 to 0.02, and their absolute difference between Cox Hazard Ratios ranged from 0.001 to 0.009.
Analyses from three diverse datasets suggest that variables included in an FI can be coded either as dichotomous or ordinal, with negligible impact on the performance of the index in predicting mortality.
As individuals age, their vulnerability to adverse outcomes (including death) increases. Some individuals experience a state of increased vulnerability, known as frailty, which can be quantified using a frailty index . The frailty index, introduced more than a decade ago , is a useful tool for assessing the health status of individuals, and for predicting an individual’s risk of adverse health outcomes [3, 4]. Following a standard procedure, a frailty index can be constructed as the proportion of age-related health deficits an individual has accumulated . Deficits can be any diseases, signs, symptoms, laboratory abnormalities, or functional or cognitive impairments, as long as about 30 measures are included which comprise a range of physiological systems. The more deficits one has, the higher their frailty index and the more vulnerable they are to adverse outcomes. Frailty indices demonstrate similar characteristics across diverse samples and settings, even when they employ different variables or different numbers of variables . For example, frailty index values consistently increase with age, are strongly associated with mortality, and show higher values in women than in men.
Searle et al.  proposed criteria for selecting and coding health measures for inclusion as variables in a frailty index: deficits should be age-related, associated with adverse outcomes, contain little missing data (a 5% threshold was proposed) and not saturate with age (i.e. was not present in most people by some age, typically operationalized as being present in >80% of people by age 80). Searle and colleagues suggested that each variable included in a frailty index should be mapped to a 0 to 1 interval, assigned a value of 0 when a deficit is absent and 1 when it is fully expressed. It is not known, however, whether continuous or ordinal variables should be transformed into the dichotomous 0 and 1 values, or whether intermediate ordinal scores (e.g. self-rated health of “good” or “fair”), should be assigned intermediate values (e.g. 0.25 or 0.5). It is possible that by converting continuous variables (e.g. blood pressure) into dichotomous variables (e.g. “hypertension” present/absent), important information might be lost. The purpose of this study was to systematically assess whether dichotomizing deficits included in a frailty index affects the information value of the whole index. Within each of three large datasets with different settings and different durations of follow-up, we compared two different frailty indices using the same variables, but differing in whether the variables were dichotomized or categorized as ordinal variables. Specifically, we assessed for significant differences between: 1. descriptive characteristics of each frailty index, and; 2. its predictive validity using mortality as the primary outcome.
This is a secondary analysis of three longitudinal studies: the Nova Scotia Health Survey (NSHS), the Survey of Health, Ageing and Retirement in Europe (SHARE), and the Yale Precipitating Events Project (Yale-PEP).
Nova Scotia health survey
The Nova Scotia Health Survey began in 1995 and employed a representative probability sample designed by Statistics Canada. The sample included 3227 non-institutionalized Nova Scotians aged 18-99 (mean age = 48.1, SD = 19.8). There were approximately equal number of men and women in the sample (women n = 1618, 50.1%). Demographic, anthropometric, lifestyle, and risk factor data were collected at baseline and mortality data were obtained via linkage with the National Vital Statistics database 10 years following the baseline assessment. Full details of the data collection are presented elsewhere .
The survey of health, ageing and retirement in Europe
The Survey of Health, Ageing and Retirement in Europe (SHARE) represents community-dwelling people aged 50 years and older across many European countries, and their spouses/partners . Here, we included baseline data from the first two waves of SHARE (wave 1, 2004/2005: Austria, Belgium, Denmark, France, Germany, Greece, Israel, Italy, The Netherlands, Spain, Sweden, and Switzerland; wave 2, 2006/2007: Czech Republic, Poland, and Ireland) in which 37,546 individuals participated (mean age = 64.2, SD = 10.5 years; 56% female). We excluded spouses/partners under 50 years of age (n = 1,228). Mortality at 5 years was obtained from the second (2006/2007) and third waves (2008/2009) of SHARE for all countries except Israel and Ireland, where follow up data were not collected.
The Yale precipitating events project
The Yale Precipitating Events Project (Yale-PEP) is a cohort study based in greater New Haven, Connecticut. Individuals were 70 years and older at the study’s inception (mean age = 78.4, SD = 5.3) [8, 9]. The Yale-PEP survey contains longitudinal data of 754 community-dwelling, English-speaking, non-disabled persons who were not terminally ill. At baseline, most participants were women (n = 487, 64.6%), and the majority were white (n = 682, 90.5%), with a mean Mini-Mental State Examination (MMSE)  score of 26.8 (SD = 2.50). Death data were obtained from a follow up survey 155 months after baseline. Mortality is ascertained monthly and all deaths have been confirmed via death certificates .
For each dataset, we constructed two frailty indices from baseline data following a standard procedure . Within each dataset, one frailty index (FIdichotomous) was constructed from only dichotomous health variables (indicating the absence or presence of a deficit) and in the other frailty index (FIordinal), as many variables as possible were ordinally coded (indicating the severity of a deficit, e.g. 0, 0.5, 1); both frailty indices in each dataset included the same variables and were constructed by the same member of our team. Across datasets, the frailty indices varied in the number of variables included, the proportion of variables that were only available as dichotomous measures, and in duration of follow-up. The FIordinal for NSHS included nineteen 2-level variables, two 3-level, one 4-level, and one 5-level variable (Total 23 variables, Table 1). The FIordinal for SHARE included sixty 2-level variables, one 3-level variable, two 4-level, and seven 5-level variables (Total 70 variables, Table 2). The FIordinal for Yale-PEP included twelve 2-level variables, fourteen 3-level variables, one 4-level, and six 5-level variables (Total 33 variables, Table 3). Scores on all frailty indices were calculated by dividing the sum of values on all included variables (deficits) out of the total number of non-missing variables. Participants missing >20% of data for variables in a frailty index were excluded for analysis of that frailty index .
For each frailty index, we calculated mean scores (adjusted for sex) and tested the statistical significance of differences between the FIordinal and FIdichotomous within each dataset using analyses of variance (ANOVA). To assess the impact of any differences between the FIdichotomous and FIordinal for each dataset, we verified proportionality and performed multivariable Cox regression analyses for survival. The hazard ratios (HR) and 95% confidence intervals (CIs) for the two indices were adjusted for age and sex. Finally, we evaluated the difference in the ability of both FIs to predict mortality using Receiver Operating Characteristic (ROC) curves and compared the areas under the ROC curve. The statistical significance level was set to 0.05 and all calculations were performed using PASW18. Approval for the secondary analyses presented here came from the Research Ethics Committee of the Capital District Health Authority, Halifax, Nova Scotia, Canada.
In SHARE and Yale-PEP, mean frailty index scores were significantly greater (p < 0.001) for the FIordinal compared with the FIdichotomous, whereas in NSHS the FIdichotomous was greater (p < 0.001) (Table 4). The differences in mean scores between the two frailty indices from the same dataset were less than 0.02 in all datasets, which represents less than 1 deficit for NSHS and Yale-PEP and about 1.5 deficits for SHARE. The confidence intervals obtained from the Cox regression hazard ratios for FIordinal and FIdichotomous overlapped (Table 5); absolute differences between the hazard ratios for the two FIs was 0.001 for SHARE, 0.008 for NSHS, and 0.009 for Yale-PEP. In each model, the age and sex covariates were also significant (p-value < 0.05). In Yale-PEP, 72.7% of the participants had died by 13 years follow-up, in NSHS 12.1% were deceased after 10 years, and in SHARE 11.7% had died by 5 years. The areas under the ROC curves for mortality prediction were the same for the FIordinal and FIdichotomous in each dataset (Figure 1); their absolute differences ranged from 0 (SHARE, NSHS) to 0.02 (Yale-PEP) (Table 4). This pattern did not change when analyses were stratified by sex.
In this secondary analysis of data from the NSHS, SHARE, and Yale-PEP studies, we constructed two frailty indices for each dataset using the same variables but coding them differently; one included both dichotomous and ordinal variables (FIordinal) whereas the other included only dichotomous variables (FIdichotomous). After comparing the ordinal and dichotomous frailty indices within each dataset, we found that their mean values and their ability to predict mortality were nearly identical. These findings, based on three diverse datasets, quantitatively confirm the flexible nature of the frailty index approach in relation to deficit variable coding. Our findings must be interpreted with caution. The samples from all three studies only included community-dwelling individuals. Our findings may not be generalizable to other populations such as institutionalized older adults and hospitalized patients. Even so, to maximize the generalizability of our results we included three diverse samples from different studies. The SHARE included Europeans age 50 and older with a follow up period of 5 years and Yale-PEP included Americans age 70 and older with a follow up period of 12 years. Note that the NSHS included Canadians age 18 and older with a follow up period of 10 years. In prior work we have calculated a frailty index for persons across the lifespan, starting at age 15 years . It appears that the frailty index serves as a proxy measure of ageing [2, 14, 15]. This view (of the frailty index reflecting the deficit accumulation that drives mortality) has also been developed by other groups [16–18]. Other lines of evidence point to the importance of deficit accumulation prior to age 50 as a determinant of what happens in later life. For example, genes associated with greater longevity are typically associated with less deficit accumulation at younger ages  whereas states such as intellectual disability typically are associated with higher levels of deficit accumulation at younger ages (which nevertheless increase across the life course) . Hence, there is a need to understand the impact that different scoring systems might have across the lifespan. Further, we chose to compare the predictive validity of the frailty indices using all-cause mortality at different lengths of follow-up. While the frailty index is not meant simply to predict mortality, all-cause mortality is useful here as it is dichotomous, easily verifiable and non-arbitrary.
In SHARE and Yale-PEP, the mean frailty index score was slightly higher for the FIordinal compared with the FIdichotomous, whereas in the NSHS the opposite was true. Even so, in all datasets the difference was minor, representing less than 1-1.5 items in the index. This is somewhat expected behavior for a frailty index. The criteria used to create a dichotomous variable from an ordinal one rely on the researcher (in this case, members of our research group), and it is expected that there may be differences between cutoff points among different researchers. However, this difference is expected to be minimal when at least 30 variables are included, and is expected not to be consistently higher or lower.
The mean frailty index scores differed across datasets, being highest in the oldest dataset (Yale-PEP) and lowest in the youngest (NSHS). Similarly, the mean score of the frailty indices and their predictive ability were different across datasets, but not between the two paired frailty indices within a dataset, which was the intended comparison. Across datasets, the risk associated with each increment in the frailty index crucially depends on the ambient or background level of risk, and so will differ. For this ambient risk, the outcomes of people with the lowest cores (e.g. frailty index = 0) can serve as an estimate .
The frailty index provides a useful way to quantify the accumulation of relatively small health deficits across multiple physiological systems, and to identify and grade a state of overall vulnerability to outcomes. Based on our analysis of three diverse datasets we found that, if enough variables are included, dichotomizing variables or using them in ordinal form appears to have little impact on three important properties of the frailty index: the mean score, gender differences, and the ability to predict mortality.
Nova Scotia Health Survey
Survey of Health, Ageing and Retirement of Europe
Yale Precipitating Events Project
Receiver operating characteristic
Analysis of variance
Cox hazard ratio
Mini-Mental State Examination.
Rockwood K, Mitnitski A: Frailty defined by deficit accumulation and geriatric medicine defined by frailty. Clin Geriatr Med. 2011, 27: 17-26. 10.1016/j.cger.2010.08.008.
Mitniski AB, Mogilner AJ, Rockwood K: Accumulation of deficits as a proxy measure of aging. Sci. World J. 2001, 1: 323-336.
de Vries NM, Staal JB, van Ravensberg CD, Hobbelen JSM, Olde Rikkert MGM, Nijhuis-van der Sanden MWG: Outcome instruments to measure frailty: a systematic review. Ageing Res Reviews. 2011, 10: 104-114. 10.1016/j.arr.2010.09.001.
Clegg A, Young J, Iliffe S, Rikkert MO, Rockwood K: Frailty in elderly people. Lancet. 2013, 381 (9868): 752-762. 10.1016/S0140-6736(12)62167-9.
Searle SD, Mitnitski A, Gahbauer EA, Gill TM, Rockwood K: A standard procedure for creating a frailty index. BMC Geriatrics. 2008, 8: 24-10.1186/1471-2318-8-24.
Davidson KW, Schwartz JE, Kirkland SA, Mostofsky E, Fink D, Guernsey D, Shimbo D: Relation of inflammation to depression and incident coronary heart disease (from the Canadian Nova Scotia Health Survey [NSHS95] Prospective Population Study). Am J Cardiol. 2009, 103: 755-761. 10.1016/j.amjcard.2008.11.035.
Börsch-Supan A, Jürges J: The Survey of Health, Ageing and Retirement in Europe—Methodology. 2005, Mannheim: Mannheim Research Institute for the Economics of Aging, Available at http://www.share-project-org
Gill TM, Desai MM, Gahbauer EA, Holford TR, Williams CS: Restricted activity among community-living older persons: incidence, precipitants, and health care utilization. Ann Intern Med. 2001, 135: 313-321. 10.7326/0003-4819-135-5-200109040-00007.
Gill TM, Gahbauer EA, Allore HG, Han L: Transitions between frailty states among community-living older persons. Arch Intern Med. 2006, 166: 418-423. 10.1001/archinte.166.4.418.
Folstein MF, Folstein SE, McHugh PR: Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975, 13 (3): 189-198.
Gill TM, Gahbauer EA, Han L, Allore HG: Trajectories of disability in the last year of life. N Engl J Med. 2010, 362: 1173-1180. 10.1056/NEJMoa0909087.
Theou O, Brothers TD, Rockwood MR, Haardt D, Mitnitski A, Rockwood K: Exploring the relationship between national economic indicators and relative fitness and frailty in middle-aged and older Europeans. Age Ageing. 2013, 42 (5): 614-619. 10.1093/ageing/aft010.
Rockwood K, Song S, Mitnitski A: Changes in relative fitness and frailty across the adult lifespan: evidence from the Canadian National Population Health Survey. CMAJ. 2011, 183 (8): E487-E494. 10.1503/cmaj.101271.
Mitnitski A, Bao L, Skoog I, Rockwood K: A cross-national study of transitions in deficit counts in two birth cohorts: implications for modeling ageing. Exp Gerontol. 2007, 42 (3): 241-246. 10.1016/j.exger.2006.10.001.
Mitnitski A, Song X, Rockwood K: Assessing biological aging: the origin of deficit accumulation. Biogerontology. 2013, [Epub ahead of print]
Goggins W, Woo J, Sham A, Ho S: Frailty index as a measure of biological age in a Chinese population. J Gerontol A Biol Sci Med Sci. 2005, 60 (8): 1046-1051. 10.1093/gerona/60.8.1046.
Yashin AI, Arbeev KG, Kulminski A, Akushevich I, Akushevich L, Ukraintseva SV: Health decline, aging and mortality: how are they related?. Biogerontology. 2007, 8 (3): 291-302. 10.1007/s10522-006-9073-3.
Yashin AI, Arbeev KG, Kulminski A, Akushevich I, Akushevich L, Ukraintseva SV: What age trajectories of cumulative deficits and medical costs tell us about individual aging and mortality risk: findings from the NLTCS-Medicare data. Mech Ageing Dev. 2008, 129 (4): 191-200. 10.1016/j.mad.2007.12.005.
Yashin AI, Arbeev KG, Wu D, Arbeeva LS, Kulminski A, Akushevich I, Culminskaya I, Stallard E, Ukraintseva SV: How lifespan associated genes modulate aging changes: lessons from analysis of longitudinal data. Front Genet. 2013, 22: 4-3
Schoufour JD, van Wijngaarden J, Mitnitski A, Rockwood K, Evenhuis HM, Echteld MA: Characteristics of the least frail adults with intellectual disabilities: a positive biology perspective. Res Dev Disabil. 2013, [Epub ahead of print]
Mitnitski A, Bao L, Rockwood K: Going from bad to worse? Transitions between the number of accumulated health deficits in relation to mortality. Mech Ageing Dev. 2006, 127: 490-493. 10.1016/j.mad.2006.01.007.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2318/14/25/prepub
Funding Sources: KR receives funding from the Dalhousie Medical Research Foundation as the Kathryn Allen Weldon Professor of Alzheimer Research. KR and AM are supported by operating grants from the Canadian Institutes of Health Research. OT is supported by a Banting Fellowship.
We thank Karina Davidson and Daichi Shimbo, investigators for the Nova Scotia Health Survey, for making the data available to us.
This paper uses data from SHARE wave 4 release 1.1.1, as of March 28th 2013 or SHARE wave 1 and 2 release 2.5.0, as of May 24th 2011 or SHARELIFE release 1, as of November 24th 2010. The SHARE data collection has been primarily funded by the European Commission through the 5th Framework Programme (project QLK6-CT-2001-00360 in the thematic programme Quality of Life), through the 6th Framework Programme (projects SHARE-I3, RII-CT-2006-062193, COMPARE, CIT5- CT-2005-028857, and SHARELIFE, CIT4-CT-2006-028812) and through the 7th Framework Programme (SHARE-PREP, N° 211909, SHARE-LEAP, N° 227822 and SHARE M4, N° 261982). Additional funding from the U.S. National Institute on Aging (U01 AG09740-13S2, P01 AG005842, P01 AG08291, P30 AG12815, R21 AG025169, Y1-AG-4553-01, IAG BSR06-11 and OGHA 04-064) and the German Ministry of Education and Research as well as from various national sources is gratefully acknowledged (see http://www.share-project.org for a full list of funding institutions).
R37AG17560 (which funds PEP) and Dr. Gill is supported, in part, by K07AG043587 and P30AG021342.
KR and AM are applying for funding to commercialize a version of the Frailty Index based on a Comprehensive Geriatric Assessment. A company called Videx Canada has been incorporated for this. The version of the frailty index presented here is not the one that Videx aims to commercialize.
FP, OT, LW, TB conceived the article, conducted the analyses, and wrote the final draft. AM arranged for the SHARE data to be made available to our group and assisted with data analysis. KR contributed to the conception and initial design of the study, arranged funding, and revised previous drafts. TG and EG arranged for the Yale-PEP data to be made available to our group and revised previous drafts. SK arranged for the NSHS data to be made available to our group and revised previous drafts. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Peña, F.G., Theou, O., Wallace, L. et al. Comparison of alternate scoring of variables on the performance of the frailty index. BMC Geriatr 14, 25 (2014) doi:10.1186/1471-2318-14-25
- Frailty index
- Deficit accumulation