- Research article
- Open Access
AMPI-AB validity and reliability: a multidimensional tool in resource-limited primary care settings
BMC Geriatrics volume 20, Article number: 124 (2020)
The early identification of individuals at high risk for adverse outcomes by a Comprehensive Geriatric Assessment (CGA) in resource-limited primary care settings enables tailored treatments, however, the evidence concerning its benefits are still controversial. The main objective of this study was to examine the validity and reliability of the “Multidimensional Assessment of Older People in Primary Care (AMPI-AB)”, a CGA for primary care in resource-limited settings.
Longitudinal study, with median follow-up time of 16 months. Older adults from a public primary care unit in São Paulo, Brazil, were consecutively admitted. Reliability was tested in a sample from a public geriatric outpatient clinic. Participants were classified by the AMPI-AB score as requiring a low, intermediate or high complexity of care. The Physical Frailty Phenotype was used to explore the AMPI-AB’s concurrent validity. Predictive validity was assessed with mortality, worsening of the functional status, hospitalizations, emergency room (ER) visits and falls. The area under the ROC curve and logistic regression were calculated for binary outcomes, and a Cox proportional hazards model was used for survival analysis.
Older adults (n = 317) with a median age of 80 (74–86) years, 67% female, were consecutively admitted. At the follow-up, 7.1% of participants had died, and increased dependency on basic and instrumental activities of daily living was detected in 8.9 and 41.1% of the participants, respectively. The AMPI-AB score was accurate in detecting frailty (area under the ROC curve = 0.851), predicted mortality (HR = 1.25, 95%CI = 1.13–1.39) and increased dependency on basic (OR = 1.26, 95%CI = 1.10–1.46) and instrumental (OR = 1.22, 95%CI = 1.12–1.34) activities of daily living, hospitalizations (OR = 2.05, 95%CI = 1.04–1.26), ER visits (OR = 1.20, 95%CI = 1.10–1.31) and falls (OR = 1.10, 95%CI = 1.01–1.20), all models adjusted for sex and years of schooling. Reliability was tested in a sample of 52 older adults with a median age of 72 (85–64) years, 63.5% female. The AMPI-AB also had good interrater (ICC = 0.87, 95%CI = 0.78–0.92), test-retest (ICC = 0.86, 95%CI = 0.76–0.93) and proxy reliability (ICC = 0.84, 95%CI = 0.67–0.93). The Cronbach’s alpha was 0.69, and the mean AMPI-AB administration time was 05:44 ± 02:42 min.
The AMPI-AB is a valid and reliable tool for managing older adults in resource-limited primary care settings.
A comprehensive geriatric assessment (CGA) is a multidimensional diagnostic process focused on determining the social, physical, cognitive, psychological, and functional capabilities of older adults with the purpose of developing a coordinated and integrated management plan . The early identification of individuals at high risk for adverse outcomes by a CGA enables tailored treatments and better allocation of resources [1, 2]. However, the evidence concerning the benefits of applying the broad principles of CGAs in primary care are controversial . In a recent systematic review, CGAs based on primary care practice were acceptable to the individuals involved but had an inconsistent impact on mortality, functional status and hospital admission rates . Moreover, there is a need for new models that address the complexity of care of older adults living with multimorbidity or frailty in primary care, as these models can lead to quality of care improvements without increases in costs [3,4,5,6,7].
In Brazil, primary care aims to provide universal access and comprehensive health care, to coordinate and to expand the coverage to more complex levels of care (e.g., specialized ambulatory care and in-hospital care), and to implement intersectoral actions for health promotion and disease prevention. Brazilian primary care clinics are focused on families and communities, integrating medical care with health promotion and public health actions . To standardize the evaluation of older people according to the Brazilian primary care norms, specialists in Geriatrics, Gerontology and Public Health from the Secretariat of Municipal Health of São Paulo developed a CGA tool in 2015, named the Multidimensional Assessment of Older People (AMPI-AB). The main purpose of the AMPI-AB was to standardize the evaluation of older people at primary care units in the city of São Paulo. The detection of the main geriatric syndromes and individual risks among primary care older adults can guide healthcare plans targeting the maintenance of functional status, while considering the multidimensional complexity of the aging process.
The AMPI-AB is composed of 17 questions that can be applied by any trained healthcare professional. The questions are based on well-known and validated scores used to detect relevant geriatric problems, such as lack of social support, multimorbidity, polypharmacy, cognitive and sensory impairment, physical limitations, depression, falls, functional dependence, weight loss and poor oral health. The final score classifies older adults in low, intermediate or high complexity of care and guides individualized healthcare plans and referrals to secondary care. Despite the wide application of the AMPI-AB in São Paulo, no studies have validated the AMPI-AB. The objectives of this study were to examine the validity and reliability of the AMPI-AB in resource-limited primary care settings.
Study design and population
This longitudinal study included participants aged 60 years and older who were consecutively admitted to the public primary care unit Geraldo de Paula Souza of University of São Paulo Public Health School, in São Paulo, Brazil between August 2016 and February 2017. This unit is responsible for the primary care of approximately 4000 older adults from the west region of São Paulo. We excluded individuals with incomplete data. All subjects provided their written informed consent to their voluntary participation in this study and for the publication of their data. The study and informed consent form have been approved by the local ethics committee of São Paulo University Faculty of Medicine.
At baseline, demographic characteristics were self-reported, and body mass index was calculated after anthropometry assessments at the primary care consultation. The AMPI-AB was applied by a multidisciplinary team composed of physicians, a nurse and a nutritional therapist. The instrument evaluates the following domains: age, self-rated health, social support, chronic conditions, polypharmacy, recent hospital admissions, falls, vision, hearing, physical limitations, cognition, humor, basic and instrumental activities of daily living, incontinence, unintentional weight loss and oral health. The total score ranges from 0 to 21, and initially proposed cutoffs, defined by specialists in Geriatrics, Gerontology and Public Health of the Secretariat of Municipal Health of São Paulo based on subject matter knowledge, classify the complexity of care as follows: low (0–5 points), intermediate (6–10 points) or high (≥ 11 points). The questions and the scoring method are detailed in Supplementary Figure 1.
Frailty defined according to the Physical Frailty Phenotype  was used to explore the concurrent validity of the AMPI-AB and was measured at baseline. We also assessed basic and instrumental activities of daily living (BADL and IADL) using the Katz and Lawton scales, respectively [10, 11]. Clinical conditions and drugs were obtained through chart reviews, allowing to calculate the Charlson Comorbidity Index to determine multimorbidity and to assess polypharmacy, respectively .
Predictive validity: assessment of outcomes
Outcomes were assessed after 1 year by a structured telephone interview performed by two examiners blinded to the baseline assessments. If the participant had a diagnosis of dementia at baseline, the interview was conducted with a caregiver. Interviews assessed the occurrence of death, hospitalizations, emergency room (ER) visits and falls during the follow-up period. BADLs and IADLs were also evaluated using the Katz and Lawton scores, respectively. We considered mortality as the primary outcome and the other parameters as secondary outcomes.
Time to complete the instrument and interrater, test-retest and proxy reliability
We tested the average time to complete the instrument and interrater, test-retest and proxy reliability in another consecutive sample of older adults (n = 52) cared in a public tertiary geriatric outpatient unit at the Clinics Hospital from University of São Paulo Medical School between December 2018 and April 2019. Two blinded examiners administered the AMPI-AB. To assess interrater reliability, the instruments were administered on the same day 4 h apart. To assess test-retest reliability, the instrument was administered twice by the same examiner within a 7-day interval. To assess proxy reliability, we applied the AMPI-AB by telephone to a proxy (who had contact with the participant at least once a week, including spouses, caregivers and family members) within 30 days after the participant’s in-person assessment. Time taken to complete the instrument was recorded in the first assessment for each participant and proxy.
In a post hoc study power calculation, we found that the study had 96% power to demonstrate a survival rate difference of 15% between older adults with low/intermediate complexity of care and older adults with high complexity of care, considering an allocation ratio of 5:1 between groups, 23 events, and an alpha-level of 0.05.
Characteristics of the participants according to the AMPI-AB classification in low/intermediate complexity of care versus high complexity of care were compared with the chi-square test and the Wilcoxon rank sum test for categorical and nonnormally distributed continuous variables. The accuracy of the AMPI-AB to discriminate frailty was investigated with a receiver operating characteristic (ROC) curve. The ROC curve was also used to discriminate the following dichotomous outcomes: acquired dependency in BADLs and IADLs (worsening disability), hospitalization, ER visits, and falls. Since the AMPI-AB’s original cut-off point determined by the specialists was arbitrary, the Youden index was calculated to determine the threshold that yields optimal discriminative performance. Additionally, we investigated the association of the AMPI-AB score with worsening disability, hospitalizations, ER visits and falls with logistic regression models adjusted for sex and years of schooling. We plotted Kaplan-Meier curves to illustrate the survival rate estimates for the AMPI-AB classification (low/intermediate complexity of care versus high complexity of care) and the log-rank test was used to compare the survival rate distributions between groups. We also investigated the association of the AMPI-AB total score and classification with mortality with Cox proportional hazards models, which were also adjusted for sex and years of schooling. The Schoenfeld residual test indicated that the proportional hazards assumption was met. To compare the predictive ability of the AMPI-AB with other well-established geriatric prognostic instruments, we also evaluated the association of the Charlson Comorbidity Index, Katz and Lawton scores with one-year mortality using Cox-proportional hazards models adjusted for age, sex and schooling years.
To assess interrater, test-retest and proxy reliability we calculated intraclass correlation coefficient (ICC) estimates and their 95% confident intervals (CI) for the AMPI-AB score based on a single-rating (two raters), absolute-agreement, 2-way random effects model . The AMPI-AB categorical classification agreement was evaluated by the linearly weighted Kappa coefficient, and internal consistency was evaluated by Cronbach’s alpha . All statistical tests were two-tailed, and an alpha level of 0.05 was used to determine significance. Analyses were performed using Stata version 15.1 (Stata Corp., College Station, TX).
We included 317 participants in the validity assessment, 67% female and 66% white, with a median follow-up time of 16 (16–14) months, median age of 80 (74–86) years, and median schooling duration of 8 (4–13) years. Sample baseline characteristics can be seen in Table 1. The median AMPI-AB score of the sample was 7 (9–4), and 16.4% of participants were classified as high complexity of care.
The median AMPI-AB score among frail (13.5%) and non-frail participants was 10 (12–8) and 6 (8–4), respectively (p < 0.001). The area under the ROC curve for the discrimination of frailty according to the AMPI-AB score was 0.851 (95%CI: 0.789–0.912). The cutoff point of 11, which was previously proposed by the Secretariat of Municipal Health, had 48.6% sensitivity and 92.2% specificity. The cutoff point of 8, calculated by the Youden Index, had 85.7% sensitivity and 68.3% specificity (Fig. 1). The AMPI-AB score was associated with frailty after adjustment for sex and schooling years (OR = 1.96, 95%CI = 1.43–2.68, p < 0.001).
We could not contact 7 participants (2.2%) for the outcome evaluation. Baseline characteristics did not differ between the contacted and not contacted participants (results not shown). At follow-up, 7.1% of participants had died, and increased dependency in BADL and IADL was detected in 8.9 and 41.1% of the participants, respectively. Hospitalizations, ER visits and falls were reported in 18.5, 38.4 and 27.6% individuals, respectively. The survival rate in the high complexity of care participants was lower than in the low/intermediate complexity of care participants (log-rank test p-value < 0.001), as shown in Fig. 2. High complexity of care determined by the AMPI-AB (HR = 3.65, 95%CI = 1.56–8.54, p = 0.003) and the AMPI-AB total score (HR = 1.25, 95%CI = 1.13–1.39, p < 0.001) were both associated with mortality after adjustment.
The Charlson Comorbidity Index was also associated with mortality (HR = 1.28, 95%CI = 1.03–1.58, p = 0.027), however, BADL (HR = 1.01, 95%CI = 0.84–1.22, p = 0.869) and IADL (HR = 0.96, 95%CI = 0.88–1.04, p = 0.345) at baseline were not associated with mortality, after adjustment for age, sex and schooling years.
The association of the AMPI-AB total score with hospitalizations, worsening disability, ER visits and falls are shown in Table 2. Higher AMPI-AB scores were associated with all outcomes after adjustment.
Reliability and time to complete the instrument
To assess reliability, we included 52 participants, 63.5% female and 69% white, with a median age of 72 years (85–64) and a median schooling duration of 11 (4–15) years. The median AMPI-AB score was 5 (3–8). The mean time taken to complete the AMPI-AB was 05:44 ± 02:42 min. To assess proxy reliability, we contacted 24 proxies. The proxies took 06:16 ± 02:24 min on average to complete the AMPI-AB. Interrater reliability was assessed in 50 participants, with ICC = 0.87 (95%CI = 0.78–0.92). Test-retest reliability was assessed in 40 participants, with ICC = 0.86 (95%CI = 0.76–0.93). Finally, proxy reliability was assessed in 24 participants, with ICC = 0.84 (95%CI = 0.67–0.93). All of the ICCs showed good reliability . Linearly weighted Kappa coefficients (95%CI) of the AMPI-AB categories for interrater, test-retest, and proxy reliability were 0.69 (0.64–0.84), 0.66 (0.41–0.91) and 0.76 (0.60–0.83), respectively. The Cronbach’s alpha was 0.69. All the aforementioned coefficients demonstrated substantial agreement .
In this study, the AMPI-AB was accurate in detecting physical frailty, and it predicted mortality, worsening disability, hospitalizations, ER visits and falls among older adults in primary care. Moreover, we demonstrated that the AMPI-AB had good reliability and agreement in interrater, test-retest and proxy reliability analyses.
To the extent of our knowledge, this is the first study to evaluate the AMPI-AB, an instrument for managing the healthcare of older people in a public health system in a lower-middle income country, developed by a consensus of specialists in public health, geriatrics and gerontology. The AMPI-AB aims (1) to identify older adults with high complexity of care needs who may benefit from specialized care and (2) to guide primary care professionals in the assessment of older adults and the formulation of individualized healthcare plans. Moreover, the AMPI-AB is a practical and simple instrument that does not require specialized equipment to assess the complexity of care, and it is a valuable option for managing older adults in resource-limited primary care settings in Latin America and worldwide.
The discriminative ability of several screening instruments to identify older persons at increased risk of functional decline and mortality ranges from 0.60 to 0.80 in the literature, which are very similar to those demonstrated by our study [4, 15,16,17,18,19,20]. However, none of these instruments have been evaluated in the context of resource-limited primary care settings. The Identification of Seniors At Risk – Primary Care (ISAR-PC) instrument is also a validated and easy-to-apply screening instrument to identify individuals at increased risk of functional decline in primary care. Through the assessment of age, dependence in IADL, and impaired memory, a study with 790 participants with median age of 75 years, showed that the ISAR-PC had an area under the ROC curve of 0.63–0.64 to discriminate functional decline in BADLs in 1 year. However, this study excluded patients with dementia, and 19% of participants were lost at follow-up .
According to international standards, Brazilian older adults have high access to primary care, however, many hospital admissions could be prevented by more effective prevention and treatment actions at this level of care . Differences in the performance of primary care units are found and there is a continued need to update and adapt the model of healthcare delivery . Low-income and lower-middle-income countries could benefit from the large-scale implementation of simple and short CGA instruments such as the AMPI-AB, that can be applied by any trained healthcare professionals, allowing the standardization of the assessment of older adults in primary care. Moreover, our study demonstrated that the AMPI-AB can also be applied by telephone with a proxy, facilitating the management of older adults with impaired access to health centers.
Nevertheless, our study results should be interpreted with caution due to some limitations. We used a convenience sample from a single primary care unit, and the sociodemographic profile of our population is not representative of the entire population of the city of São Paulo, hindering the generalizability of our results. Furthermore, outcomes such as functional status, falls and recent hospitalization are also components of the AMPI-AB, and this may have influenced the final result. However, the AMPI-AB assess functional status with less details than the instruments used to assess the outcomes. Moreover, we considered worsening disability as the outcome (the negative variation of functional status during follow-up) instead of the functional score per se. In addition, functional status at baseline alone was not a predictor of mortality in our study. Finally, the AMPI-AB is a multidimensional and complex instrument, and the main purpose of this paper was to validate the whole construct, and not individual items.
Future longitudinal studies with older adults from different Brazilian cities and other countries are required to confirm our data and extend the applicability of the AMPI-AB. Additionally, future studies should evaluate the ability of the AMPI-AB to manage the healthcare of older adults and its impact on a public health system, especially with regard to healthcare costs.
In conclusion, the study presents evidence supporting the validity and reliability of the AMPI-AB for older adults in resource-limited primary care settings.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Multidimensional Assessment of Older People in Primary Care
Basic activities of daily living
Comprehensive geriatric assessment
Instrumental activities of daily living
Intraclass correlation coefficient
Identification of Seniors At Risk – Primary Care
Receiver operating characteristic
Rubenstein LZ, Stuck AE, Siu AL, Wieland D. Impacts of geriatric evaluation and management programs on defined outcomes: overview of the evidence. J Am Geriatr Soc. 1991;39(9 Pt 2):8S–16S discussion 17S–18S.
Solomon DH. Geriatric assessment: methods for clinical decision making. JAMA. 1988;259(16):2450–2.
Pilotto A, Cella A, Pilotto A, et al. Three Decades of Comprehensive Geriatric Assessment: Evidence Coming From Different Healthcare Settings and Specific Clinical Conditions. J Am Med Dir Assoc. 2017;18(2):192.e1–192.e11. https://doi.org/10.1016/j.jamda.2016.11.004.
Garrard JW, Cox NJ, Dodds RM, Roberts HC, Sayer AA. Comprehensive geriatric assessment in primary care: a systematic review. Aging Clin Exp Res. 2019. https://doi.org/10.1007/s40520-019-01183-w.
Counsell SR, Callahan CM, Tu W, Stump TE, Arling GW. Cost analysis of the geriatric resources for assessment and Care of Elders care management intervention. J Am Geriatr Soc. 2009;57(8):1420–6. https://doi.org/10.1111/j.1532-5415.2009.02383.x.
Chen P, Steinman MA. Perception of primary care physicians on the impact of comprehensive geriatric assessment: what is the next step? Isr J Health Policy Res. 2016;5:46. https://doi.org/10.1186/s13584-016-0106-3.
Tran H-PT, Leonard SD. Geriatric assessment for primary care providers. Prim Care. 2017;44(3):399–411. https://doi.org/10.1016/j.pop.2017.05.001.
Paim J, et al. The Brazilian health system: history, advances, and challenges. Lancet. 2011;377(9779):1778–97.
Fried LP, Tangen CM, Walston J, et al. Frailty in older adults: evidence for a phenotype. J Gerontol A Biol Sci Med Sci. 2001;56(3):M146–56.
Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. The index of ADL: a standardized meauser of biological and psychosocial funcion. JAMA. 1963;185:914–9.
Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9(3):179–86.
Charlson M, Szatrowski TP, Peterson J, Gold J. Validation of a combined comorbidity index. J Clin Epidemiol. 1994;47(11):1245–51.
Koo TK, Li MY. A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63. https://doi.org/10.1016/j.jcm.2016.02.012.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.
Suijker JJ, Buurman BM, van Rijn M, et al. A simple validated questionnaire predicted functional decline in community-dwelling older persons: prospective cohort studies. J Clin Epidemiol. 2014;67(10):1121–30. https://doi.org/10.1016/j.jclinepi.2014.05.014.
Daniels R, van Rossum E, Beurskens A, van den Heuvel W, de Witte L. The predictive validity of three self-report screening instruments for identifying frail older people in the community. BMC Public Health. 2012;12:69. https://doi.org/10.1186/1471-2458-12-69.
Saliba D, Elliott M, Rubenstein LZ, et al. The vulnerable elders survey: a tool for identifying vulnerable older people in the community. J Am Geriatr Soc. 2001;49(12):1691–9. https://doi.org/10.1046/j.1532-5415.2001.49281.x.
Gobbens RJJ, van Assen MALM, Luijkx KG, Schols JMGA. The predictive validity of the Tilburg frailty Indicator: disability, health care utilization, and quality of life in a population at risk. Gerontologist. 2012;52(5):619–31. https://doi.org/10.1093/geront/gnr135.
Ravindrarajah R, Lee DM, Pye SR, et al. The ability of three different models of frailty to predict all-cause mortality: results from the European male aging study (EMAS). Arch Gerontol Geriatr. 2013;57(3):360–8. https://doi.org/10.1016/j.archger.2013.06.010.
Romero-Ortuno R. The frailty instrument for primary care of the survey of health, ageing and retirement in Europe predicts mortality similarly to a frailty index based on comprehensive geriatric assessment. Geriatr Gerontol Int. 2013;13(2):497–504. https://doi.org/10.1111/j.1447-0594.2012.00948.x.
Macinko J, Bof de Andrade F, PRB S-J, Lima-Costa MF. Primary care and healthcare utilization among older Brazilians (ELSI-Brasil). Rev Saude Publica. 2018;52(Suppl 2):6s.
We would like to thank Doralice Severo da Cruz, Leonardo José Costa de Lima, Rosa Maria Bruno Marcucci and Sandra Cristina Coelho Teixeira, who have participated in the development and implementation of the AMPI-AB in the city of São Paulo. We would also like to thank Marcia Maria Porto Rosseto Mazza and Paulo Rogério Gallo for supporting our research in the Geraldo de Paula Souza Health Center School.
Ethics approval and consent to participate
All subjects provided their written informed consent to their voluntary participation in this study and for the publication of their data. The study and informed consent form have been approved by the local ethics committee of São Paulo University Faculty of Medicine (CAAE: 79087517.5.0000.0065).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Saraiva, M.D., Venys, A.L., Abdalla, F.L.P. et al. AMPI-AB validity and reliability: a multidimensional tool in resource-limited primary care settings. BMC Geriatr 20, 124 (2020). https://doi.org/10.1186/s12877-020-01508-9
- Comprehensive geriatric assessment
- Screening tool
- Primary care