AMPI-AB validity and reliability: a multidimensional tool in resource-limited primary care settings

Background The early identification of individuals at high risk for adverse outcomes by a Comprehensive Geriatric Assessment (CGA) in resource-limited primary care settings enables tailored treatments, however, the evidence concerning its benefits are still controversial. The main objective of this study was to examine the validity and reliability of the “Multidimensional Assessment of Older People in Primary Care (AMPI-AB)”, a CGA for primary care in resource-limited settings. Methods Longitudinal study, with median follow-up time of 16 months. Older adults from a public primary care unit in São Paulo, Brazil, were consecutively admitted. Reliability was tested in a sample from a public geriatric outpatient clinic. Participants were classified by the AMPI-AB score as requiring a low, intermediate or high complexity of care. The Physical Frailty Phenotype was used to explore the AMPI-AB’s concurrent validity. Predictive validity was assessed with mortality, worsening of the functional status, hospitalizations, emergency room (ER) visits and falls. The area under the ROC curve and logistic regression were calculated for binary outcomes, and a Cox proportional hazards model was used for survival analysis. Results Older adults (n = 317) with a median age of 80 (74–86) years, 67% female, were consecutively admitted. At the follow-up, 7.1% of participants had died, and increased dependency on basic and instrumental activities of daily living was detected in 8.9 and 41.1% of the participants, respectively. The AMPI-AB score was accurate in detecting frailty (area under the ROC curve = 0.851), predicted mortality (HR = 1.25, 95%CI = 1.13–1.39) and increased dependency on basic (OR = 1.26, 95%CI = 1.10–1.46) and instrumental (OR = 1.22, 95%CI = 1.12–1.34) activities of daily living, hospitalizations (OR = 2.05, 95%CI = 1.04–1.26), ER visits (OR = 1.20, 95%CI = 1.10–1.31) and falls (OR = 1.10, 95%CI = 1.01–1.20), all models adjusted for sex and years of schooling. Reliability was tested in a sample of 52 older adults with a median age of 72 (85–64) years, 63.5% female. The AMPI-AB also had good interrater (ICC = 0.87, 95%CI = 0.78–0.92), test-retest (ICC = 0.86, 95%CI = 0.76–0.93) and proxy reliability (ICC = 0.84, 95%CI = 0.67–0.93). The Cronbach’s alpha was 0.69, and the mean AMPI-AB administration time was 05:44 ± 02:42 min. Conclusion The AMPI-AB is a valid and reliable tool for managing older adults in resource-limited primary care settings.

Conclusion: The AMPI-AB is a valid and reliable tool for managing older adults in resource-limited primary care settings.
Keywords: Comprehensive geriatric assessment, Screening tool, Primary care, Validation Background A comprehensive geriatric assessment (CGA) is a multidimensional diagnostic process focused on determining the social, physical, cognitive, psychological, and functional capabilities of older adults with the purpose of developing a coordinated and integrated management plan [1]. The early identification of individuals at high risk for adverse outcomes by a CGA enables tailored treatments and better allocation of resources [1,2]. However, the evidence concerning the benefits of applying the broad principles of CGAs in primary care are controversial [3]. In a recent systematic review, CGAs based on primary care practice were acceptable to the individuals involved but had an inconsistent impact on mortality, functional status and hospital admission rates [4]. Moreover, there is a need for new models that address the complexity of care of older adults living with multimorbidity or frailty in primary care, as these models can lead to quality of care improvements without increases in costs [3][4][5][6][7].
In Brazil, primary care aims to provide universal access and comprehensive health care, to coordinate and to expand the coverage to more complex levels of care (e.g., specialized ambulatory care and in-hospital care), and to implement intersectoral actions for health promotion and disease prevention. Brazilian primary care clinics are focused on families and communities, integrating medical care with health promotion and public health actions [8]. To standardize the evaluation of older people according to the Brazilian primary care norms, specialists in Geriatrics, Gerontology and Public Health from the Secretariat of Municipal Health of São Paulo developed a CGA tool in 2015, named the Multidimensional Assessment of Older People (AMPI-AB). The main purpose of the AMPI-AB was to standardize the evaluation of older people at primary care units in the city of São Paulo. The detection of the main geriatric syndromes and individual risks among primary care older adults can guide healthcare plans targeting the maintenance of functional status, while considering the multidimensional complexity of the aging process.
The AMPI-AB is composed of 17 questions that can be applied by any trained healthcare professional. The questions are based on well-known and validated scores used to detect relevant geriatric problems, such as lack of social support, multimorbidity, polypharmacy, cognitive and sensory impairment, physical limitations, depression, falls, functional dependence, weight loss and poor oral health. The final score classifies older adults in low, intermediate or high complexity of care and guides individualized healthcare plans and referrals to secondary care. Despite the wide application of the AMPI-AB in São Paulo, no studies have validated the AMPI-AB. The objectives of this study were to examine the validity and reliability of the AMPI-AB in resource-limited primary care settings.

Study design and population
This longitudinal study included participants aged 60 years and older who were consecutively admitted to the public primary care unit Geraldo de Paula Souza of University of São Paulo Public Health School, in São Paulo, Brazil between August 2016 and February 2017. This unit is responsible for the primary care of approximately 4000 older adults from the west region of São Paulo. We excluded individuals with incomplete data. All subjects provided their written informed consent to their voluntary participation in this study and for the publication of their data. The study and informed consent form have been approved by the local ethics committee of São Paulo University Faculty of Medicine.

Baseline measurements
At baseline, demographic characteristics were self-reported, and body mass index was calculated after anthropometry assessments at the primary care consultation. The AMPI-AB was applied by a multidisciplinary team composed of physicians, a nurse and a nutritional therapist. The instrument evaluates the following domains: age, self-rated health, social support, chronic conditions, polypharmacy, recent hospital admissions, falls, vision, hearing, physical limitations, cognition, humor, basic and instrumental activities of daily living, incontinence, unintentional weight loss and oral health. The total score ranges from 0 to 21, and initially proposed cutoffs, defined by specialists in Geriatrics, Gerontology and Public Health of the Secretariat of Municipal Health of São Paulo based on subject matter knowledge, classify the complexity of care as follows: low (0-5 points), intermediate (6-10 points) or high (≥ 11 points). The questions and the scoring method are detailed in Supplementary Figure 1.
Frailty defined according to the Physical Frailty Phenotype [9] was used to explore the concurrent validity of the AMPI-AB and was measured at baseline. We also assessed basic and instrumental activities of daily living (BADL and IADL) using the Katz and Lawton scales, respectively [10,11]. Clinical conditions and drugs were obtained through chart reviews, allowing to calculate the Charlson Comorbidity Index to determine multimorbidity and to assess polypharmacy, respectively [12].

Predictive validity: assessment of outcomes
Outcomes were assessed after 1 year by a structured telephone interview performed by two examiners blinded to the baseline assessments. If the participant had a diagnosis of dementia at baseline, the interview was conducted with a caregiver. Interviews assessed the occurrence of death, hospitalizations, emergency room (ER) visits and falls during the follow-up period. BADLs and IADLs were also evaluated using the Katz and Lawton scores, respectively. We considered mortality as the primary outcome and the other parameters as secondary outcomes.

Time to complete the instrument and interrater, testretest and proxy reliability
We tested the average time to complete the instrument and interrater, test-retest and proxy reliability in another consecutive sample of older adults (n = 52) cared in a public tertiary geriatric outpatient unit at the Clinics Hospital from University of São Paulo Medical School between December 2018 and April 2019. Two blinded examiners administered the AMPI-AB. To assess interrater reliability, the instruments were administered on the same day 4 h apart. To assess test-retest reliability, the instrument was administered twice by the same examiner within a 7-day interval. To assess proxy reliability, we applied the AMPI-AB by telephone to a proxy (who had contact with the participant at least once a week, including spouses, caregivers and family members) within 30 days after the participant's in-person assessment. Time taken to complete the instrument was recorded in the first assessment for each participant and proxy.

Analysis
In a post hoc study power calculation, we found that the study had 96% power to demonstrate a survival rate difference of 15% between older adults with low/intermediate complexity of care and older adults with high complexity of care, considering an allocation ratio of 5:1 between groups, 23 events, and an alpha-level of 0.05.
Characteristics of the participants according to the AMPI-AB classification in low/intermediate complexity of care versus high complexity of care were compared with the chi-square test and the Wilcoxon rank sum test for categorical and nonnormally distributed continuous variables. The accuracy of the AMPI-AB to discriminate frailty was investigated with a receiver operating characteristic (ROC) curve. The ROC curve was also used to discriminate the following dichotomous outcomes: acquired dependency in BADLs and IADLs (worsening disability), hospitalization, ER visits, and falls. Since the AMPI-AB's original cut-off point determined by the specialists was arbitrary, the Youden index was calculated to determine the threshold that yields optimal discriminative performance. Additionally, we investigated the association of the AMPI-AB score with worsening disability, hospitalizations, ER visits and falls with logistic regression models adjusted for sex and years of schooling. We plotted Kaplan-Meier curves to illustrate the survival rate estimates for the AMPI-AB classification (low/intermediate complexity of care versus high complexity of care) and the log-rank test was used to compare the survival rate distributions between groups. We also investigated the association of the AMPI-AB total score and classification with mortality with Cox proportional hazards models, which were also adjusted for sex and years of schooling. The Schoenfeld residual test indicated that the proportional hazards assumption was met. To compare the predictive ability of the AMPI-AB with other well-established geriatric prognostic instruments, we also evaluated the association of the Charlson Comorbidity Index, Katz and Lawton scores with oneyear mortality using Cox-proportional hazards models adjusted for age, sex and schooling years.
To assess interrater, test-retest and proxy reliability we calculated intraclass correlation coefficient (ICC) estimates and their 95% confident intervals (CI) for the AMPI-AB score based on a single-rating (two raters), absolute-agreement, 2-way random effects model [13]. The AMPI-AB categorical classification agreement was evaluated by the linearly weighted Kappa coefficient, and internal consistency was evaluated by Cronbach's alpha [14]. All statistical tests were two-tailed, and an alpha level of 0.05 was used to determine significance.

Results
We included 317 participants in the validity assessment, 67% female and 66% white, with a median follow-up time of 16  months, median age of 80 (74-86) years, and median schooling duration of 8 (4-13) years. Sample baseline characteristics can be seen in Table 1. The median AMPI-AB score of the sample was 7 (9-4), and 16.4% of participants were classified as high complexity of care.
The association of the AMPI-AB total score with hospitalizations, worsening disability, ER visits and falls are shown in Table 2. Higher AMPI-AB scores were associated with all outcomes after adjustment.

Discussion
In this study, the AMPI-AB was accurate in detecting physical frailty, and it predicted mortality, worsening disability, hospitalizations, ER visits and falls among older adults in primary care. Moreover, we demonstrated that the AMPI-AB had good reliability and agreement in interrater, test-retest and proxy reliability analyses.
To the extent of our knowledge, this is the first study to evaluate the AMPI-AB, an instrument for managing the healthcare of older people in a public health system in a lower-middle income country, developed by a consensus of specialists in public health, geriatrics and gerontology. The AMPI-AB aims (1) to identify older adults with high complexity of care needs who may benefit from specialized care and (2) to guide primary care professionals in the assessment of older adults and the formulation of individualized healthcare plans. Moreover, the AMPI-AB is a practical and simple instrument that does not require specialized equipment to assess the complexity of care, and it is a valuable option for managing older adults in resource-limited primary care settings in Latin America and worldwide.
The discriminative ability of several screening instruments to identify older persons at increased risk of functional decline and mortality ranges from 0.60 to 0.80 in the literature, which are very similar to those demonstrated by our study [4,[15][16][17][18][19][20]. However, none of these instruments have been evaluated in the context of resource-limited primary care settings. The Identification of Seniors At Risk -Primary Care (ISAR-PC) instrument is also a validated and easy-to-apply screening instrument to identify individuals at increased risk of functional decline in primary care. Through the assessment of age, dependence in IADL, and impaired memory, a study with 790 participants with median age of 75 years, showed that the ISAR-PC had an area under the ROC curve of 0.63-0.64 to discriminate functional decline in BADLs in 1 year. However, this study excluded patients with dementia, and 19% of participants were lost at follow-up [15].
According to international standards, Brazilian older adults have high access to primary care, however, many hospital admissions could be prevented by more effective prevention and treatment actions at this level of care [21]. Differences in the performance of primary care units are found and there is a continued need to update and adapt the model of healthcare delivery [21]. Lowincome and lower-middle-income countries could benefit from the large-scale implementation of simple and short CGA instruments such as the AMPI-AB, that can be applied by any trained healthcare professionals, allowing the standardization of the assessment of older adults in primary care. Moreover, our study demonstrated that the AMPI-AB can also be applied by telephone with a proxy, facilitating the management of older adults with impaired access to health centers.
Nevertheless, our study results should be interpreted with caution due to some limitations. We used a convenience sample from a single primary care unit, and the sociodemographic profile of our population is not representative of the entire population of the city of São Paulo, hindering the generalizability of our results. Furthermore, outcomes such as functional status, falls and recent hospitalization are also components of the AMPI-AB, and this may have influenced the final result. However, the AMPI-AB assess functional status with less details than the instruments used to assess the outcomes. Moreover, we considered worsening disability as the outcome (the negative variation of functional status during follow-up) instead of the functional score per se. In addition, functional status at baseline alone was not a predictor of mortality in our study. Finally, the AMPI-AB is a multidimensional and complex instrument, and the main purpose of this paper was to validate the whole construct, and not individual items.
Future longitudinal studies with older adults from different Brazilian cities and other countries are required to confirm our data and extend the applicability of the AMPI-AB. Additionally, future studies should evaluate the ability of the AMPI-AB to manage the healthcare of older adults and its impact on a public health system, especially with regard to healthcare costs.

Conclusions
In conclusion, the study presents evidence supporting the validity and reliability of the AMPI-AB for older adults in resource-limited primary care settings.
Additional file 1: Figure S1. Multidimensional Assessment of Older People in Primary Care (AMPI-AB).