- Research Article
- Open Access
Age effect on the prediction of risk of prolonged length hospital stay in older patients visiting the emergency department: results from a large prospective geriatric cohort study
BMC Geriatrics volume 18, Article number: 127 (2018)
With the rapid growth of elderly patients visiting the Emergency Department (ED), it is expected that there will be even more hospitalisations following ED visits in the future. The aim of this study was to examine the age effect on the performance criteria of the 10-item brief geriatric assessment (BGA) for the prolonged length of hospital stay (LHS) using artificial neural networks (ANNs) analysis.
Based on an observational prospective cohort study, 1117 older patients (i.e., aged ≥ 65 years) ED users were admitted to acute care wards in a University Hospital (France) were recruited. The 10-items of BGA were recorded during the ED visit and prior to discharge to acute care wards. The top third of LHS (i.e., ≥ 13 days) defined the prolonged LHS. Analysis was successively performed on participants categorized in 4 age groups: aged ≥ 70, ≥ 75, ≥ 80 and ≥ 85 years. Performance criteria of 10-item BGA for the prolonged LHS were sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], likelihood ratios [LR], area under receiver operating characteristic curve [AUROC]). The ANNs analysis method was conducted using the modified multilayer perceptron (MLP).
Values of criteria performance were high (sensitivity> 89%, specificity≥ 96%, PPV > 87%, NPV > 96%, LR+ > 22; LR- ≤ 0.1 and AUROC> 93), regardless of the age group.
Age effect on the performance criteria of the 10-item BGA for the prediction of prolonged LHS using MLP was minimal with a good balance between criteria, suggesting that this tool may be used as a screening as well as a predictive tool for prolonged LHS.
A growing number of older adults (i.e., age 65 and over) visit the emergency departments (EDs) . In Europe, they account for around 20% of all EDs visitors [1, 2]. These older ED visitors, particularly the oldest group (i.e., age 85 and over), generally have a longer length of hospital stay (LHS) after their ED discharge to acute care wards compared to younger ED visitors [1,2,3]. The high morbidity burden and related-disabilities expose older patients to an increased risk of non-fatal health outcomes like a long LHS [2, 4, 5]. With the rapid growth of the oldest segment of ED visitors, hospitalization after an ED admission is expected to be even greater in the future and, thus, hospitals need to confront this new challenging issue [1,2,3, 6].
One way to reduce LHS is early identification of older ED visitors at greater risk of prolonged LHS after an ED discharge to acute care wards [1, 5, 6]. This screening is a crucial step for targeting appropriate interventions to prevent or decrease the occurrence of non-fatal health outcomes. The predictive tools designed for this purpose should provide a relevant stratification of risk and give information early; ideally before the hospital admission in order to avoid or plan the admission [4, 7]. The use of clinical information collected by a physician has been shown to be the best strategy to develop predictive tools of unplanned hospital admissions compared to self-reported and administrative data collection [8, 9] A limited number of studies have used tools aimed at identifying older patients at greater risk of prolonged LHS after an ED visit, with low predictive accuracy [2, 3, 5, 6]. Recently, the 10-item Brief Geriatric Assessment (BGA), was reported to have a high specificity (97%) but a lower sensitivity 63% . This study reported the best criteria performance to date. This result was explained in part by the use of artificial neural networks (ANNs), and in particular the modified multilayer perceptron (MLP) . Indeed, ANNs analysis is particularly adapted to predict an inherent complex event like prolonged LHS [11, 12]. The main limit of this previous study was the unbalance between sensitivity and specificity, which could be related to the high amount of data required by ANNs [11, 12]. In addition, because the risk of hospitalization increases with age, it could be suggested that the best balance with greater values of performance criteria could be reported specifically in the oldest age group (i.e., age 85 and over) of ED users [1,2,3,4]. The reported study aims to examine the effect of age on the predictive abilities (i.e., sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], likelihood ratios [LR], area under receiver operating characteristic curve [AUROC]) of the 10-item BGA for the prolonged LHS using MLP in geriatric ED visitors.
A total of 1117 older patients (i.e., aged ≥ 65 years) were recruited upon their hospitalization after an ED visit in a University Hospital (France) from January 2013 and December 2013. This study is an ongoing study which began in 2011 and its procedure for participant’s recruitment has been previously described in detail [6, 10]. To be included, patients had to be hospitalized on acute care wards after an ED visit, age 65 years and over, and willingness to participate in research. Patients who died during hospitalization were excluded.
The 10-item BGA was fulfilled upon admission to the ED and was composed of the following items: age ≥ 85 years, male gender, polypharmacy defined as ≥5 drugs per day, use psychoactive drugs (i.e., benzodiazepines, antidepressants or neuroleptics), history of falls in previous 6 months, temporal disorientation (i.e., inability to give the month and/or year), presence of acute organ failure plus reason for admission, living situation (home versus institution), and non-use of formal and/or informal home-help services. The nature of the acute organ failure for ED visit was categorized in five groups: cardio-vascular diseases, respiratory diseases, digestive diseases, neuropsychiatric diseases, and other acute diseases (Table 1). Other acute diseases referred to a heterogenous groups of diseases including traumatic injuries, hepatic failure, hematological failure and kidney failure.
The LHS was calculated using the administrative registry of the University Hospital and corresponded to number of days between the first day of ED visit and the last day of hospitalization on an acute care ward. Prolonged LHS was defined as being in the top third of LHS, which corresponded to more than 13 days in the studied sample. The main issue to identify this threshold value is that there is no consensus on the definition of a prolonged length of hospital stay is in geriatric acute care unit. The absence of definition is due to the fact that a prolonged length of hospital stay depends on an accumulation and complex interplay between several variables. These variables are related to the health status of patients but also to the environment where they are hospitalized (e.g., flux of patients, number of health professionals, type of hospital, organization of care etc.…). Thus, the unique solution to determine this threshold is to use the consensus methods of tertilization [6, 10].
Standard protocol approvals, registrations, and participant consents
Patients recruited in this study provided themselves a verbal consent or received help from their trusted person. The consent to participate was recorded in the patients’ digital files. Ethical Committee of Angers, France, approved the entire procedure.
Participants were split into two subgroups based on the presence or absence of a prolonged LHS. The top third of LHS defined the prolonged LHS (i.e., > 13 days). Univariate logistic regression models were used to examine the association between prolonged LHS (dependent variable) and 10-item BGA (independent variables). Artificial neural networks (ANNs) are inspired by animals’ brain and provide computational processing based on machine learning. ANNs are more appropriate to examine “chaotic” events, such as prolonged LHS, because they are not linear statistical models. These systems are interconnected and composed of multiple layers. Nodes from one layer are connected to all nodes in the following layer, but there were no lateral connections within the layer (Fig. 1). The output layer comprised one neuron, indicating the presence or absence of prolonged LHS.The “neuralnet: Training of neural networks” R package was used for Modified multilayer perceptron (MLP) combining with a specific algorithm (9, 10). To perform ANNs analysis, the sample of participants was randomized in two subgroups (i.e., a training group and a testing group). There was no significant difference between training and testing group (data not shown). Between-group comparisons were performed using unpaired t-test, Pearson’s Chi-squared test with Yates’ continuity correction, as appropriate. Four age groups were identified: ≥ 70, ≥ 75, ≥ 80 and ≥ 85 years old. Performance criteria were sensitivity, specificity, PPV, NPV, LR+, LR- and AUROC. All statistics were performed using R 3.1.0 and Net Beans IDE 8.0.
There was a trend for a greater mean age (P = 0.0699), a greater prevalence of temporal disorientation (OR = 2.65, P < 0.001) in participants with prolonged LHS compared to those with short LHS (Table 2). In addition, participants with prolonged LHS visited the ED less often for digestive diseases (OR = 0.48, P = 0.0189) and more often for other diseases (OR = 1.46, P = 0.089) compared to those with short LHS. The mean LHS was 21.6 ± 8.8 days for older ED users with prolonged LHS and 5.2 ± 3.7 days for those who had no prolonged LHS. Whatever the age group considered, predictive performance were high(sensitivity> 89%, specificity≥ 96%, PPV > 87%, NPV > 96%, LR+ > 22; LR- ≤ 0.1 and AUROC> 93). Participants over 75 years showed the best performance (sensitivity = 89.7%, specificity = 97.8%, PPV = 93.4, NPV = 96.5, LR + =41.0; LR- = 0.1 and AUROC = 93.7) (Table 3).
The findings show that effect of age was minimal on predictive abilities of the 10-item BGA. These results suggest that analysis provided by ANNs may enable to use 10-item BGA as a screening tool but also as a predictive tool to identify older patients at higher risk of prolonged LHS, whatever their age.
The best criteria performances for prolonged LHS were shown with patients aged 75 years and over. This is an unexpected finding because it was hypothesized that greater values of criteria performance could be reported in the oldest segment of ED users. This result is discordant with previous studies which reported a strong association between age and the risk of prolonged LHS [1,2,3,4]. Age has previously been identified as an important predictor for prolonged LHS [10, 13, 14]. For instance, in a similar sized cohort of patients admitted to ED (993 patients, mean age = 87.04 years) age and gender explained 21.6% of area under receiver operating characteristic curve value . In the same way, Campbell et al. reported in a larger cohort of patients admitted to ED (1626 patients, mean age = 78.7 years) that age over 85 years was strongly associated with prolonged LHS (OR = 7.6, P < 0.001) . The association between increased age and prolonged LHS has been explained by incident disabilities that exceed 50% in hospitalized patients aged 85 years and over [14,15,16]. This finding is consistent with Sourial et al. who showed that age and gender had the highest contribution (C statistic values from 0.51 to 0.67) in predictive accuracy of incident disability in a cohort composed of 6657 patients (mean age = 73.68 years) .
A possible explanation of the discordance about age effect on the prediction of prolonged LHS shown in our study compared to previous studies could be related to the profile of population recruited, which is oldest old patients with a mean age around 85 years. Moreover, ANNs provide a different statistical approach that consider the complex interplay between all items [11, 12, 18]. Indeed, previous results of ANNs reported that using numerous variables increased predictive accuracy (area under cover values lie between 84.1 with 9 items and 90.5 with 10 items) but also modified the contribution of demographic items in the predictive performance (from 12.8% with 9 items to 21.3% with 10 items) . Categorizing age groups provide an additional variable that limits the analysis of ANNs to a single age group and may modify the distribution and weight of 10 items in contribution of the predictive accuracy. Thus, ANNs take into account the variations in the contribution of all types of variables (demographic items, acute or chronic diseases, and environmental items) to increase predictive performance and to learn to recognize patterns of prolonged LHS in each age group.
Our findings underscored that regardless of age, values of criteria performance were high (sensitivity> 88%, specificity≥ 96%, PPV > 87%, NPV > 96%, LR+ > 22; LR- ≤ 0.1 and AUROC> 93). To the best of our knowledge, the current study demonstrates the best performance and balance between criteria reported for prediction of LHS after an ED visit. This result is discordant from a recent previous study which reported lower values and an unbalance between sensitivity and specificity . The main explanation as suggested in our hypothesis could be related to a difference in the number of participants. In our study, we included 1117 individuals which likely increased the accuracy of prediction. It has been shown that ANNs may provide accurate information on an event only if there is a sufficient quantity of data points to be analysed [11, 12].
In order for a screening test to be applicable, it requires high level sensitivity to limit false-negative results. With sensitivity above 89% in all age group conditions, the 10-items BGA is a useful screening tool that can be applied to identify early older ED users at higher risk of prolonged LHS after their discharge to acute care wards. In addition, our results demonstrated a high specificity above 95%, which implies that there is a low false positive rate when applying the 10-items BGA to predict length of hospitalization. Thus, the combination of both high sensitivity and specificity indicates that the 10-items BGA is not only a simple screening tool but also a diagnostic tool with excellent predictive accuracy. The ability to analyse our data with the use of ANNs methods analysis is the main explanation for these findings. Indeed, ANNs are data analysis tools developed to overcome the limitations of traditional linear models as a method to predict health events . ANNs are computational models which are capable of machine learning and pattern recognition [7,8,9,10,11,12]. Because they apply non-linear statistics to pattern recognition, ANNs are particularly adapted to “chaotic” events like prolonged LHS. Nowadays, the advances generated by ANNs combined with improvement of computer technology affords the opportunity to explore new perspectives using ANNs as decision-making diagnostic aids for physicians. Thus, these results can be applied directly to clinical practice because they can be developed as software applications for computers and hand held devices. The 10-item BGA may provide answers to facilitate clinical decision-making process because this tool provides a risk stratification of patients at risk of non-fatal health outcomes. Such information may be relevant to make the right decision for the patients like the discharge to home or to a medical ward, and to continue the appropriate interventions in the right patients and at the right time by the right professionals (i.e., geriatric intervention versus no geriatric intervention).
Our results also showed that patients with digestive diseases had a shorter LHS compared to patients admitted for other diseases. One explanation could be that admissions for digestive diseases are more often semi-urgent or non-urgent . This low degree of urgency may explain a lower LHS. In contrast, “Other diseases” as a reason for admission was associated with an increases LHS. This group refers in part to traumatic injuries related to falls. Unlike young patients, the most common mechanism for traumatic injuries in older patients is due to fall . Falls have been identified as major cause of unintentional injury leading to prolonged LHS and death, especially over 80 years, that could explain our results .
The strengths of this study include the large number of participants, the prospective cohort design, the hard outcome represented by prolonged LHS, and the use of sophisticated new statistical models. However, limitations need to be considered including recruitment of participants from a single center and the fact that important items related to prolonged LHS could have been forgotten. Besides, we included inpatients who died during their hospitalization and those discharged in another hospital. Both date of death or transfer to another hospital were considered as the last day of hospitalization. Thus, a bias might exist because of a suspected higher complexity of those patients.
Age stratification provided minimal effect on the abilities of the 10-item BGA to predict prolonged LHS using ANNs. Indeed, ANNs provided homogeneous predictive performance that enable to use 10-item BGA as a screening tool but also as a predictive tool to identify older patients at higher risk of prolonged LHS, whatever their age.
Artificial neural networks
Area under receiver operating characteristic curve
Length of hospital stay
Modified multilayer perceptron
Negative predictive value
Positive predictive value
Samaras N, Chevalley T, Samaras D, Gold G. Older patients in the emergency department: a review. Ann Emerg Med sept. 2010;56(3):261–9.
Aminzadeh F, Dalziel WB. Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions. Ann Emerg Med mars. 2002;39(3):238–47.
Xu KT, Nelson BK, Berk S. The changing profile of patients who used emergency department services in the United States: 1996 to 2005. Ann Emerg Med déc. 2009;54(6):805–10. e1–7
Beauchet O, Launay CP, Fantino B, Lerolle N, Maunoury F, Annweiler C. Screening for elderly patients admitted to the emergency department requiring specialized geriatric care. J Emerg Med nov. 2013;45(5):739–45.
Hastings SN, Schmader KE, Sloane RJ, Weinberger M, Goldberg KC, Oddone EZ. Adverse health outcomes after discharge from the emergency department--incidence and risk factors in a veteran population. J Gen Intern Med. nov 2007;22(11):1527–31.
Launay CP, de Decker L, Kabeshova A, Annweiler C, Beauchet O. Screening for older emergency department inpatients at risk of prolonged hospital stay: the brief geriatric assessment tool. PLoS One. 2014;9:e110135.
García-Pérez L, Linertová R, Lorenzo-Riera A, Vázquez-Díaz JR, Duque-González B, Sarría-Santamera A. Risk factors for hospital readmissions in elderly patients: a systematic review. QJM Mon J Assoc Physicians août. 2011;104(8):639–51.
Wallace E, Stuart E, Vaughan N, Bennett K, Fahey T, Smith SM. Risk prediction models to predict emergency hospital admission in community-dwelling adults: a systematic review. Med Care août. 2014;52(8):751–65.
Baldonado A, Hawk O, Ormiston T, Nelson D. Transitional care management in the outpatient setting. BMJ Qual Improv Rep. 2017;6(1):u212974.
Launay CP, Rivière H, Kabeshova A, Beauchet O. Predicting prolonged length of hospital stay in older emergency department users: use of a novel analysis method, the artificial neural network. Eur J Intern Med sept. 2015;26(7):478–82.
Baxt WG, Skora J. Prospective validation of artificial neural network trained to identify acute myocardial infarction. Lancet Lond Engl. 6 janv. 1996;347(8993):12–5.
Baxt WG Application of artificial neural networks to clinical medicine. Lancet. 1995;346:1135–8.
McCusker J, Bellavance F, Cardin S, Trépanier S. Screening for geriatric problems in the emergency department: reliability and validity. Identification of Seniors at Risk (ISAR) Steering Committee. Acad Emerg Med. 1998;5(9):883–93.
Campbell SE, Seymour DG, Primrose WR, Lynch JE, Dunstan E, Espallargues M, et al. A multi-Centre European study of factors affecting the discharge destination of older people admitted to hospital: analysis of in-hospital data from the ACMEplus project. Age Ageing sept. 2005;34(5):467–75.
De Buyser SL, Petrovic M, Taes YE, Vetrano DL, Corsonello A, Volpato S, et al. Functional changes during hospital stay in older patients admitted to an acute care ward: a multicenter observational study. PLoS One. 2014;9(5):e96398.
Covinsky KE, Palmer RM, Fortinsky RH, Counsell SR, Stewart AL, Kresevic D, et al. Loss of independence in activities of daily living in older adults hospitalized with medical illnesses: increased vulnerability with age. J Am Geriatr Soc avr. 2003;51(4):451–8.
Sourial N, Bergman H, Karunananthan S, Wolfson C, Payette H, Gutierrez-Robledo LM, et al. Implementing frailty into clinical practice: a cautionary tale. J Gerontol A Biol Sci Med Sci déc. 2013;68(12):1505–11.
Akl A., Ghoneim M. Forcasting the clinical outcome: artificial neural networks or multivariate statistical models?, Artificial neural networks - methodological advances and biomedical applications, Prof. Kenji Suzuki (Ed.), 2011;InTech, ISBN: 978–953-.
Martinez-Fuerte R, Sierra-Martinez L, Sanz-Gonzalez N. Emergencies in primary care for digestive disorders. European Geriatric Medicine. 2015;6S1:S32–S156.
Aschkenasy MT, Rothenhaus TC. Trauma and falls in the elderly. Emerg Med Clin North Am. 2006;24(2):413–32.
Majdan M, Mauritz W. Unintentional fall-related mortality in the elderly: comparing patterns in two countries with different demographic structure. BMJ Open. 2015;5(8):e008672.
We are grateful to the participants for their cooperation.
The study was supported by Biomathics, Paris, France. The sponsor had no role in the design and conduct of the study, in the collection, management, analysis, and interpretation of the data, or in the preparation, review, or approval of the manuscript.
Availability of data and materials
Data and material concerning this article may be provided by the authors after justified request.
Ethics approval and consent to participate
The study was conducted in accordance with the ethical standards set forth in the Helsinki Declaration (1983). All participants recruited in this study provided a verbal informed consent as the study did not change the usual clinical practice. The verbal informed consent was obtained from the patients themselves in the presence of a reliable caregiver who was usually a family member assisting in decision-making for medical issues. The consent to participate was recorded in patients’ digital files. The study and the consent procedure were approved by the Ethical Committee of Angers, France, N°1,408,714.
The authors declare that they have no competing interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Launay, C.P., Kabeshova, A., Lanoé, A. et al. Age effect on the prediction of risk of prolonged length hospital stay in older patients visiting the emergency department: results from a large prospective geriatric cohort study. BMC Geriatr 18, 127 (2018). https://doi.org/10.1186/s12877-018-0820-5
- Length of hospital stay