Computer-based cognitive tests and cerebral pathology among Japanese older adults
BMC Geriatrics volume 23, Article number: 226 (2023)
This study aimed to identify the appropriate computer-based cognitive tests and cut-off values for estimating amyloid burden in preclinical Alzheimer’s disease drug trials.
Data from 103 older individuals, who underwent 18F-florbetapir positron emission tomography and cognitive testing, were analyzed. Cognitive tests evaluated word list memory (immediate recognition and delayed recall), attention (Trail Making Test-part A), executive function (Trail Making Test-Part B), and processing speed (Digit Symbol Substitution Test [DSST]).
The Aβ burden was significantly associated with word list memory (odds ratio [OR] = 0.42, 95% confidence interval [CI], 0.19–0.91) and DSST (OR = 0.35; 95% CI, 0.14–0.85). Positive predictive value and number needed to screen at a cut-off of 1.5 SD were better for word list memory and DSST among predictive values.
The computer-based memory and processing speed tests have the potential to reduce failure rates while screening individuals with Aβ accumulation in community settings.
Several failed clinical trials of beta-amyloid (Aβ)-targeting drugs suggest that intervention at very early, pre-symptomatic stages of the disease may be necessary to prevent disease progression . Therefore, accurate diagnosis and timely intervention, at an early preclinical/prodromal stage of Alzheimer’s disease (AD), have been the core aims of drug development; however, the feasibility relies on identifying high-risk individuals of AD . The clinical drug trials currently underway include the Anti-Amyloid Treatment in Asymptomatic Alzheimer's Disease (A4 study), including clinically normal older individuals with elevated amyloid levels on positron emission tomography (PET) scan, and the AHEAD 3–45 study, including clinically normal individuals with elevated or intermediate amyloid levels. The challenges, inherent in these prevention trials, include the difficulty in recruiting a large sample size of pre-clinical populations and the long trial durations. To select the potential subjects from older adults without cognitive decline, the A4 trial conducted amyloid PET scans on 4,486 individuals to identify 1,323 Aβ + individuals; the amyloid PET screen failure rate was 71% .
According to recent studies, AD pathology can be defined by plasma amyloid, tau, and neurodegeneration biomarker profiles; these profiles exhibit promising accuracy for predicting clinical progression in older adults without dementia . Although brain scans, cerebrospinal fluid, or plasma biomarker profiles possess great potential in screening populations for clinical trials, cognitively normal individuals require motivation to undergo these tests. Furthermore, efforts to reduce screening failures are required to increase the efficiency of clinical trials. Computer-based cognitive tests can be performed by anyone, anytime, anywhere, and are suitable for widespread screening of populations at risk for AD. Moreover, cognitive tests are more likely to identify early abnormalities by comparing the results with age-standardized values than by using a single cut-off value to identify abnormalities. Here, we aimed to determine the suitability of computer-based cognitive tests and identify the appropriate cut-off values for age-standardized values for screening older adults with Aβ accumulation.
We included 103 individuals aged ≥ 65 years (mean age, 74 years) from a sub-study of the National Centre for Geriatrics and Gerontology–Study of Geriatric Syndromes (NCGG-SGS), a national cohort study in Japan . The subjects who were undergoing treatment for any substantial medical, neurological, or psychiatric disease, had clinically significant focal brain lesions on MRI, and/or scored < 21 on the Mini-Mental State Examination (MMSE)  were excluded. Subjects were recruited between September 2017 and December 2019, and PET and cognitive tests were performed between October 2017 and January 2020. The protocol for this study (ID: UMIN000030319) was registered in the University Hospital Medical Information Network Clinical Trials Registry website (http://www.umin.ac.jp/ctr/index.htm).
Aβ-PET imaging was performed with 18F-florbetapir. All PET scans were obtained with a PET-computed tomographic camera (Biograph 16 True Point TV, Siemens AG, Germany). Subjects underwent 3D PET imaging for 50–70 min after intravenous injection of 370 MBq 18F-florbetapir. The participants’ Aβ-PET dichotomization (Aβ + /Aβ −) status was visually assessed independently by two radiologists blinded to clinical or biomarker information. Consensus was obtained in case of disagreement between the two radiologists in visual reading.
The National Centre for Geriatrics and Gerontology-Functional Assessment Tool (NCGG-FAT)  and the MMSE were used as cognitive tests. The NCGG-FAT has high test–retest reliability, moderate-to-high criterion-related validity , and predictive validity  among community-dwelling older persons. The NCGG-FAT has several advantages over traditional neurocognitive assessments. First, the NCGG-FAT is easily administered using a tablet PC with on-screen instructions. Therefore, it is not necessary for assessors to have in-depth knowledge of neurocognitive measures, and the individual assessor does not strongly influence the results. The simplicity and portability of the application allows assessment in community and non-clinical settings by non-specialists. Participants were able to complete the NCGG-FAT battery in approximately 20–30 min. An equivalent battery of traditional psychiatric tests would take twice as long to complete the assessment. The NCGG-FAT could be useful for cognitive screening in a population-based sample to assess the risk of cognitive decline in multidimensional functions. In addition, data collected from a large population using tablet PCs can be aggregated quickly because the data are digital rather than paper-based. The assessors of the cognitive tests were blinded to clinical information and the test results. The computer-based NCGG-FAT consists of the following domains: (1) memory (word list memory-I [immediate recognition] and word list memory-II [delayed recall]); (2) attention (an electronic tablet version of the Trail Making Test, TMT-part A); (3) executive function (an electronic tablet version of the TMT-part B); and (4) processing speed (an electronic tablet version of the Digit Symbol Substitution Test, DSST). Here, for all tests, established standardized thresholds were used to define impairment in the corresponding domain for a population-based cohort comprising 19,000 community-dwelling older persons (scores > 1.5 or > 1.0, standard deviations (SDs) below the age- and education-specific means). The MMSE score was set at an absolute of < 23, < 24, or < 26 for individuals with < 12, 12–15, or ≥ 16 years of education, respectively . The MMSE is the most commonly used cognitive test around the world and was used in this study to compare the NCGG-FAT as an estimated measure of Aβ accumulation. The cognitive scores were converted to Z-scores using mean and standard deviation.
Independent t-tests were used to compare cognitive tests between Aβ + and Aβ − subjects. Multiple logistic regression analysis was used to identify the relationships between cognitive tests, and Aβ burden was adjusted for age, sex, educational attainment, hypertension, diabetes, smoking, body mass index, living alone, and the 15-item Geriatric Depression Scale (GDS-15) . We calculated accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), relative risk, and number needed to screen (NNS) for Aβ + status; for each test, the cut-off scores were > 1.5 or > 1.0 (SDs below the age- and education-specific means). We chose to include both a moderate level of impairment (1.5 SD) as well as a mild level of impairment (1.0 SD), which is used in mild cognitive impairment literature in an attempt to identify cognitive changes at the earliest possible point . The NNS was calculated as 1/ PPV (equivalent to identifying one Aβ + individual using cognitive screening).
Of the 103 participants, 18 were Aβ + (17.5%). On comparing NCGG-FAT between the Aβ + and Aβ − groups, word list memory (p = 0.039) and DSST (p = 0.004) were significantly lower in the Aβ + group; no significant differences were observed in other cognitive tests (Fig. 1). On multiple logistic regression analysis, Aβ burden was significantly associated with word list memory (odds ratio [OR] = 0.42; 95% confidence interval [95% CI], 0.19–0.91) and DSST (OR = 0.35; 95% CI, 0.14–0.85) (Table 1).
Table 2 shows the predicted values of each cognitive test for Aβ positivity. The accuracy, sensitivity, and specificity of each cognitive function test were 0.73–0.83, 0.06–0.39, and 0.80–0.94, respectively; relatively high predictive values were observed for word list memory and DSST. The word list memory and DSST with 1.5 SD showed PPVs of 0.45 (95% CI, 0.16–0.75) and 0.50 (95% CI, 0.22–0.78), respectively and NNS as 2.22 and 2.00, respectively; both were observed to be better predictors than the other items. At a cut-off value of 1.0 SD, although the sensitivity improved, the other predictive values decreased, and the NNS worsened for both the memory test and DSST (NNS, 3.45 and 2.63, respectively).
Targeting the preclinical or prodromal stages of AD is believed to provide the best window for therapeutic intervention. ClinicalTrials.gov lists more than 450 Alzheimer's disease clinical trials requiring approximately 70,000 subjects; thus, this raises the challenge of how to efficiently identify and screen subjects . Prevention trials increasingly depend on expensive brain scans or cerebrospinal fluid biomarkers to identify at-risk subjects; however, these methods have high screening failure rates because only about one-third of asymptomatic individuals may test positive . According to a recent PET study, by leveraging longitudinal data in individuals classified as Aβ − at baseline, it was possible to detect early synchrony between declining memory and increasing amyloid burden .
Here, we compared computer-based cognitive tests to determine their suitability and identified an appropriate cut-off value for screening older adults with Aβ accumulation. The results demonstrated a significant association of Aβ burden with word list memory and DSST on multiple logistic regression analysis. The PPV of word list memory and DSST, with 1.5 SD, was 0.45 and 0.50, respectively, which was higher than those of the other items. This indicated that word list memory and DSST may be useful for screening older individuals with Aβ accumulation.
A systematic review of the diagnostic accuracy of the MMSE concluded that it may not be a suitable diagnostic tool for dementia  or has no advantage over shorter tests . Recommendations pertaining to cognitive screening for MCI are even more uncertain . Moreover, the MMSE has known limitations including its length , non-linearity , a floor effect in advanced dementia, and a ceiling effect in very mild dementia . In this study, computer-based memory and processing speed tests showed better associations than the MMSE as a measure of amyloid accumulation in the brain, suggesting the greater benefit of computer-based tests in understanding brain pathology.
According to a recent study from the Trial-Ready Cohort in Preclinical/Prodromal Alzheimer’s Disease, predictive models in a web-based registry can increase the efficiency of screening in future trials for AD prevention. On A4 trial web screening test, including demographics, Cogstate brief battery, family history, and Cognitive Function Instrument, the accuracy, sensitivity, specificity, PPV, and NNS, at a standardized uptake value ratio threshold value of 1.05, were 54.9%, 60.7%, 52.8%, 31.8%, and 3.14%, respectively . The NCGG-FAT showed higher PPVs in memory and DSST of 0.45 and 0.50, respectively, compared with the A4 trial web screening test findings. We concluded that the NCGG-FAT has the potential to reduce screen failure rates in Aβ + individuals to a level equal to or greater than the A4 trial web screening test. We believe that NCGG-FAT has shown excellent findings in NNS and may be useful for low-cost screening of Aβ + individuals. However, many older adults are unfamiliar with digital devices and require adequate practice to administer the test; to address this issue, the NCGG-FAT includes a practice session prior to testing.
The study had some limitations. First, the participants were not recruited randomly from the NCGG-SGS, which may have led to an underestimation of Aβ burden-prevalence; the participants were relatively healthy older individuals with the ability to access health check-up from their homes. Second, the number of individuals with Aβ accumulation in our database was limited, and a possible bias may have affected our results. Third, we did not adjust our analysis for the measurement of Apolipoprotein E, a major biomarker of Aβ accumulation, which may have contributed to the bias. Fourth, the results of this study are based on a cross-sectional study and need to be validated by prospective studies using large populations in the future.
Availability of data and materials
Restrictions apply to the availability of data generated or analyzed during this study to preserve participant confidentiality. The corresponding author will, on request, provide details on the restrictions and any conditions under which access to some data may be provided.
Anti-Amyloid Treatment in Asymptomatic Alzheimer's Disease
Digit Symbol Substitution Test
Geriatric Depression Scale
Mini-Mental State Examination
National Centre for Geriatrics and Gerontology-Functional Assessment Tool
National Centre for Geriatrics and Gerontology-Study of Geriatric Syndromes
Number needed to screen
Negative predictive value
Positron emission tomography
Positive predictive value
Trail Making Test
Aisen PS, Siemers E, Michelson D, Salloway S, Sampaio C, Carrillo MC, Sperling R, Doody R, Scheltens P, Bateman R, et al. What have we learned from expedition III and EPOCH Trials? Perspective of the CTAD Task Force. J Prev Alzheimers Dis. 2018;5(3):171–4.
Sperling RA, Aisen PS, Beckett LA, Bennett DA, Craft S, Fagan AM, Iwatsubo T, Jack CR Jr, Kaye J, Montine TJ, et al. Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):280–92.
Sperling RA, Donohue MC, Raman R, Sun CK, Yaari R, Holdridge K, Siemers E, Johnson KA, Aisen PS, Team AS. Association of factors with elevated amyloid burden in clinically normal older individuals. JAMA neurology. 2020;77(6):735–45.
Shen XN, Li JQ, Wang HF, Li HQ, Huang YY, Yang YX, Tan L, Dong Q, Yu JT. Alzheimer’s Disease Neuroimaging I: Plasma amyloid, tau, and neurodegeneration biomarker profiles predict Alzheimer’s disease pathology and clinical progression in older adults without dementia. Alzheimers Dement (Amst). 2020;12(1):e12104.
Shimada H, Makizako H, Doi T, Yoshida D, Tsutsumimoto K, Anan Y, Uemura K, Ito T, Lee S, Park H, et al. Combined prevalence of frailty and mild cognitive impairment in a population of elderly Japanese people. J Am Med Dir Assoc. 2013;14(7):518–24.
Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–98.
Makizako H, Shimada H, Park H, Doi T, Yoshida D, Uemura K, Tsutsumimoto K, Suzuki T. Evaluation of multidimensional neurocognitive function using a tablet personal computer: test-retest reliability and validity in community-dwelling older adults. Geriatr Gerontol Int. 2013;13(4):860–6.
Shimada H, Makizako H, Park H, Doi T, Lee S. Validity of the national center for geriatrics and gerontology-functional assessment tool and mini-mental state examination for detecting the incidence of dementia in older Japanese adults. Geriatr Gerontol Int. 2017;17(12):2383–8.
Satizabal CL, Beiser AS, Chouraki V, Chene G, Dufouil C, Seshadri S. Incidence of dementia over three decades in the Framingham heart study. N Engl J Med. 2016;374(6):523–32.
Yesavage JA. Geriatric depression scale. Psychopharmacol Bull. 1988;24(4):709–11.
Trittschuh EH, Crane PK, Larson EB, Cholerton B, McCormick WC, McCurry SM, Bowen JD, Baker LD, Craft S. Effects of varying diagnostic criteria on prevalence of mild cognitive impairment in a community based sample. J Alzheimers Dis. 2011;25(1):163–73.
Doraiswamy PM, Narayan VA, Manji HK. Mobile and pervasive computing technologies and the future of Alzheimer’s clinical trials. NPJ Digit Med. 2018;1:1.
Cummings J, Aisen P, Barton R, Bork J, Doody R, Dwyer J, Egan JC, Feldman H, Lappin D, Truyen L, et al. Re-engineering Alzheimer clinical trials: global Alzheimer’s platform network. J Prev Alzheimers Dis. 2016;3(2):114–20.
Landau SM, Horng A, Jagust WJ. Alzheimer’s Disease Neuroimaging I: Memory decline accompanies subthreshold amyloid accumulation. Neurology. 2018;90(17):e1452–60.
Tombaugh TN, McIntyre NJ. The mini-mental state examination: a comprehensive review. J Am Geriatr Soc. 1992;40(9):922–35.
Jacova C, Kertesz A, Blair M, Fisk JD, Feldman HH. Neuropsychological testing and assessment for dementia. Alzheimer’s Dementia. 2007;3(4):299–317.
Petersen RC, Stevens JC, Ganguli M, Tangalos EG, Cummings JL, DeKosky ST. Practice parameter: early detection of dementia: mild cognitive impairment (an evidence-based review). Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 2001;56(9):1133–42.
Mitchell JI, Long JC, Braithwaite J, Brodaty H. Social-professional networks in long-term care settings with people with Dementia: an approach to better care? A systematic review. J Am Med Dir Assoc. 2016;17(2):183 e117-127.
Proust-Lima C, Amieva H, Dartigues JF, Jacqmin-Gadda H. Sensitivity of four psychometric tests to measure cognitive changes in brain aging-population-based studies. Am J Epidemiol. 2007;165(3):344–50.
Mitchell AJ. A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res. 2009;43(4):411–31.
Langford O, Raman R, Sperling RA, Cummings J, Sun CK, Jimenez-Maggiora G, Aisen PS, Donohue MC. Predicting amyloid burden to accelerate recruitment of secondary prevention clinical trials. J Prev Alzheimers Dis. 2020;7(4):213–8.
We would like to thank the following researchers for their assistance with the study assessments: Dr. Sangyoon Lee, Dr. Bae Seongryu, Mr. Kenji Harada, Mr. Yohei Shinkai, Dr. Osamu Katayama, and Mr. Ippei Chiba. We would like to thank FUJIFILM Toyama Chemical Co., Ltd. for providing us with 18F-florbetapir.
This work was supported by the JSPS KAKENHI Grant-in-Aid for Scientific Research (B) (Grant 23300205) and for Challenging Exploratory Research (Grant 17H06286). The funding source had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.
Ethics approval and consent to participate
The Ethics Committee of the National Centre for Geriatrics and Gerontology approved the study protocol (registration number: 1021–7), and informed consent was obtained from all participants before inclusion. All methods in the study were performed in accordance with guidelines and regulations of the Declaration of Helsinki.
Consent for publication
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Shimada, H., Makino, K., Kato, T. et al. Computer-based cognitive tests and cerebral pathology among Japanese older adults. BMC Geriatr 23, 226 (2023). https://doi.org/10.1186/s12877-023-03918-x
- Alzheimer’s disease