Skip to main content
  • Research article
  • Open access
  • Published:

Predicting incidence of long-term care insurance certification in Japan with the Kihon Checklist for frailty screening tool: analysis of local government survey data

Abstract

Background

Predicting incidence of long-term care insurance (LTCI) certification in the short term is of increasing importance in Japan. The present study examined whether the Kihon Checklist (KCL) can be used to predict incidence of LTCI certification (care level 1 or higher) in the short term among older Japanese persons.

Methods

In 2015, the local government in Tokyo, Japan, distributed the KCL to all individuals older than 65 years who had not been certified as having a disability or who had already been certified as requiring support level 1–2 according to LTCI system. We also collected LTCI certification data within the 3 months after collecting the KCL data. The data of 17,785 respondents were analyzed. First, we selected KCL items strongly associated with incidence of LTCI certification, using stepwise forward-selection multiple logistic regression. Second, we conducted receiver operating characteristic (ROC) analyses for three conditions (1: Selected KCL items, 2: The main 20 KCL items (nos. 1–20), 3: All 25 KCL items). Third, we estimated specificity and sensitivity for each condition.

Results

During a 3-month follow-up, 81 (0.5%) individuals required new LTCI certification. Eight KCL items were selected by multiple logistic regression as predictive of certification. The area under the ROC curve in the three conditions was 0.92–0.93, and specificity and sensitivity for all conditions were greater than 80%.

Conclusions

Three KCL conditions predicted short-term incidence of LTCI certification. This suggests that KCL items may be used to screen for the risk of incident LTCI certification.

Peer Review reports

Background

The long-term care insurance (LTCI) system was introduced in Japan in the year 2000 to support people with disabilities via social security expenses [1]. The number of LTCI certifications in Japan is estimated to reach 9.88 million (8.9% of the total population) by 2040, a figure that is approximately double that of 2010 [2]. Increasing social security expenses represents a serious future issue [3].

The LTCI system in Japan provides people with services according to certification as one of seven levels (support levels 1–2 and care levels 1–5) depending on their disease condition or functional ability. LTCI care levels require more care than LTCI support levels, and the higher the level, the more care required. LTCI certification levels are defined for example as follows Support level 1: “limited in instrumental activities of daily living but independent in basic activities of daily living”; care level 2: “requiring assistance in at least one basic task of activities of daily living”; care level 5: “requiring care in all tasks of activities of daily living.” People with support level 1–2 can receive preventive services, as may people who are not certified as having a disability [1]. To assess whether a person is eligible for preventive services or LTCI services, use of the Kihon Checklist (KCL) has been recommended by the Ministry of Health, Labour and Welfare, of Japan [4]. The KCL is a screening tool developed in Japan for use with older people with frailty [5]. It is used by local governments and in community consultations to screen for persons eligible for long-term care prevention programs and to assess the effectiveness of interventions. The KCL was automatically sent to all individuals older than 65 years on an annual basis up until 2014, but is now administered at the discretion of each local government. The KCL consists of 25 items; it requires approximately 15 min for an older person to answer all items [6].

In Japan, LTCI certifications are determined at the Municipal Certification Committee, based on the assessment of the degree of functional disability using a questionnaire developed by the Ministry of Health, Labour and Welfare and reference to the “Doctor’s Opinion Paper” prepared by the attending physician [1]. The “Doctor’s Opinion Paper” contains the opinion of the primary care doctor on the applicant’s chronic condition in daily life that cannot be determined by the questionnaire. There is a cost of 10,000 to 20,000 Yen (100 to 200 US dollars) per applicant associated with determining LTCI certification at the Municipal Certification Committee [7]. However, if preventive services were provided to applicants who are not assessed as requiring care level 1 or higher based on KCL screening, the costs of unnecessary certification would reduce. Preventive services are programs for individuals who are not certified as having a disability or have LTCI support level 1–2, and are thus available without LTCI certification. One preventive service (service “C”) aims to gain intensive improvement in participants’ activities of daily living, instrumental activities of daily living, and physical fitness over a short term of 3 to 6 months; the service program is reviewed a minimum of 3 months after onset [4]. Therefore, to reduce social security expenses, it is necessary to use the KCL to determine persons who will likely change their care needs to care level 1 or higher in a short term of 3 months.

Previous studies have shown that each item and the seven domains of the KCL are associated with incidence of LTCI certification [8,9,10,11,12,13]. One study reported that each domain of the KCL was useful for predicting the occurrence of LTCI certification (support level 1 or higher) in the subsequent year; the main 20 items of the KCL (nos. 1–20) had the greatest predictive ability [9]. Another study reported that the nutrition domain, memory domain, and mood domain of the KCL were associated with 2-year incidence of LTCI certification (care level 2 or higher) [10].

However, these studies examined the association between the KCL and incidence of LTCI certification over more than a 1-year period; to the best of our knowledge, the ability of the KCL to predict incidence of LTCI certification in the short term has not been reported. Moreover, it would be useful to be able to predict incidence of LTCI certification using a subset of KCL items focused on practical situations.

Accordingly, the present study examined whether the KCL can be used to predict incidence of LTCI certification (care level 1 or higher) in the short term. To do so, we compared the utility of different KCL items for predicting incidence of LTCI certification during a 3-month period, among the different item sets, using a large dataset collected by a local government in Japan.

Methods

Participants

We used KCL data that were collected by the local government in Tokyo, Japan, in May 2015 to screen for frail older persons. The KCL was distributed by mail to 26,630 individuals older than 65 years who were not certified as with a disability or with LTCI support level 1–2; the data of 17,785 individuals (66.8%) who provided responses were analyzed. The ethics committee of the Tokyo Metropolitan Institute of Gerontology approved the present study. We explained in writing to the local government the purpose of the present study and how the results would be disseminated, and we obtained written consent to use the data.

KCL at baseline

The KCL consists of 25 self-report (yes/no) questions that assess seven domains: instrumental activities of daily living, physical strength, nutrition, oral function, isolation, memory, and mood [5]. The main 20 items of the KCL (nos. 1–20), which are often used as the overall score of frailty, include six domains and exclude a mood domain [8]. Responses are summed to obtain a KCL score for each participant. A higher KCL score indicates a higher risk of frailty.

Follow-up (incidence of LTCI certification)

Combined anonymized data on both the KCL and incidences of LTCI certification in the 3 months after collecting the KCL data were retrieved from the local government. Incidence of LTCI certification was defined as care level 1 or higher.

Statistical analysis

First, we used stepwise forward-selection logistic regression analysis, adjusting for sex and age, and using all 25 items of the KCL as explanatory variables to determine KCL items that were strongly associated with incidence of LTCI certification. We conducted receiver operating characteristic (ROC) analyses to calculate the area under the curve (AUC) and 95% confidence intervals for incidence of LTCI certification for following three conditions: i) KCL items selected based on the regression analysis described above, ii) the main 20 items of the KCL (nos. 1–20) which had better predictive utility based on previous study [9], and iii) all 25 items of the KCL. We additionally conducted a five-fold cross validation test for the selected items of the KCL. We created five data sets by randomly splitting all of the data into a training set and a test set with the ratio of 70–30%, and estimated the accuracy of predicting incidence of LTCI certification. Third, we estimated the specificity and sensitivity for incidence of LTCI certification in each condition, based on the Youden Index value [14, 15]. Specificity indicates the rate of individuals who are assessed as not requiring LTCI certification when they are not certified, and sensitivity indicates the rate of individuals who are assessed as requiring it when they are certified. All data were analyzed using SPSS Statistics 23.0 (SPSS Inc., Chicago, IL, USA). All statistical tests were two-sided, and differences at P < 0.05 were accepted as significant.

Results

Population characteristics

The study population comprised 7827 men (44.0%) and 9958 women (56.0%), with a median age of 74 years (range: 65–107 years). During the 3-month follow-up, 81 persons (0.5%) required new LTCI certification.

Selected KCL items

Table 1 shows the eight KCL items that were selected using the regression, and sex- and age-adjusted odds ratios (ORs) and confidence intervals (CIs) of incidence of LTCI certification for each item. The selected eight KCL items included three items from the instrumental activities of daily living domain, one item from the physical strength domain, one item from the nutrition domain, one item from the memory domain, and two items from the mood domain.

Table 1 Association between the selected eight Kihon checklist items and incidence of long-term care insurance certification

ROC analysis

The AUC of the selected eight KCL items was 0.93 (95% CI 0.90–0.96), which was the highest of the three conditions (Fig. 1). The AUCs of the main 20 items and all 25 items were 0.92 (95% CI 0.88–0.95) and 0.92 (95% CI 0.89–0.95), respectively. The five-fold cross validation test for the selected eight items indicated that the average AUC of the five training sets was 0.94, and that of the five test sets was 0.92.

Fig. 1
figure 1

Area under the receiver operating characteristic curve in the three Kihon Checklist conditions

Specificity and sensitivity

The optimal cut-off points determined by the Youden Index were ≥ 3/8 for the selected eight items (specificity 82.1%, sensitivity 92.6%), ≥8/20 for the main 20 items (specificity 89.7%, sensitivity 82.7%), and ≥ 9/25 for all 25 items (specificity 85.1%, sensitivity 87.7%; Table 2).

Table 2 Specificity and sensitivity according to the cut-off points in the three Kihon Checklist conditions

Discussion

In the present study, we investigated that whether the KCL can be used to predict incidence of LTCI certification (care level 1 or higher) in the short term. Eight KCL items that were strongly associated with LTCI certification included three items from the instrumental activities of daily living domain, one item from the physical strength domain, one item from the nutrition domain, one item from the memory domain, and two items from the mood domain. The AUC of these eight items with respect to incidence of LTCI certification was 0.93, which was as high as that of the main 20 items and all 25 items. This result suggests that the eight items sufficiently predicted short-term incidence of LTCI certification.

There was likely no significant difference in the age distribution of the participants in the present study and overall older population in Japan because we used large-scale data from complete enumeration obtained by the local government [16]. Moreover, the proportion of participants with LTCI certification (support level 1–2) at baseline was comparable to that of the whole Japanese population [16, 17]; thus, the data in the present study were representative of community-dwelling older persons in Japan. Tomata [9] reported that the 2.2% of individuals required new LTCI certification (care level 1 or higher) during a one-year follow-up, versus 0.5% during a three-month follow-up in the present study; the latter value might have been affected by the shorter follow-up period, however, we believe that the accuracy of the statistical analysis performed on the large-scale sample of 17,785 people is sufficient, and that the findings obtained can be generalized.

Tomata reported that the items of the KCL that were associated with 1-year incidence of LTCI certification (support level 1 or higher), through forced-entry logistic regression analysis, were two items from the instrumental activities of daily living domain, two items from the physical strength domain, one item from the nutrition domain, one item from the isolation domain, and three items from the cognitive function domain [9]. Of these items, item 4 (“Do you sometimes visit your friends?”), of the domain of instrumental activities of daily living, and item 19 (“Do you make a call by looking up phone numbers?”), of the cognitive function domain, were also selected in our study. In addition to these items, items from the mood domain were selected in the present study. Fukutomi reported no association between the mood domain and two-year incidence of LTCI certification (support level 1 or higher) [11]. However, Hamazaki reported a significant association between the mood domain and two-year incidence of LTCI certification (care level 2 or higher) [10]. Therefore, items of the mood domain might be associated with short-term and more severe need for LTCI certification.

The utility of KCL items for predicting incidence of LTCI certification in the present study was notably greater than that reported in previous studies: The AUC in ROC analysis was 0.62–0.83 in Tomata et al. [9] and 0.78 in Tsuji et al. [13] The present study focused on the need for LTCI certification at care level 1 or higher over a shorter follow-up period than that in previous studies; the results suggested that the KCL was useful for predicting incidence of LTCI certification in the short term. Moreover, we found that the eight selected items were as efficacious for predicting incidence of LTCI certification as were the main 20 items and all 25 items. This result was also confirmed by a five-fold cross validation test.

The specificity and sensitivity for incidence of LTCI certification based on the selection criteria of the LTCI for people at high risk were 57.8 and 73.5%, respectively, according to national research in Japan, and 63.4 and 78.1%, respectively, in Tomata et al. [9] Moreover, specificity and sensitivity were 73.1 and 70.5%, respectively in a previous study that used part of the KCL [13]. Specificity and sensitivity in the present study were both more than 80%, values higher than those reported in previous studies.

In the present study, sensitivity using the eight selected items was highest at 92.6%, using a cut-off point of ≥3/8 items; however, specificity was highest at 91.3%, using a cut-off point of ≥4/8 items. These results suggest that it might be useful to change the cut-off point of the eight selected items according to various situations. The ≥3/8 cut-off point might be recommended if fewer false negatives are required during primary screening, while the ≥4/8 cut-off point might be recommended if fewer false positives are required during medical examinations.

Using the results of the present study, we simulated the reduction in social security expenses when the local government determined no LTCI certification at Municipal Certification Committees for service applicants who were assessed as not having a disability or as having LTCI support level 1–2, based on the KCL. In the 2015 national report, there were one million people targeted for them. Thus, this would be expected to reduce social security expenses by 10 billion Yen (100 million US dollars) per year, given a cost of 10,000 Yen (100 US dollars) per person to determine LTCI certification at the Municipal Certification Committee [7, 18]. It is recommended that local governments evaluate applicants for LTCI services using the eight items of the KCL to determine whether LTCI services or preventive services are suitable. If by using the eight items, older adults are deemed unlikely to be certified as requiring care level 1 or higher after 3-months, they may use preventive services. Globally, governments seeking to introduce LTCI might be able to reduce social security expenses by using the eight items to conduct a primary screening to assess those requiring LTCI certification before determining the service applicant’s LTCI certification.

The present study had several limitations. We used data that were collected by the local government to screen for older persons with frailty; however, the response rate was 66.8%. Thus, the study population might not have included people at higher risk of incidence of LTCI certification. Because not all participants eligible for LTCI certification actually apply for LTCI certification, the present study might have included detection bias. Since we did not investigate disease and functional ability, how health status affected LTCI certification was unclear.

Conclusions

In conclusion, the present study showed that the main 20 items and all 25 items could be used to predict incidence of LTCI certification during a subsequent 3-month period with high accuracy. Moreover, eight selected items of the KCL also could be used to make this prediction. Therefore, the items of the KCL that are included could be changed according to various situations when predicting short-term incidence of LTCI certification.

Availability of data and materials

The data of this study cannot be released publicly due to ethicolegal restrictions imposed by the Ethics Committee at Tokyo Metropolitan Institute of Gerontology. Datasets generated may be available from the corresponding author on reasonable request, after ethical considerations.

Abbreviations

LTCI:

Long-term care insurance

KCL:

Kihon Checklist

ROC:

Receiver operating characteristic

AUC:

Area under the curve

OR:

Odds ratio

CI:

Confidence interval

BMI:

Body mass index

References

  1. Ministry of Health, Labour and Welfare. Long-Term Care Insurance System of Japan: Ministry of Health, Labour and Welfare; 2016. https://www.mhlw.go.jp/english/policy/care-welfare/care-welfare-elderly/dl/ltcisj_e.pdf. Accessed 3 Jan 2020.

  2. Ministry of Economy, Trade and Industry. Study Group on Elderly Person Care System for Future Care Supply and Demand: Ministry of Economy, Trade and Industry; 2018. https://www.meti.go.jp/press/2018/04/20180409004/20180409004-2.pdf. Accessed 3 Jan 2020. (in Japanese).

  3. Ministry of Health, Labour and Welfare. The situation surrounding the Long-Term Care Insurance System. Ministry of Health, Labour and Welfare. https://www.mhlw.go.jp/seisakunitsuite/bunya/hukushi_kaigo/kaigo_koureisha/chiiki-houkatsu/dl/link1-2.pdf. Accessed 3 Jan 2020. (in Japanese).

  4. Ministry of Health, Labour and Welfare. Outline of Prevent Long-term Care Need and Daily Living Service Projects: Ministry of Health, Labour and Welfare; 2016. https://www.mhlw.go.jp/file/06-Seisakujouhou-12300000-Roukenkyoku/0000088276.pdf. Accessed 3 Jan 2020. (in Japanese).

  5. Arai H, Satake S. English translation of the Kihon checklist. Geriatr Gerontol Int. 2015;15(4):518–9.

    Article  Google Scholar 

  6. Sewo Sampaio PY, Sampaio RA, Yamada M, Ogita M, Arai H. Validation and translation of the Kihon checklist (frailty index) into Brazilian Portuguese. Geriatr Gerontol Int. 2014;14(3):561–9.

    Article  Google Scholar 

  7. Ministry of Internal Affairs and Communications. Reduction of administrative burden related to Long-Term Care Insurance Certification. Ministry of Internal Affairs and Communications. https://www.soumu.go.jp/main_content/000315422.pdf. Accessed 3 Jan 2020. (in Japanese).

  8. Sewo Sampaio PY, Sampaio RA, Yamada M, Arai H. Systematic review of the Kihon checklist: is it a reliable assessment of frailty? Geriatr Gerontol Int. 2016;16(8):893–902.

    Article  Google Scholar 

  9. Tomata Y, Hozawa A, Ohmori-Matsuda K, et al. Validation of the Kihon checklist for predicting the risk of 1-year incident long-term care insurance certification: the Ohsaki cohort 2006 study. Nihon Koshu Eisei Zasshi. 2011;58(1):3–13.

    PubMed  Google Scholar 

  10. Hamazaki Y, Morikawa Y, Nakamura K, Morimoto S, Nakagawa H. Relationship between participation in a secondary screening test for the frail elderly and new onset of the need for long-term care among the elderly in a rural town. Nihon Koshu Eisei Zasshi. 2012;59(11):801–9.

    PubMed  Google Scholar 

  11. Fukutomi E, Okumiya K, Wada T, et al. Relationships between each category of 25-item frailty risk assessment (Kihon checklist) and newly certified older adults under long-term care insurance: a 24-month follow-up study in a rural community in Japan. Geriatr Gerontol Int. 2015;15(7):864–71.

    Article  Google Scholar 

  12. Kamegaya T, Yamaguchi H, Hayashi K. Evaluation by the basic checklist and the risk of 3 years incident long-term care insurance certification. J Gen Fam Med. 2017;18(5):230–6.

    Article  Google Scholar 

  13. Tsuji T, Takagi D, Kondo N, Kondo K. Development of risk assessment scales for needed support/long-term care certification: a longitudinal study using the Kihon checklist and medical assessment data. Nihon Koshu Eisei Zasshi. 2017;64(5):246–57.

    PubMed  Google Scholar 

  14. Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32–5.

    Article  CAS  Google Scholar 

  15. Perkins NJ, Schisterman EF. The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristics curve. Am J Epidemiol. 2006;163(7):670–5.

    Article  Google Scholar 

  16. Statistics Office of Japan (e-Stat). Estimated future population. Statistics Office of Japan (e-Stat); 2016. https://www.e-stat.go.jp/stat-search/files?page=1&layout=datalist&toukei=00200524&bunya_l=02&tstat=000000090001&cycle=0&year=20160&tclass1=000001011678&tclass2=000001091515. Accessed 3 Jan 2020. (in Japanese).

  17. Ministry of Health, Labour and Welfare. Long-Term Care Insurance Status Reports: Ministry of Health, Labour and Welfare; 2015. https://www.mhlw.go.jp/topics/kaigo/osirase/jigyo/m15/1510.html. Accessed 3 Jan 2020. (in Japanese).

  18. Ministry of Health, Labour and Welfare. A summary of municipal governments. Ministry of Health, Labour and Welfare. https://www.google.com/url?client=internal-elementcse&cx=005876357619168369638:ydrbkuj3fss&q=https://www.mhlw.go.jp/file/06-Seisakujouhou-12300000-Roukenkyoku/sityousongaiyou_1.xlsx&sa=U&ved=2ahUKEwjmvoDlkrnnAhUVc3AKHa0xDBIQFjAEegQIBRAB&usg=AOvVaw30cDRuhE4uO1AIj8iboku8. Accessed 3 Jan 2020. (in Japanese).

Download references

Acknowledgements

Not applicable.

Funding

No funding was received for this study.

Author information

Authors and Affiliations

Authors

Contributions

K.I., H.K., and S.O. conceived and designed the experiments. K.I., H.K., and S.O. performed the experiments. K.I. analyzed the data. K.I. and H.K. wrote the paper. H.T. verified the analytical methods. All authors discussed the results and approved the final manuscript.

Corresponding author

Correspondence to Shuichi Obuchi.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for the study was granted by the ethics committee of the Tokyo Metropolitan Institute of Gerontology (Acceptance no. 4, 2018). No administrative permissions or licenses were required to utilize the survey data in our research. The local government provided the data on the basis of research consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ito, K., Kawai, H., Tsuruta, H. et al. Predicting incidence of long-term care insurance certification in Japan with the Kihon Checklist for frailty screening tool: analysis of local government survey data. BMC Geriatr 21, 22 (2021). https://doi.org/10.1186/s12877-020-01968-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12877-020-01968-z

Keywords