Agreement between administrative data and the Resident Assessment Instrument Minimum Dataset (RAI-MDS) for medication use in long-term care facilities: a population-based study

Data sources

Data for this study were from the province of Saskatchewan, Canada, which has a population of approximately 1.1 million according to the 2011 Statistics Canada Census, and which has a universal healthcare program and therefore captures virtually all healthcare contacts for the entire population. The province maintains comprehensive health care databases in electronic format and these can be anonymously linked via a unique personal health number [7].

RAI-MDS Version 2.0, prescription drug, and person registry system records from the 2010/11 fiscal year (a fiscal year extends from April 1 to March 31), the most current year available at the time of the study, were used to conduct the research. The RAI-MDS, originally developed in the US by the Centers for Medicare and Medicaid Services, captures information about care and functioning of LTCF residents. This information is collected by trained assessors, usually nurses, who use interviews with the person and family members, consultation with other clinicians, and chart review, to complete a form. Forms are required to be completed within 14 days of LTCF admission, quarterly and annually thereafter, and whenever there is a major change in a resident’s health status. The RAI-MDS also captures dates of LTCF admission and discharge and some characteristics of the LTCF itself. Saskatchewan was the first Canadian province to make the RAI-MDS mandatory in all LTCFs; this requirement was introduced in April 2001, although full implementation was not achieved until 2004.

The prescription drug database contains records of outpatient drugs dispensed to provincial residents eligible for insurance coverage. It does not capture inpatient medications, medications for residents of a small number of LTCFs with in-house pharmacies, and approximately 10% of the population who are covered by a federally funded pharmacare program (e.g., military, federal police, federal prisoners, First Nations residents). Each available record includes the date of dispensation and national drug identification number (DIN). DINs are linked to codes in the American Hospital Formulary System (AHFS) Pharmacologic-Therapeutic Classification System (www.ashp.org), which is used to group drugs with similar pharmacologic, therapeutic, and/or chemical characteristics.

The person registry system captures dates of health insurance coverage, demographic information, and location of residence. The accuracy and completeness of Saskatchewan’s administrative health data for research has been well documented [8-12] although not specifically for LTCF populations. Ethics approval for database access was received from the University of Saskatchewan Biomedical Research Ethics Board. Data were accessed and analyzed at the provincial Health Quality Council in accordance with a standing data sharing agreement between that organization and the Ministry of Health.

Study cohort

The study cohort inclusion criteria were: (a) resident in a LTCF for at least 60 days, (b) at least one admission, quarterly, or annual RAI-MDS assessment during the residency period, and (c) at least one record for a prescription drug during the residency period. The latter criterion was used to ensure that the cohort included individuals eligible to receive prescription drug benefits.

The assessment date of the first admission, quarterly, or annual RAI-MDS assessment in 2010/11 was the study index date. The observation period extended 30 days before and 30 days after the index date. LTCF residents who did not have continuous health insurance coverage and were not eligible for prescription drug benefits during the study observation period were excluded.

Study variables

Medication information was extracted from both the RAI-MDS and prescription drug databases. Socio-demographic variables were defined using the person registry data. Additional information on resident and facility characteristics were obtained from the RAI-MDS data.

The RAI-MDS captures the number of days of medication use in the seven-day period prior to the assessment reference date. The following medication classes are included in Section O of the assessment form: anti-psychotic, anti-depressant, anti-anxiety, hypnotic, and diuretic. Based on previous research, anti-anxiety and hypnotic medication classes were combined into a single category because they often have a similar indication and diuretics were excluded because they tend to have lower prevalence [13]. In the prescription drug administrative data, anti-psychotic, anti-depressant, and anti-anxiety/hypnotic medications were identified using generic drug names (see Additional file 1) and AHFS codes from the provincial formulary (i.e., drugs covered by prescription drug benefits). The prescription drug database was searched for dispensations of the three medications in the study observation window. We selected a 60-day observation window because the medications under investigation are commonly dispensed in one-month quantities [14]. A sensitivity analysis was conducted using a 28-day observation window (i.e., 14 days before and 14 days after the index date).

LTCF residents were described on the socio-demographic characteristics of age, sex, region of residence, and income quintile. All measures were defined at the index date. Urban or rural region of residence was assigned based on the postal code contained in the person registry. Urban residents were those living in one of the two health regions in the province that contain major urban centres, while rural residents were those individuals living in the remaining 11 health regions. Income quintile was an area-level measure assigned based on average household income from the Statistics Canada Census. Each individual’s postal code from the person registry was assigned to a dissemination area (DA), the smallest geographic unit for which Census data are reported. The entire Saskatchewan population was then divided into five approximately equal groups according to the DA average household income [15]. Preliminary analysis of the data showed that for three quarters (i.e., 74.0%) of cohort members, the index date corresponded to the initial assessment; accordingly, the resident’s postal code would correspond to the place of resident prior to admission.

Selected chronic diseases identified in the RAI-MDS were also captured for each member of the study cohort, including Alzheimer’s disease, dementia, mood disorders (i.e., anxiety, depression, bipolar), and schizophrenia, which are potential indications for the medications under investigations. These conditions were defined at the study index date using admission and annual assessments, or the most recent admission or annual assessment if the index date was the date of a quarterly assessment because not all diagnoses (e.g., schizophrenia) were captured in quarterly assessments.

Facility characteristics captured in the RAI-MDS data in Saskatchewan include the type of facility and its affiliation. These characteristics were defined at the study index date. LTCFs were classified as special care home or other; the former are public facilities for which residence is determined based on need, while the latter are typically private facilities. LTCF affiliation includes amalgamate, affiliate, and contract. Health regions may operate facilities on their own (amalgamate), or the facility may be operated by an independent health care organisation (affiliate), or through a contract for services with an independent organization (contract).

Statistical analysis

The study cohort was described using frequencies, percentages, means, and standard deviations (SDs). Crude prevalence estimates (percentages) were calculated for each type of prescription medication. Cohen’s κ was used to estimate agreement between the RAI-MDS and administrative data; 95% confidence intervals (CIs) were also computed. The interpretation of κ adopted in this study was [16]: κ < 0.20 is poor agreement, 0.20 ≤ κ ≤ 0.39 is fair agreement, 0.40 ≤ κ ≤ 0.59 is moderate agreement, 0.60 ≤ κ ≤ 0.79 is good agreement, and κ ≥ 0.80 is very good agreement.

Mixed-effects multiple logistic regression models were fit separately to the data for each medication to test resident and facility variables associated with disagreement between the two data sources [17]. All individuals who were identified as medication users in one data source but not in the other data source were included in the disagreement category, while individuals who were either identified as medication users or as medication non-users in both data sources were included in the agreement category. Two models were fit to the data: (a) null model, which contained a random facility intercept only, to account for clustering of patients within facilities, and (b) full model which contained a random intercept as well as resident and facility fixed-effect covariates. Facility type was excluded because it was collinear with affiliation. The intra-class correlation (ICC) was computed for the null and full models [17] using a latent variable method [18]. The Akaike Information Criterion (AIC) was used to compare model fit between the null and full models [19]. The c-statistic, which is equal to the area under the receiver operating characteristic (ROC) curve for dichotomous outcomes, was used to assess discriminative performance [20]; it was estimated for the full model when the clustering effect.

All regression coefficients were exponentiated and reported as odds ratios (ORs), along with 95% CIs. Analyses were conducted using SAS software, version 9.2 [21].