Measuring knowledge of Alzheimer’s: development and psychometric testing of the UJA Alzheimer’s Care Scale

Background Care for people with Alzheimer’s disease or other dementias should be based on up-to-date clinical practice guidelines. The education and training of nurses and other healthcare staff in dementia is a key factor in providing quality care. Knowledge of Alzheimer’s disease can be measured through questionnaires. The aim of this study was to develop and validate a scale to measure Alzheimer’s disease knowledge among both nursing staff and students. Methods This was a cross-sectional survey study undertaken in three stages: 1) development of the questionnaire and item wording; 2) content validation by an expert panel; 3) questionnaire testing with two samples to establish psychometric properties. Sample 1 comprised 361 Registered Nurses, Assistant Nurses and eldercare workers from 24 nursing homes in Jaén (southern Spain). Sample 2 comprised 297 nursing students. The data were analysed through item analysis and a Rasch model. Convergent and construct validity and internal consistency were also examined. Results The 23-item UJA Alzheimer’s Care Scale shows good outfit and infit values based on the Rasch model. One item presented differential functioning between Registered Nurses and Assistant Nurses. The intraclass correlation coefficient between the UJA Alzheimer’s Care Scale and the Spanish version of the Dementia Knowledge Assessment Tool 2 showed strong agreement among nursing staff (0.63) and students (0.79). The scale is able to distinguish between professionals with low or high knowledge of Alzheimer’s care. The overall Cronbach’s alphas were 0.70 (nursing staff) and 0.82 (nursing students). The intraclass correlation coefficient between the first test and the retest was good (0.84). Conclusions The UJA Alzheimer’s Care Scale is a useful tool for measuring knowledge of Alzheimer’s disease and dementia care among nursing professionals or nursing students. The initial validation study obtained good psychometric properties concerning validity and reliability. Electronic supplementary material The online version of this article (10.1186/s12877-019-1086-2) contains supplementary material, which is available to authorized users.


Background
Increased life expectancy and decreasing birth rates are among the causes of population ageing worldwide [1]. It is estimated that between 2015 and 2050, the proportion of the world's population over 60 years of age will increase from 12 to 22% (from 900 million to about 2000 million people) [2]. Dementia is one of the major health problems affecting older people according to the World Health Organization [2]. Age is the factor responsible for the greatest increase in the risk of dementia, as both the incidence and prevalence of dementia double every 5 years from the age of 65 [3]. Alzheimer's disease (AD) is the most frequent form of dementia (60-70% of cases), followed by vascular dementia (12.5-25%) [4]. In Spain, the figures of dementia prevalence range from 5 to 14.9% in people aged 65 or older and from 9 to 17.2% in those older than 70 [5,6].
To address the needs of the ageing population, particularly of people with dementia, the health systems in most countries have had to adapt by modifying socio-economic policies, changing available resources and designing new intervention strategies [2]. The implementation of clinical practice guidelines (CPGs) has a basic role in providing high-quality healthcare for people with dementia. Good CPGs must be developed by multidisciplinary teams and using recommendations based on the best available evidence [7]. However, there is still a gap between knowledge and usual clinical practice. Health professionals know the recommendations for AD care, but frequently do not put them into practice [8,9]. Specific training for health professionals providing care for AD or those with other forms of dementia is a key factor for quality care in all settings. The literature contains a number of studies measuring the knowledge of health professionals concerning dementia and AD care, using well-validated questionnaires. Education is important as most such studies have concluded that Registered Nurses (RNs) have higher knowledge than Assistant Nurses (ANs) [10,11]. A wide range of professionals participate in dementia care, with different education and specific training [12]. General practitioners (GPs) have considerable knowledge regarding the diagnosis and treatment of dementia, but relatively low knowledge of epidemiology. These professionals also have difficulties in communicating the dementia diagnosis to patients, coordinating with social services and managing behavioural changes [13]. In nursing homes and other residential facilities, ANs or eldercare staff provide most of the basic care for people with dementia, but these professionals have the lowest levels of training [14]. In some cases, family carers of people with dementia may have greater knowledge regarding care than nurses and other care workers [15]. Some authors highlight that the level of knowledge regarding dementia is not only influenced by training, but also experience and real contact with people with dementia [16]. Low knowledge of AD care can lead to diagnostic mistakes and inadequate treatment, among other issues [12]. Hughes et al. (2008) show that nursing home care staff have little training in the needs of the elderly, especially with regard to mental health, depression and dementia. Improving the training of healthcare staff has a major impact on care. The education of health professionals and specific training are the main factors driving organizational changes and better care. Interactive training and blended learning are the best ways of achieving small but lasting improvements in professional practice [17,18]. It is also important to consider knowledge of dementia care among healthcare students in the fields of nursing and medicine, inter alia. Several studies have found that clinical placements and mentorship play an important role in improving outcomes [19,20].
To design appropriate dementia care education programmes for healthcare workers, it is very important to measure the level of knowledge and identify the issues that are not well known. A number of questionnaires and scales have been developed and tested to measure knowledge of dementia and AD. These instruments have different psychometric characteristics and have been validated among different populations [12,21,22]. According to the year of publication, some of the most relevant questionnaires are the following: the Alzheimer's Disease Knowledge Test (ADKT) [23]; the Alzheimer's Disease Knowledge Scale (ADKS) [11]; the Dementia Knowledge Assessment Tool Version 2 (DKAT2) [24]; the Dementia Knowledge Assessment Scale (DKAS) [10,16]. All these questionnaires have been developed and tested in English-speaking countries. Searching the literature, we have found no questionnaire on dementia knowledge developed or culturally adapted to Spanish-speaking countries. There are differences in both the culture and health context between Anglo-Saxon countries and Spain and other Spanish-speaking countries. This is the reason that led us to develop and validate a questionnaire specifically targeting Spanish-speaking countries based on recommendations of both international and Spanish guidelines on dementia care. The new developed questionnaire should be focused on nursing care for people with Alzheimer's disease, covering a range of topics including palliative care. The aim of this study was to develop and test a scale measuring the knowledge about dementia and AD, useful for nursing professionals and nursing students.

Design
The research constituted a cross-sectional survey study to develop and validate a new scale. Both classic measurement theory [25] and item response theory [26,27] were used for measuring the psychometric properties. The research included three phases, as follows: 1. Development of the questionnaire and item wording. 2. Content validation by an expert panel. 3. Questionnaire testing through a survey in two samples to establish psychometric properties.
The period of data collection was from November 2016 to January 2017.

Questionnaire development
Before writing the items, a literature search was performed to identify CPGs on dementia and AD care published or updated since 2010. In all, eight CPGs in English or Spanish were recovered. The recommendations included in the CPGs were extracted and grouped by topic and duplications were eliminated. Recommendations (n = 566) focused on care or nursing interventions were selected and used to write the items (n = 51) of the questionnaire.

Content validation
A panel of 15 experts in dementia care (5 nurses working in universities, 9 clinical nurses working in nursing homes or in dementia care specialised centres and 1 general practitioner) was used for content validation. All of them have more than 10 years of professional experience. Each expert rated the relevance of the items with a 5 point scale (1 No relevant to 5 Very relevant). The V-Aiken index [28] was calculated to establish the consensus among the experts, using the value of 0.80 as threshold for retaining the items. Three consecutive rounds were conducted, first round yielded 48 recommendations selected from which 51 items were written for the questionnaire; second round yielded 22 items accepted and 10 items were modified and moved on to the third round. After the whole process, we produced an initial version of the scale with 30 items. Upon attaining expert consensus, the questionnaire was tested for comprehension among a small sample of health professionals.

Psychometric testing: Sampling
The questionnaire was tested among two different samples. Sample 1 comprised nursing professionals, i.e. Registered Nurses ([RNs], 4-year university degree), Assistant Nurses ([ANs], 2-year diploma) and eldercare workers (1-year technical education), working in 24 nursing homes in the province of Jaen (southern Spain). Sample 2 comprised nursing students (2nd, 3rd and 4th year) from the Faculty of Health Sciences of the University of Jaen (Spain). The sample size in each case was estimated at 300 (for the 30-item version of the questionnaire), according to the methodological recommendations for the validation of questionnaires, namely 5-10 individuals per item [29].

Instruments
Three instruments were used to collect the data: a questionnaire gathering participants' demographic data; a questionnaire concerning knowledge of care for people with AD and other dementias (newly developed); the Spanish version of the DKAT2.
Questionnaire concerning knowledge of care for people with Alzheimer's disease and other dementias The initial 30-item version of the questionnaire obtained after content validation and pilot testing was used. For each item there were three response options: "Yes", "No" and "I don't know". For some items "Yes" was the correct response and for others it was "No". The "I don't know" option was included to allow respondents to acknowledge ignorance. The questionnaire was selfadministered.

Dementia knowledge assessment tool 2 (DKAT2)
A Spanish version of the DKAT2 questionnaire was used to test the convergent validity of the new questionnaire. The original DKAT2 instrument, with 21 items, has good internal consistency (Cronbach's alpha = 0.79) and three response options: "Yes", "No" and "I don't know". The questionnaire was developed and tested in Australia [24]. Permission to use and translate this questionnaire was obtained from the authors. The Spanish-adapted version was developed by translation and back-translation undertaken by two independent translators; the back-translated version was revised by one of the authors of the original tool (Christine Toye). The Spanish version of DKAT2 has adequate internal consistency (nursing professionals: Cronbach's alpha = 0.76; students: Cronbach's alpha = 0.83).

Data collection procedure
To access nursing home staff, the directors of all nursing homes in the province of Jaen (public and private management) were contacted and asked to participate in the study. Only two centres refused to participate. After approval, each centre received the appropriate number of questionnaires according to the staff numbers. Completed questionnaires were collected over approximately two weeks. A small sub-sample of nursing home staff completed the questionnaires twice (over a three-week interval) in order to do a test-retest. For nursing students, lecturers were informed of the aims of the study and approval was obtained to administer the questionnaires to students in the classroom.

Ethics
The Committee of Research Ethics of Jaen approved the study. The anonymity and confidentiality of the data were guaranteed in accordance with the Spanish Law of Data Protection.

Data analysis
Data were tabulated, coded and cleaned in a spreadsheet before the analysis. For each scale, the total score was calculated by adding the number of items with the correct answer. Several methods were used for the analysis, namely item analysis, Rasch modelling and testing validity and reliability.

Item analysis
The analysis was conducted for the two samples, but the values obtained from the sample of nursing professionals were used to make decisions concerning the retention or elimination of items. Three indices were calculated for each item: the difficulty index (DF: percentage of correct answers), the ignorance index (percentage of "I don't know" answers) and the discrimination index (difference between the percentage of correct answers of the 27% of the questionnaires with the highest overall score minus the 27% of the questionnaires with the lowest overall score) [29,30]. Items with a discrimination index lower than 10% were eliminated for analysis, as these items were not useful for distinguishing between participants with a high or low knowledge. According to the DF, items were classified into six categories: very easy (> 90% correct answers); easy (75.1-90%); somewhat easy (50.1-75%); somewhat difficult (25.1-50%); difficult (10.1-25%) and very difficult (< 10%).

Rasch model
The version of the questionnaire produced after the item analysis was further analysed using a Rasch model. The Rasch model is a method based on item response theory, undertaken for the psychometric validation of questionnaires; it allows differentiation between information concerning the items and persons' performance. Rasch analysis was conducted in JMetrik software using the joint maximum likelihood method [31]. Fit statistics were calculated: unweighted mean square of standardized residuals (outfit) and weighted mean square of standardized residuals (infit). The value 1 denotes a perfect fit; outfit and infit values between 0.5 and 1.5 were deemed acceptable [32]. The statistic Yen's Q3 was used to test the local independence between items [33]. Finally, we conducted two differential item functioning (DIF) analyses, the first between nursing professionals and nursing students and the second between RNs and ANs. Effect sizes (common Odd Ratio) and 95% confidence intervals were estimated, considering that an item has no DIF when the common OR is between 0.65 and 1.53; has a large DIF when common OR < 0.53 or > 1.89; and has a low DIF when common OR range between 0.53-0.65 or 1.53-1.89 [34].

Validity
We tested both convergent criterion validity and construct validity. For convergent criterion validity we compared the newly developed questionnaire with the DKAT2 questionnaire in the Spanish version (as the gold standard). The intraclass correlation coefficient (ICC) was used as the statistic for concordance estimation. Also, we used the Bland-Altman graphic method, which shows the plot of difference between the two measurements versus the average of the two scores [35]. Construct validity was tested through exploratory factorial analysis (EFA) with principal axis factorization as the extraction method and different rotations (varimax, quartimax and oblimin). The method of Parallel analysis with Horn's criteria was used to calculate the number of factors to extract in the EFA [36,37]. Each model was tested against the theoretical model that has guided the construction of the questionnaire.
Furthermore, we tested a hypothesis in known groups (groups with expected high and low knowledge of dementia care), i.e.: -Nursing professionals versus students.
-RNs versus ANs or eldercare workers.
-Staff who had attended specific courses versus those who had not.
-Students with some care experience versus those without.
The adjustment of the knowledge score data to the normal distribution was tested. Comparisons of mean knowledge scores were undertaken using Student's t test or one-way ANOVA.

Reliability
The internal consistency of the questionnaire was estimated using Cronbach's alpha and the ICC statistics. Temporal stability (test-retest) was measured using the ICC between the first and second administration of the questionnaire.

Characteristics of participants
Sample 1 comprised 361 members of nursing staff from the 24 nursing homes (response rate 51.5%). Staff included RNs, ANs and eldercare workers. Sample 2 comprised 297 students enrolled in the 2nd, 3rd and 4th years of a nursing degree in the Faculty of Health Sciences at the University of Jaen (response rate 67.3%). Table 1 shows the main characteristics of these samples.

Item analysis
Difficulty, ignorance and discrimination indices were estimated from the 30-item version of the questionnaire. According to the percentage of correct answers (difficulty index) there were 13 items that were very easy, 5 items that were easy, 8 items that were somewhat easy and 4 items that were somewhat difficult. No items were difficult or very difficult. Six items presented a discrimination index lower than 10%, so they were removed. An additional item had a discrimination index of 15.6%, but included more than one concept, so it was also removed. This 23-item version of the questionnaire was used for analysis and named the UJA Alzheimer's Care Scale (Table 2). Table 3 shows the fit values for the 23-item UJA Alzheimer's Care Scale (outfit and infit). All the items have very good fit (WMS and UWMS values lower than 1.20). The difficulty values range from − 2.12 for the easiest item (item 14: "Inform the caregiver about the disease and its possible complications, and the social resources and support systems available") to 2.43 for the most difficult (item 23: "Use a nasogastric tube or percutaneous gastrostomy in the patient with advance dementia as a regular feeding route if dysphagia"). Two analyses of differential item functioning were conducted, one between professionals and students and the other between ANs and RNs. This analysis establishes if all the items perform equally well in the different populations in which the questionnaire is intended to be used. The results are shown in Table 4. Between RNs and ANs only one item (item 2-Use the Zarit scale to quantify the caregiver's burden) has a significant amount of DIF, favouring the RN group. Between nursing professionals and students there are five items with DIF. Two of these favour the group of students: item 2-Use the Zarit scale to quantify the caregiver's burden) and item 5-Associate non-pharmacological and pharmacological measures to manage the different behavioural and psychological symptoms of dementia. Three favour the group of nursing staff: item 4-Provide a normal diet while assessing the causes of dysphagia; item 7-The management of extreme agitation, violence and aggressiveness must take place in a safe, low stimulation environment, separate from other users of the service; item 12-Use the oral route for fluid supply at the end of life, whenever possible. Because the amount of DIF was low in these 6 items, we decided to retain them in the questionnaire.

Convergent criterion validity
The concordance of the scores obtained with the UJA Alzheimer's Care Scale and the DKAT2 Spanish version is shown in Table 5. The ICC ranges from 0.56 in the AN group to 0.79 for the student group.
The Bland-Altman plot also show good concordance, with most of the values within the 95% confidence interval (Fig. 1).

Construct validity
Parallel analysis initially yielded three factors to extract. We tested different exploratory factor analysis models with main axis factorization extraction using one to four factors and several rotation methods (varimax, quartimax and oblimin). None of these models with more than one factor showed high values of explained variance, nor did they exhibit an adequate fit to a theoretical model of the latent variable measured by this questionnaire. Thus, the UJA Alzheimer's Care Scale is proposed as a one-dimensional scale.
We hypothesized that the scores obtained with the UJA Alzheimer's Care Scale would be higher in some groups with a priori greater knowledge of AD. Score data were normally distributed, so parametric tests were used. The test for means comparisons shown in Table 6 confirm this hypothesis. The scale is able to differentiate correctly between people with a high and low knowledge.

Reliability
The internal consistency of the 23-item UJA Alzheimer's Care Scale was Cronbach's alpha = 0.70 for nursing staff and 0.82 for students. There was no item that could increase the value of Cronbach's alpha if deleted.

Temporal stability
A sub-sample of 21 nursing staff members from nursing homes completed the scale twice over a three-week interval (test-retest). The ICC between the two administrations was 0.84 (95% CI = 0.60-0.93).

Discussion
This research has focused on the development and validation of a new tool to measure knowledge of care  3. Cuando la familia no puede garantizar la atención a la persona con demencia, su ingreso en una institución evita el aislamiento social y previene el maltrato.  20. Los programas de intervención sobre las actividades de la vida diaria no disminuyen la sobrecarga del cuidador a medio plazo.   It is important to highlight that this scale has three response options, including the "I don't know" option. Several authors have stated that having three response options is better than two options (Yes, No) in knowledge scales because respondents can acknowledge their ignorance concerning a specific theme, avoiding the need to mark "Yes" or "No" at random [39,40]. Our results concur with those of other studies in that internal consistency increases with the inclusion of the third response option in the knowledge scale [41]. This feature of the scale allows the identification of those topics on which staff or students have erroneous knowledge (errors) vs topics of which respondents recognize having no knowledge. We think that this is useful information when planning educational activities.
The newly developed UJA Alzheimer's Care Scale contains items exploring person-centred care, like the DKAS [10], as opposed to other scales focusing predominantly on biomedical issues [11,21,23,42].
Our scale has a robust building process, starting from CPGs and including the collaboration of a large panel of 15 experts. Having a panel with a sufficient number of experts is an important point for content validation when developing scales. Most of the previously developed AD knowledge questionnaires have had between 4 and 10 experts [24,[42][43][44] or the authors acting as experts [11]. Only the DKAS, with 18 experts, had a higher number than ours [10].
Our purpose was to develop a final version of this scale able to measure the latent variable "Knowledge of AD and dementia care" among a wide range of nursing professionals, so we needed to include items with different levels of difficulty. Through the item analysis, some of the easy or very easy items were deleted because they made no contribution to the discriminant capacity of the scale. Analysis with the Rasch model confirmed that the 23-item final version has a good fit and has items with different levels of difficulty. The scale works well with both nursing staff from nursing homes and nursing students, although there are five items that present differential functioning in professionals and in student, namely: the caregiver burden; nutritional care; behavioural symptom management; agitation and aggressiveness management.
The UJA Alzheimer's Care Scale is a one-dimensional scale that produces a total score to measure the amount of knowledge concerning AD, just by adding up the number of correct answers. Although the process of writing the items started by grouping them into several categories according to the CPG recommendations (such as Basic care; Caregivers; Diagnosis; Behavioural symptoms; End of life decisions), when the scale is used by nursing staff or students, there are no data to support an internal structure with sub-scales or factors. We consider that this one-dimensional structure is an advantage in its use, because it has just one score and is easy to use. Also, it is comparable with studies that have used most dementia knowledge tools, such as DKAT2 [24] or ADKS [11].
Overall, the new scale has good evidence of validity. Convergent validity is strong when it is compared with the DKAT2 scale, which has been well validated and widely used in recent research [15,24,45]. Our research also provides evidence of construct validity for the UJA Alzheimer's Care Scale. This scale is able to discriminate between people with different levels of knowledge [21]. Our data agree with some other validation studies for the ADKS [11], KAML-C [43], UAB-ADKT [21], DK-20 [44], DKAT2 [24] and DKAS [10].
This new scale also has evidence of reliability, high internal consistency with values similar to other scales for nurses, DKAT2 [24]; ADKS [11]; AUB-ADKT [21]; also for nursing students, DQ [42], DKAS [10] and KAML-C [43]. The measures of AD knowledge produced by this new scale appear to be stable over time. Only a few validation studies have conducted a test-retest -ADKS [11], AUB-ADKT [21] and DK-20 [44] but our data are consistent with most of them. Overall, our research provides data to support the reliability of the UJA Alzheimer's Care Scale for use by both nursing staff and students.

Limitations and strengths
This research has several limitations that should be considered. Firstly, the sampling was not random and some nursing homes had a low response rate. A disproportionate number of staff with more knowledge or motivation may have completed the survey, so this could lead to an overestimation in the results for AD knowledge. Second, the sample of nursing students was obtained from only one centre, so it is not possible to generalize these results.
As strengths, we can mention the rigorous and robust process in the development and content validation of this scale. Also, we would highlight the large number of nursing homes (up to 24) that participated in the study. There were centres with different characteristics in terms of size, location and type of management. Being a multi-centre study increases the representativeness of the results.
The UJA Alzheimer's Care Scale has shown sufficient evidence of validity and reliability in this initial validation study to warrant further research on its use. For future studies, it will be necessary to test this scale with other samples of nurses working in hospitals or primary care. In addition, it should be used with a large sample of nursing students from different universities to check its performance. Finally, we think that the scale could be used with family caregivers who care for people with dementia as a way of identifying their formative needs.