- Research article
- Open Access
Process evaluation of a tailored intervention to Reduce Inappropriate psychotropic Drug use in nursing home residents with dementia
BMC Geriatrics volume 21, Article number: 414 (2021)
Research suggests that collaborative and tailored approaches with external expertise are important to process implementations. We therefore performed a process evaluation of an intervention using participatory action research, tailored information provision, and external coaching to reduce inappropriate psychotropic drug use among nursing home residents with dementia. The process evaluation was conducted alongside a randomized controlled trial assessing the utility of this approach.
We used Leontjevas’ model of process evaluation to guide data collection and analysis, focusing on the relevance and feasibility, extent of performance, and barriers and facilitators to implementation. Data on the relevance and feasibility and on the extent of performance were collected using a questionnaire targeting internal project leaders at nursing homes and our external coaches. Implementation barriers and facilitators were identified by individual semi-structured interviews. The Consolidated Framework for Implementation Research was used to structure and describe the identified barriers and facilitators.
The intervention was viewed positively, but it was also considered time consuming due to the involvement of many people and designing a tailored action and implementation plan was viewed as complex. The extent of performance differed between nursing homes. Delays in implementation and suboptimal execution of actions may have reduced effectiveness of the RID intervention in some nursing homes. Barriers to implementation were reorganizations, staff turnover, communication issues, unclear expectations, and perceived time pressures. Implementation also depended on the involvement and skills of key stakeholders, and organizations’ readiness to change. Although external coaches stimulated implementation, their additional value was rated variably across organizations.
Barriers to implementation occurred on several levels and some barriers appear to be inherent to the nursing home environment and could be points of leverage of future implementation trajectories. This underlines the importance of assessing and supporting organizations in their readiness to change. Sensitivity analyses, taking into account the week in which nursing homes started with implementation and the degree to which actions were implemented as intended, will be appropriate in the effect analyses of the trial.
Neuropsychiatric symptoms (NPS) are common in nursing home (NH) residents with dementia. Over 80% exhibits NPS such as agitation and apathy . NPS are often treated with psychotropic drugs (PDs), including antipsychotics, hypnotics or sedatives, anxiolytics, antidepressants, anticonvulsants, and anti-dementia drugs [2,3,4,5]. However, PDs are associated with significant side effects. Antipsychotics may increase the risk of stroke and mortality [2, 6]. They are also associated with extrapyramidal symptoms and drowsiness . The use of sedatives, hypnotics, antidepressants, and benzodiazepines is associated with falls . In addition, there is evidence that PDs are of limited effectiveness on NPS in residents with dementia [9, 10], especially when used in the long-term . Despite these concerns, about 60% of NH residents uses PDs for NPS and only 10% of PDs are prescribed fully appropriate, with indication, evaluation and therapy duration contributing to inappropriate use . Consequently, the appropriateness of prescribing should be optimized. Moreover, clinical guidelines recommend restricted use of PDs and propose non-pharmacological interventions as first-line treatment for managing NPS [13, 14]. As a result, various interventions have been developed over the years, aimed at reducing inappropriate prescribing and/or targeting a greater use of non-pharmacological interventions in practice [15,16,17,18,19]. Often, these interventions comprise complex, multicomponent interventions . However, the strength of the effects varies for complex and multicomponent interventions to reduce inappropriate psychotropic drug prescribing or to increase the use of non-pharmacological interventions among nursing home (NH) residents with dementia. Although complex interventions can be effective , to date they have tended to have relatively small or lacking effects [16, 22, 23], with suboptimal implementation emerging as a prime reason. Process evaluations in NHs have demonstrated that suboptimal implementation results from barriers [24,25,26], not least of which are skepticism about using non-pharmacological approaches . From a broader healthcare perspective, implementation problems relate to perceptions that the issue is not a priority , and the use of standardized “one size fits all” solutions . Complex healthcare interventions may work best if tailored to local circumstances rather than being standardized [20, 28], especially if they identify and target modifiable barriers to change before implementation [28, 29]. Consistent with this, process evaluations of complex interventions among NH residents with dementia have underlined that we must adapt to the specific needs and features of each care organization [24, 26]. It also appears that collaborative approaches that introduce external expertise can address the concerns and problems faced by NH staff, while ensuring awareness of their preferences and increasing awareness .
This information was taken into account when designing the Reducing Inappropriate Psychotropic Drug Use in NH Residents with Dementia (RID) study . We hypothesized that implementation would be facilitated if NH staff were actively involved in determining the problem(s) and potential solutions, if interventions could be tailored to the local setting, and if the implementation was guided by a coach. To resolve the challenges of existing strategies, we developed a RID intervention that incorporated three active elements : participatory action research (PAR), which allowed staff to formulate problems and potential solutions concerning inappropriate psychotropic drug use (PDU) and neuropsychiatric symptom (NPS) management ; tailored information provision about inappropriate PDU and NPS management; and  external coaching. It was anticipated that these active elements would lead to the implementation of a tailored action and implementation plan (AIP) to reduce inappropriate PDU for NH residents with dementia . Including a process evaluation can then provide insight into the true contribution of the intervention to daily practice, helping us to understand why the intervention was successful or unsuccessful, and indeed, how it could be optimized . Information on the relevance and feasibility of a given intervention, as well as the extent of performance, is essential for the credibility of research. For example, difficulties with implementation may result in low treatment fidelity, meaning that the intervention could not be carried out as intended, which in turn may lead to a loss of effect .
In this research, we aimed to present the process evaluation of the RID study. To our knowledge, no other researchers have evaluated the implementation of such a complex and tailored intervention in NHs. Given its three active elements, we hypothesize that the degree of implementation of the intervention will be good.
Alongside an effect study, executed between July 2016 and December 2018, a process evaluation was conducted between March 2018 and January 2019. The study was performed in sixteen Dutch NHs caring for residents with dementia who reside in Dementia Special Care Units (DSCU). DSCUs designed to deliver care for residents with Korsakov syndrome, acquired brain injury, Down syndrome, or young-onset dementia were excluded. In- and exclusion criteria were not proposed for the internal project leader, although the tasks and aspects associated with this role were communicated with the NHs (such as creating support, logistics management and provision of information). The external coaches needed to be knowledgeable about dementia and have previous consultation expertise in NHs. More information about recruitment and in- and exclusion criteria can be found in our protocol article . The effect study constituted a two-armed, stepped wedge, cluster randomized controlled trial (RCT) . The data collection period was 16 months per NH, split into two 8-month periods. In the first period, eight NHs started with the RID intervention and another eight NHs deferred the intervention. In the second period, the NHs that started with the intervention continued with implementation of the AIP and the other eight NHs started the intervention. Thus, we had an intervention group and a deferred intervention group, with measurements performed at 0, 8, and 16 months. Participation was on a voluntary basis. Besides the external coaching being offered freely to participating NHs, no financial incentives or additional external influences were provided. Further information about the RID study is provided elsewhere . Since information from the process evaluation can be incorporated in the effect analyses , we performed the process evaluation before conducting the effect analyses of the RCT.
We used Leontjevas’ model to guide the collection and evaluation of process data . In the process evaluation, we investigated whether our RID intervention with the active elements PAR, tailored information provision, and external coaching successfully addressed the problems identified with earlier studies. Specifically, we studied two elements : the intervention quality, consisting of the relevance and feasibility of the RID intervention and the extent of performance of the RID intervention (degree of implementation); and  the barriers and facilitators to implementation.
The RID study was registered with the Netherlands Trial Registry (NTR5872) on May 27, 2016, https://www.trialregister.nl/trial/5719. We complied with the Consolidated Standards of Reporting Trials (CONSORT) guidelines in conducting and reporting this study .
The RID study examined the effectiveness of an intervention to change practice through cooperation between practice and research, using PAR, external coaching, and tailored information provision to implement tailored AIPs to reduce inappropriate PDU. A multidisciplinary project team (MPT) in each NH (including an internal project leader), an external coach, and researchers, were given specific tasks within the cyclic intervention.
Organizing stakeholder efforts:
Researchers organized a kick-off meeting in the NH.
NHs formed an MPT consisting of at least nursing staff, psychologist(s), physician(s), and an internal project leader. The MPT preferably included stakeholders, such as management and representatives of the residents. Each MPT was supported during their intervention period by an external coach.
Throughout the process, the MPT and external coach had several meetings (total number was not pre-defined).
Researchers carried out a problem analysis (both quantitative and qualitative data) using interviews and questionnaires. Problems (as perceived by NH staff) regarding inappropriate PDU and NPS management were examined (observation phase).
Researchers presented the results of the tailored problem analysis to the MPT and external coach, which was followed by interpretation and reflection in the context of the local NH (reflection phase).
Designing tailored AIP
The MPT and external coach created an AIP that matched the identified problems (planning phase).
The external coach and researchers provided feedback on the AIP (relevance and feasibility of actions, concreteness).
Implementation of tailored AIP
The MPT started by implementing the tailored AIP (action phase).
Researchers carried out an interim measurement on inappropriate PDU. The eight MPTs that started as the intervention group were given interim results at 8 months (observation phase).
The external coach and MPT discussed and reflected on the interim results (reflection phase).
Adjustments to tailored AIP
The MPT was able to adjust the AIP based on the interim results (planning phase) and implement any changes during the second period (action phase).
Providing the final results
Researchers carried out a final measurement with respect to inappropriate PDU and provided the MPT with their final results after 16 months.
Intervention quality and barriers and facilitators to implementation
Table 1, supplemented by Additional File 1, provides an overview of the operationalization of the intervention quality and of the barriers and facilitators to implementation. We examined the quality of the RID intervention. For the relevance and feasibility of the RID intervention, the role of each (group of) stakeholder(s) was evaluated (i.e., researchers, internal project leaders, MPTs, and external coaches). For the extent of performance, each intervention task was evaluated with respect to whether implementation was as intended. We then evaluated the barriers and facilitators to implementation. We did not evaluate the relevance and feasibility of the AIPs. Given that the RID intervention focused on PAR, tailored information provision, and external coaching, we decided that evaluating the relevance and feasibility of each of the self-created and highly variable actions would be impractical and offer limited information in the confines of this study. However, we did examine the extent of performance on the AIP tasks because of their expected impact on the overall effectiveness of the RID intervention.
After the final measurement, the researcher (CGK) sent a web-based questionnaire to the internal project leader and external coach at each NH. Approximately 1 or 2 weeks after completion of the questionnaire, the same researcher (CGK) held individual semi-structured telephone interviews with the internal project leader and external coach. The information derived from the questionnaires was explored in-depth during the interviews, to gain a more thorough understanding. The duration of an interview was approximately 1 h. As with the interviews, the questionnaires were completed individually at a time that was convenient for them. External coaches evaluated the process for each NH separately. Relevance and feasibility were examined using both sources, while the extent of performance was based only on the questionnaire. Given that our study was designed to consider the barriers and facilitators identified in previous studies, we explored this matter in depth based on data from the telephone interviews. The telephone interviews were audio recorded and transcribed verbatim.
Quantitative data were analyzed with IBM SPSS version 25 (IBM Corp., Armonk, NY, USA), using descriptive statistics. Qualitative data were analyzed by deductive content analysis , using the Consolidated Framework for Implementation Research (CFIR) to structure and describe the barriers and facilitators . Two researchers independently coded the data (CGK and DG), with another three authors available for discussion in case of disagreement (CVT, MS, SZ). The process evaluation focused on a general evaluation of the implementation process, but we also examined differences in the extent of performance among NHs.
The respondent characteristics are summarized in Table 2. All 16 internal project leaders participated in the process evaluation, although one completed the questionnaire partially and one did not respond to the request for an interview. All six external coaches also participated in the process evaluation, and a completed digital questionnaire was received for each NH; however, due to a change of jobs, one external coach could not give an interview for one of the NHs. Therefore, we have data for 31 questionnaires and 30 interviews (response rates of 97 and 94%, respectively). The majority of respondents is female (over 80%). The respondents have varying educational backgrounds and current positions (see Table 2).
Intervention quality: relevance and feasibility of the RID intervention
The results for the quantitative evaluation of the contributions by researchers, internal project leaders, MPTs, and external coach are summarized in Table 3. The majority of the respondents perceived the tailored information provision to be of high added value. External coaching was perceived to be of high added value by the majority of the internal project leaders, whereas the majority of the external coaches indicated that coaching was of added value ‘to a reasonable extent’. The majority of the external coaches indicated that coaching is a necessity for (continued) implementation, while the majority of the internal project leaders indicated this is not a necessity. Both the majority of the internal project leaders as well as the external coaches perceived the other party to be competent or very competent. The qualitative evaluation is provided below.
The problem analysis was often perceived to create urgency for change because researchers provided MPTs with information about inappropriate PDU and NPS management. This initiated a dialog and resulted in NHs comparing themselves to other organizations. The fixed measures motived NHs to achieve change, and the provision of interim results not only provided valuable insights into their progression but also provided an opportunity to make changes. Although the ability to tailor action and implementation to each organization was evaluated positively, the process was considered both time consuming and complex.
Internal project leader and MPT
The role of the internal project leader was considered essential, but this was highly dependent on their skills, such as creating support, engagement, and informing staff. The MPT was also considered relevant because of the multidisciplinary nature of managing NPS and inappropriate PDU. Actively involving the MPT in formulating the problems and solutions was positively evaluated:
Internal project leader: “The actions are self-created, which creates greater support. You can impose all sorts of things but that won't work. It really has to come from themselves, what they think might work.”
External coaching was considered especially useful for translating the problem analysis to a tailored AIP, with many stating that this process was difficult. However, although many respondents thought that external coaching was necessary, others considered it relevant but non-essential. A positive appraisal is illustrated by the following quote:
External coach: “They had project groups and those people were quite driven to get started. They definitely had a clear direction... and they acted on this as well.”
Intervention quality: extent of performance
Results for the RID intervention’s extent of performance are depicted in Table 4.
Organizing efforts of stakeholders
Researchers: As intended, the researchers carried out a kick-off meeting in each NH.
External coaches: The number of meetings (Table 4, columns 1b and 1c) with the external coach and the MPT on location varied substantially between NHs (range 5–13), as did the number of (phone) meetings between the external coach and internal project leader (range 0–12).
The researchers carried out a problem analysis at each NH and presented the results to all MPTs.
Designing tailored AIP
In NH 11, communication issues between the person who decided to participate in the project and the persons responsible for executing it delayed the process to the extent that no AIP was created (Table 4, column 3a). The AIPs otherwise contained the following actions: multidisciplinary and methodical working (including use of person-centered interventions), education and training, and adaptations to the living environment. Generally, actions in the AIPs addressed the identified problems. Apart from NH 11, which lacked an AIP, all NHs were given feedback on their AIP by the external coaches and researchers. The contributions of the external coach, internal project leader, and MPT in designing the AIP were large (Table 4, column 3b). Also, apart from NH 11, all NHs adjusted their AIP based on the feedback given by the external coach and researchers (Table 4, section 3c).
Implementation of tailored AIP
Only NHs 1, 4, and 13 did not need more than the allocated 8 weeks to start AIP implementation, and NHs 3 and 11 only started implementation after 16 weeks (Table 4, column 4a). The mean implementation scores for executing the AIPs on a 10-point scale (range, 3.4–8.5) were below 6.0 in NHs 3, 10, 11, and 15 (Table 4, column 4b).
The researchers carried out an interim measurements as intended, and the relevant MPTs were given interim results at 8 months.
All external coaches discussed the interim results within each MPT.
Adjustments to tailored AIP
Most NHs adjusted their AIP after discussing the interim results. Changes mostly focused on not only what was important to keep doing or what was likely to succeed but also defining actions and strategies more precisely. Two NHs (no. 1, 6) had positive results and decided to continue with the original AIP, and one NH (no. 3) did not change the AIP despite negative interim results. In the latter case, the MPT argued that further change was unwise because they had barely started to implement the original AIP (Table 4, column 7).
Providing final results
The researchers carried out a final measurement at each NH and presented all MPTs with the final results.
Barriers and facilitators to implementation
The barriers and facilitators identified from the interviews could be categorized into three of the CFIR themes: intervention characteristics, inner setting, and process . Description of the CFIR topics are given in Additional File 2.
Involving NH staff in addressing the problems, needs, and solutions created the engagement and support needed for implementation, but involving so many people also seemed to slow the process. This was key to why some perceived the planned implementation period as too short. Several NHs in the deferred intervention group indicated that the amount of time between project registration and action was too long, which reduced their enthusiasm. It was also perceived that NH staff sometimes struggled to translate knowledge into practice after education or training, but on-the-job coaching was considered to be helpful in applying what was learned into practice.
Several barriers were common, such as reorganizations, staff shortages and turnover, and communication issues within and between disciplines (i.e., too little contact, criticizing each other, or not being receptive to feedback). Another perceived barrier was the use of self-directed teams that had responsibilities and duties assigned to teams without a formal lead. Some NHs embraced change whereas others seemed more reluctant; for example, it was observed that some MPT members questioned every suggestion or assumed that colleagues would not keep to the agreements made. Limited self-reflection was also mentioned, with respondents indicating that MPT members and NH staff sometimes found it hard to accept that the level of PDU in their NH was high, despite feedback and evidence to the contrary. Time pressure interfered with implementation in other instances, and in some cases, the NH management did not grant their staff the time needed to complete the project; however, several respondents indicated that the issue of time constraints, whether perceived or real, was about setting priorities. Implementation was facilitated when NHs developed a view or vision on PDU with sufficient alternatives, because it allowed them to build on a basic level and focus on finetuning agreements. Levels of innovative power were different among NHs, and it appeared that those used to participating in research had an easier time with implementation. It was also easier to implement the RID intervention when it could be integrated with other projects, but this was at the cost of the multiple projects placing excess demands on staff.
The involvement of stakeholders, such as internal project leaders, physicians, and psychologists, differed among the NHs, but if these key persons could continue their efforts, staff turnover did not negatively impact the project. Notably, arranging for proper transition facilitated implementation when there was turnover in these positions. Active participation by management also conveyed to the MPT that the project was important. In some cases, it also seemed that the internal project leaders lack the skills and personality for their role. In fact, a change in the internal project leader was not a barrier when that person was replaced by someone who was better suited to the role.
External coach: “She read all information about the project, was well prepared, and brought structure; she worked according to a fixed agenda, with notes and action lists. She asked about intrinsic motivation (why are you in the project team) and held them accountable … She was decisive and sought connection with relevant parties such as the policy advisor and manager.”
External coaches mentioned that there were variations between NHs, requiring that they customized their approaches to each NH. Often external coaches were perceived to facilitate implementation by providing structure and reflection, and in some instances, they “scaled up,” examined the internal dynamics, and tried to create engagement by addressing the need for change and the relative advantage. Nevertheless, staff in some NHs were perceived to remain reluctant despite these strategies. An issue was that the expectations of the project were not always in line with what was communicated. It was sometimes expected (mostly by internal project leaders, but also by a few external coaches) that the results of the problem analysis would yield manageable and directly applicable information, without the need for reflection and translation by the MPT and external coach (e.g., “what does this information mean?” and “what do we want to do?”). Also, some MPT members failed to engage in the project either because the external coach was treated as the main carrier of the project or because MPTs were not open to being coached.
External coach: “I can be very facilitating, and I can be a guide, but the organization must act itself. I can't tell people what to do ... I can only advise ‘it is smart to do this’ or ‘you can choose from this and this and choose for yourself what fits your organization’ best.”
Although MPT members were generally very enthusiastic, this did not guarantee results because the ability to move forward was sometimes perceived as limited. According to respondents, overestimating the ease of implementing the innovations led some NHs to create AIPs that included either non-specific or excessive actions that could result in an unclear division of roles and responsibilities. Despite researcher feedback in which these concerns were stressed, no changes were made by the relevant MPTs.
In general, the RID intervention with PAR, external coaching, and tailored information provision was evaluated positively. The local problem analysis, coupled with the presence of external coaches and researchers, often generated an impetus for action. However, the overall process was complex and time consuming from the problem analysis to the design of a tailored AIP, with external coaching being key during this transition. Coaching also seemed to have an empowering effect, with some NHs even considering it a prerequisite for ongoing implementation. Nevertheless, important issues were that the set time period of 8 weeks for problem analysis and AIP design was too short and that the extent of performance was suboptimal, with several differences emerging among the NHs. For example, 4 of the 16 NHs had short implementation periods because of delays. This might have resulted in limited execution of key actions that may have reduced the effectiveness of the RID intervention in these NHs.
Actively involving NH staff throughout the implementation process was deemed essential by respondents. This is consistent with existing literature showing that bottom-up approaches that include local stakeholders are key to gaining support for, and the adoption of, a given intervention . A potential downside of PAR is that actively involving staff in the whole process meant that it was often impossible to complete implementation within 6 months, because in practice, the process can be very time intensive. Staff turnover and reorganizations further complicated implementation, which is again consistent with other findings [24, 25]. To date, these contextual/environmental characteristics have been viewed as confounders or barriers, but it might be more appropriate to accept them as normal conditions into which interventions must be integrated . In our view, placing preconditions on organizations before they can participate (e.g., requiring stability) is unrealistic, and we must instead better account for discontinuity due to staff turnover or reorganization during implementation processes. As such, because the issue of staff turnover cannot be resolved, the challenge lies in learning to implement a new strategy in a changing context. Our results suggest that the negative impact of such change can be mitigated if communication and takeover are handled well.
The extent to which NH staff are willing and able to implement nonpharmacological strategies is important, considering that many nonpharmacological strategies depend upon implementation by NH staff . Earlier research has concluded that the readiness of staff for change must be considered during implementation planning . Similarly, we found that differences existed in the extent to which NH staff were open to change. For example, time pressures often were used as an argument for impeded implementation. Although this common barrier has been mentioned in other studies [42,43,44], some of our external coaches noted that time pressures were often an issue of perception and stressed that the true problem was about setting priorities. Although literature illustrates that dealing with NPS is indeed a priority in long term care, taking into account that NPS can result in distress amongst nursing staff [45,46,47], the uptake of nonpharmacological treatments in daily practice is still limited. Nonetheless, the results of our study indicate that despite discussing the relevance of change (i.e., the degree of inappropriate PDU in the NH) and the added value of intervening, some NHs remained reluctant to change. In these instances, creating engagement based on content and reflection with an external coach could therefore have been ineffective. Nevertheless, assessing and supporting an organization’s readiness for change might facilitate successful implementation. This is in line with Pimental et al. stressing that an organization’s readiness for change is essential and is a function of organizational members’ shared commitment to implementing change and a shared belief in their collective capability to do so .
Communication issues impeded implementation in our study, which is again consistent with previous research [24, 26]. Notably, our problem analysis revealed that many NHs struggled with there being little interdisciplinary contact. Although staff in some NHs recognized this as a point for improvement, budget cuts meant that NH management were often unwilling to invest in the AIP. Also, some MPTs had expectations of the external coaches and researchers that were too high, underlining a need to communicate what they can expect more clearly and to empower teams to feel confident in taking action themselves.
We confirmed two findings of our recently published systematic review on the barriers and facilitators of complex interventions for residents with dementia in long-term care . First, limited skills of internal project leaders impeded implementation in some cases. As suggested in our earlier review, greater care may be needed to ensure that we select competent and suitable staff to drive change (e.g., identifying a role model whose advice is accepted by colleagues). Second, as concluded in our review, we found that nursing staff occasionally struggled to apply their newly acquired knowledge in practice, indicating that the education or training methods we adopted may not have been suited to their learning styles. This also underlines the importance of interventions that are compatible with the intended users. Approaches such as on-the-job coaching let to enhanced applicability and should be considered in the future.
Finally, despite the aim of this study, no role was defined for pharmacists. In the Netherlands, it is common for organizations to have monthly pharmacotherapeutic consultations and annual mandatory medication reviews with their supervising pharmacists . If organizations wanted additional involvement from their pharmacist, this could be included in the tailored AIP. Nevertheless, in retrospect we do argue that informing pharmacists about the study and the possibility to be involved could have been of added value.
In a subsequent study, it could also be interesting to include the impact of NPS for nursing staff and relate this to compliance. In addition, qualitative data indicates that improvements were perceived, for example related to multidisciplinary collaboration. This was not measured as an outcome. Taking this into account, and considering that many of the implemented actions relate indirectly to PDU, we stress that future trials emphasize a broader range of outcomes such as knowledge, multidisciplinary collaboration, or use of person centred interventions.
Given that residents and their family members are relevant stakeholders and can be an important motivator to change, future studies should consider an implementation strategy in which they can contribute to realizing change.
Strengths and limitations
The use of a mixed methods approach ensured that we obtained an extensive and thorough insight into all aspects of our process evaluation. However, a critical comment is appropriate regarding our respondents. The external coach and internal project leader represented our respondents, while many other stakeholders participated in the study as well. Considering this, we have included a relatively small number of respondents. Moreover, various stakeholders have different professions and areas of expertise, which is likely to influence somebody’s vision, opinion and, as a result, evaluation of the process. Consequently, the data reflects the opinion of these two roles and might be a limited representation of reality. Nevertheless, both roles were key to the RID intervention and we considered the majority of the respondents to be relatively well aware of the perceptions of others involved in the process. Due to turnover, some respondents were not involved from the beginning of the intervention, which possibly affected completeness of the data. The evaluation parameters we used may represent an important limitation given the tailored nature of our intervention. It was overly complicated to determine what parameters needed to be included and how these should be operationalized to evaluate the implementation. For instance, the use of a mean implementation score for AIP actions was deemed suboptimal because each score does not acknowledge the different importance of each action. Finally, recall bias may have influenced the results because the data were only collected after the project, and this may have influenced the reliability and comprehensiveness of our results given the turnover of external coaches and/or internal project leaders in some NHs.
It should be noted that we deviated from our study design in two aspects . First, although the tailored information provision and external coaching were described as implementation strategies, we now believe they should be considered inherent to the core intervention. This is consistent with many complex interventions that incorporate implementation strategies in this way . Given that the process evaluation model we used assumes that intervention and implementation strategies be separate , this model may need to be adapted for process evaluations of complex interventions. The second deviation from our design concerns the examination of barriers and facilitators. To ensure that this aspect was well examined, we developed questionnaires based on common factors reported in other studies [24, 26, 34, 51] and detailed in the CFIR . However, it was difficult to answer the questions without ambiguity, which meant that the interviews were more useful for in-depth exploration.
We hypothesized that implementation would be facilitated using an intervention with PAR, external coaching and tailored information provision. Although these elements were appreciated and implementation may indeed have been facilitated, the added value and effectiveness of these elements depends on a large number of factors. Consequently, the level of implementation (e.g. extent of performance) differed between NHs.
The RID intervention was evaluated positively, but it was also considered to be time consuming and complex. Although external coaching was certainly considered relevant, it was not considered indispensable, with its added value rated differently across organizations. That said, the external coaches stimulated implementation and even had a role in mitigating the effects of some of the barriers we encountered (e.g., facilitating proper takeover of key roles). Also, the effectiveness of coaching may have been dependent on a range of factors, including the organization, openness to coaching and change in general, and whether NH staff and management can be motivated by arguments, facts, and numbers. Staff turnover and reorganization were recurring themes in the analysis of barriers, and given that these are ubiquitous to normal practice, we believe that any future implementation strategy should address innovating within the broader confines of an ever changing environment. Despite our efforts, we partially encountered well-known barriers. This process evaluation provides insights into the implementation of a complex intervention, but it also shows how difficult it is to realize quality improvement and culture change within NHs. This takes time and affects all different kinds of stakeholders and organizational levels. Therefore, future studies do well to assess and support organizations in their readiness to change. Given that the extent of performance of the NHs varied, sensitivity analyses are appropriate when investigating the effects of the RID intervention, taking into account the week in which nursing homes started with implementation and the degree to which actions were implemented as intended. We stress that future trials emphasize a broader range of outcomes such as knowledge, multidisciplinary collaboration, or use of person centred interventions.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available considering all data is in Dutch but are available from the corresponding author on reasonable request.
Action and Implementation Plan
Consolidated Framework for Implementation Research
Consolidated Standards of Reporting Trials
Dementia Special Care Units
Multidisciplinary Project Team
Participatory Action Research
Psychotropic Drug Use
Randomized Controlled Trial
Reducing Inappropriate psychotropic Drug use
Statistical Package for the Social Sciences
University Medical Center
Selbæk G, Engedal K, Bergh S. The prevalence and course of neuropsychiatric symptoms in nursing home patients with dementia: a systematic review. J Am Med Dir Assoc. 2013;14(3):161–9.
Cornegé-Blokland E, Kleijer BC, Hertogh CM van MR. Reasons to prescribe antipsychotics for the behavioural symptoms of dementia: a survey in Dutch nursing homes among physicians, nurses, and family caregivers. J Am Med Dir Assoc. 2012;13(1):e1–6.
Nijk RM, Zuidema SU, Koopmans RTCM. Prevalence and correlates of psychotropic drug use in Dutch nursing-home patients with dementia. Int Psychogeriatrics. 2009;21(3):485–93.
Selbaek G, Kirkevold Ø, Engedal K. The prevalence of psychiatric symptoms and behavioural disturbances and the use of psychotropic drugs in Norwegian nursing homes. Int J Geriatr Psychiatry. 2007;22:843–9.
Wetzels RB, Zuidema SU, de Jonghe JF, Verhey FR, Koopmans RT. Prescribing pattern of psychotropic drugs in nursing home residents with dementia. Int Psychogeriatr. 2011;23(8):1249–59.
Ballard C, Gauthier S, Cummings J, Brodaty H, Grossberg G, Robert P, et al. Management of agitation and aggression associated with Alzheimer disease. Nat Rev Neurol. 2009;5(5):245–55.
Zuidema SU, van Iersel MB, Koopmans RTCM, Verhey FRJ, Rikkert MGMO. Efficacy and adverse reactions of antipsychotics for neuropsychiatric symptoms in dementia: a systematic review. Ned Tijdschr Geneeskd. 2006;150(28):1565–73.
Woolcott J, Richardson K, Wiens M, Patel B, Marin J, Khan K, et al. Meta-analysis of the impact of 9 medication classes on falls in elderly persons. Arch Intern Med. 2009;169(21):1952–60.
Seitz D, Gill S, Herrmann N, Brisbin S, Rapoport MJ, Rines J, et al. Pharmacological treatments for neuropsychiatric symptoms of dementia in long-term care: a systematic review. Int Psychogeriatrics. 2013;25(2):185–203.
Zuidema S, de Jonghe J, Verhey F, Koopmans R. Neuropsychiatric symptoms in nursing home patients: factor structure invariance of the Dutch nursing home version of the neuropsychiatric inventory in different stages of dementia. Dement Geriatr Cogn Disord. 2007;24(3):169–76.
Sink KM, Holden KF, Yaffe K. Pharmacological treatment of neuropsychiatric symptoms of dementia: a review of the evidence. Am Med Assoc. 2005;293(5):596–608.
Van Der Spek K, Gerritsen DL, Smalbrugge M. Nelissen-Vrancken MHJMG, Wetzels RB, Smeets CHW, et al. only 10% of the psychotropic drug use for neuropsychiatric symptoms in patients with dementia is fully appropriate. The PROPER I-study. Int Psychogeriatrics. 2016;28(10):1589–95.
Verenso. Vereniging van specialisten ouderengeneeskunde. Passend gebruik van psychofarmaca voor probleemgedrag bij mensen met dementie. 2018 [cited 2020 Nov 11]. Available from: https://www.verenso.nl/richtlijnen-en-praktijkvoering/richtlijnendatabase/probleemgedrag-bij-mensen-met-dementie/passend-gebruik-van-psychofarmaca-voor-probleemgedrag-bij-mensen-met-dementie
American Geriatrics Society and American Association for Geriatric Psychiatry. Consensus statement on improving the Quality of Mental Health Care in U.S. Nursing Homes: Management of depression and behavioral symptoms associated with dementia. J Am Geriatr Soc. 2003;51(9):1287–98.
Pieper MJ, Achterberg WP, Francke AL, van der Steen JT, Scherder EJ, Kovach CR. The implementation of the serial trial intervention for pain and challenging behaviour in advanced dementia patients (STA OP!): a clustered randomized controlled trial. BMC Geriatr. 2011 Mar;11:12.
Zwijsen SA, Smalbrugge M, Eefsting JA, Twisk JWR, Gerritsen DL, Pot AM, et al. Coming to grips with challenging behavior: a cluster randomized controlled trial on the effects of a multidisciplinary care program for challenging behavior in dementia. J Am Med Dir Assoc. 2014;15(7):531.e1–531.e10.
van der Spek K, Gerritsen DL, Smalbrugge M, Nelissen-Vrancken MH, Wetzels RB, Smeets CH, et al. PROPER I: frequency and appropriateness of psychotropic drugs use in nursing home patients and its associations: a study protocol. BMC Psychiatry. 2013;13:307.
Gerritsen DL, Smalbrugge M, Teerenstra S, Leontjevas R, Adang EM, Vernooij-Dassen MJ, et al. Act in case of depression: the evaluation of a care program to improve the detection and treatment of depression in nursing homes. Study Protocol. BMC Psychiatry. 2011;11:91.
Lichtwarck B, Selbaek G, Kirkevold Ø, Rokstad AMM, Benth JŠ, Myhre J, et al. TIME – Targeted interdisciplinary model for evaluation and treatment of neuropsychiatric symptoms: protocol for an effectiveness-implementation cluster randomized hybrid trial. BMC Psychiatry. 2016;16:233.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50(5):585–92.
Westbury JL, Gee P, Ling T, Brown DT, Franks KH, Bindoff I, et al. RedUSe: reducing antipsychotic and benzodiazepine prescribing in residential aged care facilities. MJA. 2018;209(9):398–402.
Richter C, Berg A, Langner H, Meyer G, KÖpke S. Effect of person-centred care on antipsychotic drug use in nursing homes (EPCentCare): a cluster-randomised controlled trial. Age Ageing. 2019;48(3):419–25.
Smeets CHW, Smalbrugge M, Koopmans RTCM, Nelissen-Vrancken MHJMG, van der Spek K, Teerenstra S, et al. Can the PROPER intervention reduce psychotropic drug prescription in nursing home residents with dementia? Results of a cluster-randomized controlled trial. Int Psychogeriatr. 2021;33(6):577-586.
Zwijsen SA, Smalbrugge M, Eefsting JA, Gerritsen DL, Hertogh CM, Pot AM. Grip on challenging behavior: process evaluation of the implementation of a care program. Trials. 2014;15(302):1–11.
Appelhof B, Bakker C, van Duinen-van den IJssel JCL, Zwijsen SA, Smalbrugge M, Verhey FRJ, et al. Process evaluation of an intervention for the Management of Neuropsychiatric Symptoms in young-onset dementia. J Am Med Dir Assoc. 2018;19(8):663–71.
Quasdorf T, Riesner C, Dichter MN, Dortmann O, Bartholomeyczik S, Halek M. Implementing dementia care mapping to develop person-centred care: results of a process evaluation within the Leben-QD II trial. J Clin Nurs. 2016;26:751–65.
Kolanowski A, Fick D, Frazer C, Penrod J. It’s about time: use of nonpharmacological interventions in the nursing home. J Nurs Scholarsh. 2010;42(2):214–22.
Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;17(3):CD005470.
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7(50):1–17.
Lawrence V, Banerjee S. Improving care in care homes: a qualitative evaluation of the Croydon care home support team. Aging Ment Heal. 2010;14(4):416–24.
Groot Kormelinck CM, Van Teunenbroek CF, Kollen BJ, Reitsma M, Gerritsen DL, Smalbrugge M, et al. Reducing inappropriate psychotropic drug use in nursing home residents with dementia: protocol for participatory action research in a stepped-wedge cluster randomized trial. BMC Psychiatry. 2019;19(1):298.
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:1–7.
Woertman W, de Hoop E, Moerbeek M, Zuidema SU, Gerritsen DL, Teerenstra S. Stepped wedge designs could reduce the required sample size in cluster randomized trials. J Clin Epidemiol. 2013 Jul;66(7):752–8.
Leontjevas R, Gerritsen DL, Koopmans RTCM, Smalbrugge M, Vernooij-Dassen MJFJ. Process evaluation to explore internal and external validity of the “Act in Case of Depression” care program in nursing homes. J Am Med Dir Assoc. 2012;13(5):488.e1–8.
Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:332.
Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(50):1–15.
Braithwaite J, Churruca K, Long JC, Ellis LA, Herkes J. When complexity science meets implementation science: a theoretical and empirical analysis of systems change. BMC Med. 2018;16(63):1–14.
May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11(141):1–12.
Gitlin LN, Rose K. Factors associated with caregiver readiness to use nonpharmacologic strategies to manage dementia-related behavioral symptoms. Int J Geriatr Psychiatry. 2014;29(1):93–102.
Appelhof B, Bakker C, de Vugt ME, van Duinen-van den IJssel JCL, Zwijsen SA, Smalbrugge M, et al. Effects of a Multidisciplinary Intervention on the Presence of Neuropsychiatric Symptoms and Psychotropic Drug Use in Nursing Home Residents WithYoung-Onset Dementia: Behavior and Evolution of Young-Onset Dementia Part 2 (BEYOND-II) Study. Am J Geriatr Psychiatry. 2019;27(6):581-589.
Van Haeften-Van Dijk AM, Van Weert JCM, Droës RM. Implementing living room theatre activities for people with dementia on nursing home wards: a process evaluation study. Aging Ment Heal. 2015;19(6):536–47.
Lawrence V, Fossey J, Ballard C, Ferreira N, Murray J. Helping staff to implement psychosocial interventions in care homes: augmenting existing practices and meeting needs for support. Int J Geriatr Psychiatry. 2016;31(3):284–93.
McAiney CA, Stolee P, Hillier LM, Harris D, Hamilton P, Kessler L, et al. Evaluation of the sustained implementation of a mental health learning initiative in long-term care. Int Psychogeriatrics. 2007;19(5):842–58.
Hazelhof TJGM, Schoonhoven L, van Gaal BGI, Koopmans RTCM, Gerritsen DL. Nursing staff stress from challenging behaviour of residents with dementia: a concept analysis. Int Nurs Rev. 2016;63(3):507–16.
van Duinen-van den IJssel JCL, Mulders AJMJ, Smalbrugge M, Zwijsen SA, Appelhof B, Zuidema SU, et al. Nursing staff distress associated with neuropsychiatric symptoms in young-onset dementia and late-onset dementia. J Am Med Dir Assoc. 2018;19(7):627–32.
Zwijsen SA, Kabboord A, Eefsting JA, Hertogh CMPM, Pot AM, Gerritsen DL, et al. Nurses in distress? An explorative study into the relation between distress and individual neuropsychiatric symptoms of people with dementia in nursing homes. Int J Geriatr Psychiatry. 2014;29(4):384–91.
Pimentel CB, Mills WL, Snow AL, Palmer JA, Sullivan JL, Wewiorski NJ, et al. Adapting strategies for optimal intervention implementation in nursing homes: a formative evaluation. Gerontologist. 2020;60(8):1555–65.
Groot Kormelinck C, Janus S, Smalbrugge M, Zuidema S, Gerritsen D. Complex interventions targeting neuropsychiatric symptoms and/or psychotropic drug use in long term care: a systematic review on implementation barriers and facilitators. PROSPERO: International prospective register of systematic reviews. 2018 [cited 2019 Apr 3]. Available from: https://www.crd.york.ac.uk/prospero/
Hughes CM, Lapane KL. Pharmacy interventions on prescribing in nursing homes: from evidence to practice. Ther Adv Drug Saf. 2011;2(3):103–12.
Pieper MJC, Achterberg WP, van der Steen JT, Francke AL. Implementation of a stepwise, multidisciplinary intervention for pain and challenging behaviour in dementia (STA OP!): a process evaluation. Int J Integr Care. 2018;18(3):1–12.
World Medical Association. WMA Declaration of Helsinki, Ethical Principles for Medical Research Involving Human Subjects. 2018 [cited 2021 Jun 18]. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/
European Parliament and Council. Regulation on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union. Brussels; 2016 [cited 2021 Jun 18]. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679.
We thank all staff of the participating NHs, in particular the internal project leaders and external coaches, and Kas Maas for his collaboration on the RID study. We thank Dr. Robert Sykes (www.doctored.org.uk) for providing editorial services in the final drafts of this manuscript.
This work was supported by the Dutch ministry of Public Health, Welfare and Sport. The funding source had no role in study design, collection, analysis, or interpretation of the data, nor in the writing and the decision to submit this article for publication.
Ethics approval and consent to participate
The Institutional Review Board of the University Medical Center Groningen (UMCG) assessed the study and stated that this non-invasive study does not fall within the scope of the Dutch Medical Research Involving Human Subjects Act. The board had therefore no task in reviewing the protocol (METC decision: METc 2016.094). The study was conducted in accordance with the declaration of Helsinki  and the General Data Protection Regulation . Written informed consent was obtained from resident’ legal representatives regarding the randomized controlled trial assessing the utility of this approach, as reported in the protocol . With respect to the process evaluation, verbal consent was obtained, considering that NH staff were targeted and content of the data related to the evaluation of a quality improvement initiative rather than resident sensitive information. All participants consented to participation in the process evaluation and all but one participant consented to audiotaping of the interviews. Verbal informed consent was recorded on tape. The interviews were transcribed and analyzed with anonymized codes.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Groot Kormelinck, C.M., van Teunenbroek, C.F., Zuidema, S.U. et al. Process evaluation of a tailored intervention to Reduce Inappropriate psychotropic Drug use in nursing home residents with dementia. BMC Geriatr 21, 414 (2021). https://doi.org/10.1186/s12877-021-02357-w
- Process evaluation
- Complex intervention
- Participatory action research
- Nursing home
- Neuropsychiatric symptoms
- Psychotropic drugs