Health education and factors influencing acceptance of and willingness to pay for influenza vaccination among older adults
BMC Geriatrics volume 15, Article number: 136 (2015)
The influenza vaccine is recommended in older population. However the immunization coverage varies globally. It has been reported as low as 10–20 % in some countries. This study explored the acceptance of and willingness to pay for influenza vaccination, comparing acceptance and willingness to pay before and after health education.
The study was conducted with 2693 older people in Bangkok, Thailand. Participants were divided into an education group (n = 1402) and a control group (n = 1291). A validated questionnaire measuring acceptance of and willingness to pay for vaccination was administered during semi-structured interviews before and after education. Data on factors influencing acceptance were analyzed.
Participants’ mean age was 69.5 years, 80 % were women and 82.1 % had at least one co-morbidity. Of the participants, 43.5 % had previously received vaccination more than once, although 92.8 % expressed acceptance of vaccination. Acceptance was associated with a positive attitude toward vaccination (OR 2.1, 95 % CI 1.5–2.9) and a history of receiving vaccination (OR 4.1, 95 % CI 2.8–6.1). At baseline, there were no differences between the education and control groups in terms of work status (p = 0.457), co-morbidities (p = 0.07), medical status (p = 0.243), and previous vaccination (p = 0.62), except for educational background (p = 0.004). Acceptance of vaccination increased to 95.8 % (p < 0.001) after education and willingness to pay increased to 82.1 % (p < 0.001). Education significantly affected those with primary school-level education and no previous vaccination history, with acceptance increasing from 83.3 to 92.6 % (p < 0.001); more than twice as high as the control group (OR 2.4, 95 % CI 1.2–4.7). Viewing an educational video increased the proportion of participants with a high level of knowledge from 29.2 to 49.2 % (p < 0.001), and increased the proportion of participants with a positive attitude from 52.4 to 70.7 % (p <0.001). No significant difference was found in any parameter between the first and second assessment in the control group.
The strategies to increase positive attitudes may enhance the acceptance of vaccination. Health education using an educational video demonstrated a significant impact on acceptance, willingness to pay, knowledge and attitude in older people. This may lead to increased sustainability of the immunization program in older people.
Influenza remains one of the more serious diseases affecting public health, and leads to increasing rates of morbidity, mortality, and hospitalization [1, 2]. Older people, especially those with chronic diseases, are the most high-risk population for severe illness and death [3–5]. The Centers for Disease Control and Prevention (CDC) estimated that during a regular flu season, 90 % of seasonal flu-related deaths in the US occurred in those aged 65 years and older . Influenza infection also reduces functional independence, health and quality of life in older people, particularly those who suffer chronic diseases, and results in general economic loss .
The World Health Organization (WHO) and the CDC have recommended target groups for annual influenza vaccination including older adults . In older adults, the vaccination reduces illness severity and complications by up to 60 % and deaths by 80 % . A number of studies have confirmed the effectiveness and safety of the influenza vaccine. Influenza vaccination has reportedly prevented morbidity, hospitalization and mortality in older people with various conditions such as diabetes [10, 11], chronic heart disease  and patients with end-stage renal disease on hemodialysis . Influenza vaccination is cost effective and safe for older adults [14, 15].
In addition, around the world, populations are aging. A number of countries are experiencing challenges in dealing with an aging population. Thailand’s population is also aging, with 11 % of the 64.5 million Thai citizens expected to be part of an aged society in 2020 . Older people, particularly those with chronic diseases, benefit from protection against diseases through vaccination. Influenza and herpes zoster are the most common communicable diseases in older adults. If a large number of older adults are not vaccinated against influenza, the disease is likely to be more severe and result in death in more cases. If a large proportion of the older adult population receives influenza vaccination, dissemination of the disease within the community will be reduced and individuals will be protected against influenza infection.
In Thailand, influenza vaccination is provided free of charge to people aged 65 years or older and those with chronic diseases including diabetes, cerebrovascular disease, chronic kidney disease, asthma, chronic obstructive pulmonary disease, heart disease and those with cancer receiving chemotherapy. The EU Council of Ministers declared in 2009 to increase coverage of older age groups to a 75 % vaccination coverage by the 2014–2015 season . In Thailand influenza vaccination coverage among those aged 65 years and older (both general population and high-risk population) increased gradually from 11.5 % in 2010, 14.3 % in 2011, to 19.6 % in 2012 , this vaccination coverage is relatively low in Thailand. Although some studies reported the cost-effectiveness of influenza vaccination [19–22], many of these studies did not investigate knowledge about vaccination and attitude towards vaccination, and did not study the factors associated with acceptance of and willingness to pay for vaccination among older Thai people (target vaccination recipients).
International studies have shown that acceptance of influenza vaccination is influenced to different degrees by factors such as age [23–26], sex , marital status [24, 27] and education level [23, 26], with factors varying in different regions and different racial/ethnic populations .
A study conducted with high-risk patients in the US comparing education and lottery incentives as ways to increase vaccination coverage found that an educational brochure encouraged people to get vaccinated against influenza . Other studies have shown that health education using a mailed letter or a personal phone call successfully improved vaccination acceptance among various population groups, including older people, pregnant women and other high risk populations. However, most available studies were conducted in developed countries such as the USA, UK and Spain, where the health care systems, lifestyles and levels of knowledge in older people about vaccination differ from Thailand. Moreover, a survey of older Thai people conducted by the Thai National Statistical Office (NSO) reported that older Thais preferred receiving information by watching or listening rather than reading or via the internet . As a result, education tools such as leaflets, surface mail, and text messages may not be suitable for older Thai people, particularly those who have impaired eyesight or who are less interested in reading [31–37].
The present study was conducted to explore acceptance of and willingness to pay for influenza vaccination among older adults, and to compare acceptance of and willingness to pay for influenza vaccination before and after receiving health education.
The study was approved by the Ethics Committee of the Faculty of Tropical Medicine, Mahidol University, and the Bangkok Metropolitan Administration Ethics Committee for Human Research (BMAEC). Participant was informed and consented before included in the study. The study was conducted with ambulatory older people who attended one of the 68 Health Centers administered by the Department of Health, Bangkok Metropolitan Administration (BMA). Based on demographic profile, the population selected in this study represents the general older population in Thailand . Each center runs regular activities and meetings as part of their care system for the ambulatory older population.
The study population comprised people aged 60 years or older who lived in Bangkok. Participants were recruited from all of the 68 Health Centers in Bangkok. Those willing to participate in the study, who were self-assisted and able to read and sign the consent form were included in the study. Those who were not able to complete the questionnaires or could not watch the educational video were excluded.
Data were collected between 5 June and 5 July 2013. Around 40–50 participants from each of the 68 health centers were sampled using a purposive sampling technique. Participants’ acceptance of vaccination and willingness to pay for vaccination were assessed using validated questionnaires administered as semi-structured interviews at two points (before and after presentation of the educational video). Data from participants were kept confidential by the use of a participant code, meaning the participant’s name was not included on the questionnaires. Questionnaire content validity was verified by experts in the field of clinical science and social science (MD,PhD). Pre-testing was performed with a group of 30 elderly people, after which the questionnaire was modified and approved by the Ethics Committee of the Faculty of Tropical Medicine, Mahidol University, and the Bangkok Metropolitan Administration Ethics Committee for Human Research (BMAEC). The questionnaire consisted of: 1) basic sociodemographic characteristics such as age, sex, educational background, work status, monthly income, co-morbidities, medical status and history of previous flu vaccination; 2) knowledge assessment; 3) attitude assessment; and, 4) acceptance of and willingness to pay for vaccination.
Knowledge was assessed with 12 true or false questions as shown in Table 2 concerning the cause of influenza, high-risk groups and possible complications. To determine vaccination-related knowledge, the questions included the types of vaccine used in Thailand and the reasons for vaccination. Each item was given a score of 0 or 1. Knowledge scores were classified into three groups using Bloom’s Theory, which categorizes by percentage after the score is summed: ≤60 % represented low knowledge, >60–80 % moderate knowledge, and >80 % a high level of knowledge.
Attitude was assessed on a five-point Likert scale comprising 15 issues which were listed in Table 5. Questions about influenza infection included the importance and severity of the disease as well as medication. Vaccination-related items included the population groups that should be vaccinated, vaccine safety, vaccine effectiveness and optimal vaccination schedule. Scores of attitude were interpreted as follows. For the positive answers, Strongly agree, Agree, Neither agree nor disagree, Disagree, and Strongly disagree would receive scores of 5,4,3,2,and 1, respectively. For the negative answers, Strongly agree, Agree, Neither agree nor disagree, Disagree, and Strongly disagree would receive scores of 1,2,3,4, and 5, respectively. Mean attitude scores were classified into three groups by Best’s Theory: 1.00–2.33 = negative, 2.34–3.67 = neutral and 3.68–5.00 = positive.
Participants were divided into two groups: an education group and a control group. Participants enrolled at Health centers with an even number were assigned to the education group, while those enrolled at Health centers with an odd number were put in the control group. Participants in the education group were given a 1-h break after watching an educational video, after which the second assessment was performed. Participants in the control group did not watch the educational video, but were given a 1-h break after which the second assessment was performed.
The present study used a video developed by researchers as the educational tool. The video was based on health belief models describing people as taking action to prevent disease . The content included influenza topics (i.e., influenza infection, seriousness, complications, transmission, influenza prevention and practice after contracting influenza) and influenza vaccination (i.e., effectiveness, safety, possible side effects, influenza vaccination program in Thailand and practices after receiving vaccination). To ensure participants could clearly understand the message, each frame of the educational video contained images as well as written information . The video lasted for 10 min. The same group of experts in the field of clinical science and social science (MD, PhD) verified the accuracy of the video content. Pre-testing was performed with a group of elderly people, after which the video was modified and approved by the Ethics Committee of the Faculty of Tropical Medicine, Mahidol University and the BMAEC.
The data were analyzed using SPSS Version 20 for Windows (IBM Corp., Armonk, NY). Descriptive statistics were used to describe basic sociodemographic characteristics. Knowledge and attitude were represented by the percentage of participants at each level and overall mean and median. Acceptance of and willingness to pay were represented by percentages. Chi-square analysis and then logistic regression were used to determine the associations between potential factors and acceptance or willingness to pay. The Mann–Whitney U test was used to compare mean attitude scores between those who accepted and did not accept vaccination due to non-normal distribution. The McNemar test was used to compare proportion data before and after video education. The Wilcoxon matched-pairs signed-ranks test was used to compare knowledge and attitude scores before and after according to non-normal distribution. All significance levels were set at 0.05.
A total of 3026 older adults were interested in the study, and 2693 (89 %) were included in the analysis. The mean age was 69.5 years (range 60–91 years). Around 55 % were aged 65–74 years, 24 % were aged 60–64 years, and 21 % were ≥75 years. The majority of participants were female (80 %), half were married and around 94 % were Buddhist. Nearly 80 % of participants were retired. In terms of educational background, 54.5 % had primary school-level education or less, 34.9 % were educated to secondary/diploma level and 10.6 % had a Bachelor’s degree or higher. About 62 % of participants had a monthly income of ≤5000 baht (≤154 USD).
At least one co-morbidity (underlying disease) was present in 82.1 % of participants, with hypertension being the most common (52.9 %), followed by dyslipidemia (40.9 %), diabetes (21.8 %) and heart disease (8.3 %). Most of the participants (90.4 %) had visited a medical facility within the past 1 year, and 78.7 % had received influenza vaccination advice from medical personnel. Around 76.8 % had a history of ever having received influenza vaccination (Table 1).
Sources of information on influenza infection and vaccination
In total, 77 % of participants had received information on influenza infection. The most common information source was TV programs (58.9 %), followed by medical personnel (55.3 %), radio (20.6 %), newspaper (19.9 %), advertisement/leaflet (19.0 %) and friends (14.3 %).
Of the participants, 83.1 % had previously received information on influenza vaccination. The most common information source was medical personnel (61.0 %), followed by TV programs (51.4 %), radio (18.6 %), advertisement/leaflet (17.2 %), newspaper (17.1 %) and friends (13.4 %).
Knowledge and attitude toward influenza infection and vaccination
Only 30 % of participants had a high level of knowledge about influenza infection, 38.7 % had a moderate level of knowledge, and 31.1 % had a low level of knowledge. The mean knowledge score was 8.4 (1.85) and the median was 8 (IQR 7–10). Participants tended to know about influenza transmission, practice after getting influenza, people who were in the high risk group and influenza complications, with more than 80 % of participants correctly answering these questions. However, participants knew little about the influenza organism (37.8 % correct) and symptoms (29.2 % correct). In terms of the influenza vaccination, not as many participants knew about influenza vaccination prevention (66.1 %), even though they knew that people should get a flu shot every year (96.7 %) (Table 2).
Nearly 55 % of participants had a positive attitude toward influenza and vaccination, while the remaining 45 % had negative or neutral attitudes. The mean of the total attitude score was 3.74 (0.36) and the median was 3.73 (3.53–4.00). Most participants were concerned about complications from influenza, and believed influenza is preventable and that there should be health education for high risk people. However, a number of participants disagreed with the impacts on health after contracting influenza and being vaccinated against influenza every year. Lots of participants had misconceptions about taking antibiotics. Participants felt that there were benefits of influenza vaccination and that vaccination was effective, but were curious about vaccination safety.
Acceptance and willingness to pay for influenza vaccination
Most participants indicated they would like to receive free influenza vaccination (92.8 %), while 75 % indicated that vaccination would be acceptable when it incurred a cost. The number of people who indicated they needed vaccination decreased if the vaccination cost was higher. Around 37 % of participants could accept vaccination if it cost <100 baht (<3.08 USD); 29.4 % could accept if it cost 100–300 baht (3.08–9.23 USD); 6.2 % could accept if it cost 301–500 baht (9.26–15.38 USD); but only 0.5 % could accept if it cost >500 baht (>15.42 USD).
Factors associated with influenza vaccination acceptance
It was found that nine factors associated with acceptance of an influenza vaccine among older adults. Acceptance of an influenza vaccine increased with age (p = 0.005). There were the relationships between acceptability of an influenza vaccine and work status (p = 0.014), having disease (p = 0.001), having medical visit within the past 1 year (p = 0.003), receiving influenza vaccination advice from medical personnel (p < 0.001), history of having received influenza vaccination (p < 0.001), received influenza information (p < 0.008), received influenza vaccination information (p < 0.001) and being positive attitude (p < 0.001).
Logistic regression analysis was used to determine the association between influenza vaccination acceptance and selected potential variables. A positive attitude and having a history of having received influenza vaccination were associated with influenza vaccination acceptance. Those who had a positive attitude were more likely to indicate that free influenza vaccination was acceptable than those with a negative or neutral attitude (OR 2.06, 95 % CI 1.49–2.85). Those who had a history of having received influenza vaccination were more likely to indicate that free influenza vaccination was acceptable than those with no vaccination history (OR 4.14, 95 % CI 2.81–6.10) (Table 3).
Mann–Whitney U tests were conducted to determine the difference between each attitude issue score and influenza vaccination acceptance. Table 4 shows that those who accepted free vaccination perceived influenza as an important disease in older adults more than those who did not accept free vaccination. Those who accepted free vaccination were also more likely to think that there is an impact on health after an older person becomes sick from influenza infection. Attitude issues about influenza vaccination related to acceptance were perception of influenza prevention, necessity of vaccinating every year, effectiveness or benefits and vaccination safety/low risk of an adverse reaction.
Factors associated with willingness to pay for influenza vaccine among older adults who met free vaccination criteria
The six factors were significantly associated with willingness to pay for influenza vaccine among older adults who met free vaccination criteria from logistic regression model (Table 5). The model included potential variables from chi-square analysis which were marital status (p = 0.039), education level (p < 0.001), monthly income (p < 0.001), having medical visit within the past 1 year (p < 0.001), receiving vaccination advice from medical personel (p < 0.001), history of having received influenza vaccination (p < 0.001), receiving influenza information (p < 0.001), receiving influenza vaccination information (p < 0.001), knowledge level (p = 0.022) and attitude level (p < 0.001). Table 5 reveals that older adults who had secondary education/diploma, monthly income >10,000- < 30,000 Baht, received vaccination advice from medical personal, history of having received influenza vaccination, received influenza information, positive attitude were more likely to express their willingness to pay for influenza vaccine.
Factors associated with willingness to pay for influenza vaccine among older adults who did not meet free vaccination criteria
The four factors significantly associated with willingness to pay for influenza vaccine among older adults who did not meet free vaccination criteria from logistic regression model (Table 5). The model included potential variables from chi-square analysis which were gender (p = 0.012), income (p = 0.004), receiving vaccination advice from medical personnel (p = 0.003), history of having received influenza vaccination (p < 0.001), receiving influenza information (p < 0.001), receiving influenza vaccination information (p < 0.001) and attitude level (p = 0.004). Table 5 reveals that older adults who were males, had monthly income >10,000- < 30,000 Baht, history of having received influenza vaccination, and positive attitude were more likely to express their willingness to pay for influenza vaccine.
Characteristics of education and control group participants
There were 1402 participants in the education group and 1291 in the control group. Baseline characteristics for both groups were no different in terms of their stratified age proportion (p = 0.115), work status (p = 0.457), monthly income (p = 0.296), co-morbidity status (p = 0.07), having had a medical visit within the past one year (p = 0.243) and having a previous vaccination history (p = 0.62). However, overall, participants in the education group had a lower education level than those in the control group (p = 0.004). In the education group, 57.3 % had primary school education or less, 33.4 % had secondary/diploma level education and 9.3 % had a Bachelor’s degree or higher. In the control group, 51.5 % had primary school education or less, 36.4 % had secondary/diploma level education and 12.1 % had a Bachelor’s degree or higher.
Impact of video education on acceptance
In the education group, there was a significant increase in the acceptance of free influenza vaccination after video education from 91.4 % to 95.8 (p < 0.001). The increase in acceptance in those with primary school-level education was also a significant (from 91.3 to 95.8 %, p < 0.001). The control group showed no change in acceptance between the first and second assessment: 94.4 % vs 94.8 % (p = 0.568) for the whole group and 93.4 % vs 93.2 % (p = 1.00) in those with primary school-level education (Table 6).
When a subset of participants with primary school-level education and no influenza vaccination history was analyzed, the impact of the educational video was obvious (education group: 83.3 % vs 92.6 %, p = 0.002; control group: 84.1 % vs 84.1 %; p = 1.00) (Table 6). Chi-square tests found no difference between the groups at baseline (83.3 % vs 84.1 %, p = 0.842), but significantly higher acceptance in the education group compared with the control group at the second assessment (92.6 % vs 84.1 %, p = 0.009). In addition, the educational video had more than twice the effect on acceptance (sex and co-morbidity adjusted OR 2.42, 95 % CI 1.24–4.71).
Impact of video education on willingness to pay
In the education group, video education increased willingness to pay for influenza vaccination from 72.2 to 82.1 %; an increase of 9.9 % accounting for a 13.7 % increase (p < 0.001). The number of participants who expressed willingness to pay decreased as vaccination cost increased. After the educational video, the number of participants willing to pay increased from 36.9 to 43.2 % when vaccination costs were <100 baht (<3.08 USD), and from 27.6 to 30.6 % when vaccination costs were 100–300 baht (3.08-9.23 USD). In the education group, there was a significant increase in willingness to pay in those with primary school-level education (67.4 to 80.1 %; increase of 12.7 % accounting for an 18.8 % increase, p < 0.001). The control group showed no difference in willingness to pay between the first and the second assessment: 77.9 % vs 79.4 % (p = 0.121) in all control group participants and 72.1 % vs 74.7 % (p = 0.094) in those with primary school-level education.
When a subset of participants with primary school-level education and no influenza vaccination history was analyzed, the impact of the educational video was significant (education group: 56.2 % vs 69.6 %, p = <0.001; control group: 60.8 % vs 60.9 %, p = 1.00). Chi-square tests found no difference between the groups at baseline assessment (56.2 % vs 60.8 %, p = 0.364), and at the second assessment (69.6 % vs 60.9 %, p = 0.076).
Impact of video education on knowledge
Before video education, 29 % of those in the education group had a high level of knowledge, 39.1 % had a moderate level and 31.7 % had low level. After video education, a higher proportion of participants reported a high level of knowledge (from 29.2 to 49.2 %, accounting for a 68.5 % increase, p < 0.001). The control group showed no change in the proportion of participants who had a high level of knowledge between the first and second assessment (31.4 % vs 33.2 %, p = 0.134) (Table 7). In terms of knowledge level, the mean score of the education group increased from 8.38 to 9.26 and the median (IQR) increased significantly from 8 (7–10) to 9 (8–11), (p < 0.001). The mean score of the control group leveled off at 8.52 in the second assessment.
Impact of video education on attitude
At baseline, 52.4 % of participants in the education group had a positive attitude and 47.5 % were neutral. After video education, the proportion of participants who had positive attitude rose significantly from 52.4 to 70.7 %, accounting for a 34.9 % increase (p < 0.001) (Table 7). In the control group, nearly the same percentage of participants had a positive attitude in the first and second assessment (57.3 % vs 58.5 %).
Video education enhanced the attitude of participants, with the mean score increasing from 3.72 to 3.86 and the median (IQR) increasing significantly from 3.73 (3.47–3.93) to 3.87 (3.63–4.07) (p <0.001). The mean attitude scores increased significantly after video education for 11 out of 15 issues. The view that antibiotics are unnecessary in influenza infection changed from negative to neutral after education. In addition, there was a change from neutral to positive relating to the impact on health after an older person becomes sick from influenza infection and if a vaccination should be received every year. However, without exposure to the educational video, the mean score for the control group leveled off between the first and second assessment (3.76 vs 3.77), and there was no change in attitude level on any issue.
The present study was performed with a large sample size, which increased the reliability of the results. Characteristics of older participants in our study were consistent with Annual Report Situation of elderly Thais in terms of co-morbidities (DM 21.8 % from this study and 13.3 % from Annual report, heart disease 8.3 and 7 %, cerebrovascular disease 2.6 and 1.6 %), monthly income (7141 Baht and 7495 Baht), education level (primary school level 54.5 and 68.92 %), work status (still work 21.0 and 19.9 %), source of information (tv program 58.9 and 57.4 %, radio 20.6 and 32.8 %) [30, 38].
The percentage of participants who had heard of influenza infection in older adults in our study was similar to the findings of a study with people living along the Thai-Myanmar border; 76.6 % compared with 76.4 % . In terms of influenza vaccination, 83.1 % of older people in our study had heard of influenza vaccination, while 94 % of those in a study with American older people had heard that it was recommended that they be vaccinated against influenza . Our study also found that the most common source of information for older people about influenza infection was TV programs, followed by medical personnel, radio, newspaper, advertisement/leaflet and friends. The previous study with a population living along the Thai-Myanmar border found slightly different results in that a TV program was the most common, followed by radio, newspaper, health center, friends, and brochures/posters . However, this supports our finding that TV programs are the most common source through which Thais received information on influenza infection. Although the most common source for information on influenza infection was TV programs, older people reported that the most common source for information on influenza vaccination was medical personnel. This suggests that influenza vaccination information disseminated through health centers may be limited and in general, may not be sufficient. Therefore, sources of influenza vaccination information should be expanded to include TV programs, radio, newspapers and advertisements/leaflets to promote influenza vaccination programs and immunization coverage.
Most of older adults in our study (69.8 %) had moderate or low level knowledge, a finding similar to that of an earlier study (2008) in a group of 20 older people in the Muang district, an urban community of Chiang Rai province where little was known about influenza infection and influenza vaccination . In contrast, international studies have found that older people were knowledgeable about influenza causes, transmission, symptoms, complications, practices and vaccination [41, 42]. Our findings suggest that few older adults in Thailand have sufficient knowledge about influenza causes, symptoms, complications and vaccination. This indicates that current health education programs on influenza and vaccination may not be adequate.
Around half of older adults in our study had a positive attitude toward vaccination. They were concerned about the serious complications from influenza, but less concerned about impacts on health after becoming sick from influenza infection. Most had a misconception about taking antibiotics during influenza infection, a finding that differed from a 2006 study with Greek elders . In addition, we found that older people less perceived getting an influenza vaccination every year and the safety of vaccination.
Even though 92.8 % of older people in our study were willing to receive a free influenza vaccination, only 76.8 % had history of having received influenza vaccination. Of these, 31.1 % had received vaccination once, 22.5 % had received vaccination twice, and 21.0 % had received vaccination three or more times (the influenza vaccination program in Thailand has been provided free of charge since 2008). This finding highlights the relatively low immunization coverage in Thailand [4, 18] and implies that annual influenza vaccination was not consistent for individuals.
Approximately 37 % of older people were willing to pay for influenza vaccine if the cost was below 100 baht (<3.08 USD), and 29.4 % would pay if the cost was between 100 and 300 baht (3.08-9.23 USD). Only 6.7 % could afford to pay for vaccination if the cost was more than 300 baht (>9.23 USD). However, in Thai hospitals, vaccination prices ranged from around 350 (10.77 USD) in public hospitals to 700 baht (21.54 USD) in private hospitals (not including medical charges). A study with Hong Kong Chinese people in 2004 found that 46 % felt HK$50 (6.39 USD) would be a reasonable cost for vaccination, and 32 % accepted a cost in the range of HK$50–$100 (6.39-12.77 USD) .
Our study indicates that having a positive attitude and a history of previous influenza vaccination were the main factors associated with influenza vaccination acceptance. As highlighted in other studies, people who had a positive attitude were more likely to accept influenza vaccination [44–46]. In terms of previous vaccination history, our finding was consistent with previous studies that having a vaccination history was a powerful indication of vaccination acceptance [42–44, 47–51]. In addition, older people who accepted free vaccination felt that influenza is an important disease in older adults [45, 47, 49, 52–54], and that there is an impact on health after becoming sick from influenza [55–57]. Attitude issues about influenza vaccination related to acceptance were perception of its effectiveness/benefits [25, 27, 41, 50, 51, 53, 55, 56, 58] and perception of its safety/low risk adverse reaction [25, 27, 48–51, 55, 57–59].
Promoting a positive attitude toward influenza infection and influenza vaccination is a potentially effective method of increasing vaccination acceptance and immunization coverage. Moreover, some studies have found that elderly people who were vaccinated against seasonal influenza were more likely to receive pandemic influenza vaccination [44, 60].
Other sociodemographic characteristics found to be significantly related to influenza vaccination included age [23–25], marital status [24, 27], education level [23, 26], income , having co-morbidities [23, 43] and influence of healthcare personnel [15, 24, 27, 43, 46, 50, 51, 58, 59]. These characteristics varied in different countries, races, ethnicities and cultures. Improvements in technology and global travel  mean that areas tend to have mixed cultures; for example, by 2030 in the US, Blacks Hispanic and Asian older adults will comprise a significantly larger proportion than they did in the past. The US census also estimates that by 2050, the older population in the US will look different. It has been predicted that within 10 to 20 years, ethnic minorities will be the new majorities, and in essence, Asians will have the highest rates of growth numerically . The United States encounter the continuous influx of refugees from Southeast Asia since 1975. They reported that Asian Pacific Americans remain one of the most poorly understood minorities  and there are smaller studies indicating these Asian population groups . Thus, understanding of this population is needed for health policy maker and service providers in order to provide facilities against health problems and reduce health care costs of the developed nation . Moreover, in an effort to control contagious and highly infectious disease like influenza, global cooperation is needed. Each area should investigate and report population characteristics, knowledge, attitude, vaccination acceptance and influential factors to focus vaccination and improve immunization coverage throughout the world, meaning global prevention may be within the realm of possibility.
This study evaluated the impact of health education on acceptance and willingness to pay for influenza vaccination by assessing key factors before receiving education (baseline) and after receiving education according to principles of health education assessment . The education tool used correlated with the principles of health education for the elderly . We were able to examine the direct impact of the success of education (intervention), and our results showed that education using a video had a significant impact. Our study also assessed acceptance of and willingness to pay in the control group at two time points and the results showed no difference, indicating that the effect of education was true effect and there was less intervening effect that could distort our conclusions.
Our study represents older people in a developing country and has shown that vaccination acceptance in older people increased from 91.4 to 95.8 % after watching the educational video. Similar studies in developed countries have been conducted with varying results. A study in Spain showed that the use of a printed letter on led to a 3.6 % increase in the immunization rate from 38.1 to 41.7 % . A UK study found an increase in vaccination uptake in older people from 46.7 to 67.9 % after they received a personal letter of invitation to attend an influenza vaccination clinic .
The impact of the educational video was significant in older people who had primary school-level education or less and who had no previous history of influenza vaccination. It is noteworthy that this subset initially had lower vaccination acceptance than the overall study population. After video education, overall vaccination acceptance increased from 83.3 to 92.6 %. Moreover, logistic regression showed the video education increased acceptance by more than two times. Our results highlighted that video education also had benefit for the group of older Thai people who were less likely to accept influenza vaccination. A US study (1996) found that an educational brochure had a striking effect on influenza immunization among high-risk patients who did not accept immunization the prior year . Health education may be important among a number of countries where its population education is relatively low or similar to Thailand data (less than 50 % of population attain secondary education) such as Mexico, Brazil, South Africa, India, China, Indonesia, Philippines, and Vietnam .
Our study showed the importance of using a different form of health education (educational video). Other studies in developed countries have used other education forms such as a letter [31, 34, 36] or brochure , which might not have been suitable for older people due to eyesight impairment or less interest in reading long or complex statements .
The UK study also showed that health checks by a practice nurse who offered influenza vaccination resulted in an immunization rate increase from 48.7 to 74.3 %. However, this intervention was costly and its effect on influenza vaccination rates was modest . Another study focused on hard-to-reach populations such as drug abusers, sex workers, immigrants, homeless people and older people, using a multilevel community-based method that included cooperation among neighbors, local community staff, nurses and physicians. After intervention, interest in receiving vaccination increased from 80 to 94 %. Although the multilevel community-based method had an effective outcome, the intervention required collaboration from many institutes, meaning a larger budget and a longer time were required .
Our study also measured willingness to pay for vaccination, a factor that has not been considered in previous studies. Our results showed the educational video had a high impact on willingness to pay (a 13.7 % increase overall, and an 18.8 % increase in older people with primary school-level education). This increase may result from increased recognition of the benefit of influenza vaccination , and implies that vaccination recipients may be willing to pay when more expensive new generation vaccines become available, although these are relatively more expensive for the government. Although the chi-square test showed no difference in willingness to pay between the education and control groups at the second assessment, other economic factors may be related to willingness to pay such as income level; if the vaccination is more expensive, fewer older people would be willing to pay. Less than a half of older adults in our study could afford vaccination if the price was less than 100 baht (<3.08 USD), but the market price has been much higher. However, the percentage of older people in the education group who were willing to pay for vaccination after the education video increased from 56.2 to 69.6 %.
The present study included participants aged ≥60 years. Retirement is an option for Thai people at the age of 60 years. While vaccination acceptance increased after education, our results showed that affordability is a barrier to vaccination. This indicates that it is important for the Thai government to reconsider the influenza vaccination program criteria to cover older people aged ≥ 60 years.
Aside from acceptance of and willingness to pay for influenza vaccination, the educational video improved the knowledge and attitudes of older people. This is important as Thailand is a developing country with limited knowledge, negative attitude and low influenza vaccination coverage, and differs from developed countries [41, 42]. Our finding that education had an effect on knowledge and attitudes about influenza vaccination was similar to a study with hematopoietic stem cell transplantation patients . Our study found that those who accepted vaccination felt that becoming sick from influenza infection would have an impact on health, and that people should be vaccinated against influenza every year. After video education, there was an increase in positive attitudes toward these two issues. Other attitude issues related to vaccination acceptance also showed an increase in mean scores after education, including perception of vaccination preventing influenza and the effectiveness/benefit of vaccination. Improvement in knowledge and attitude resulted in increased awareness of influenza prevention, which in turn may lead to increased immunization coverage and sustainability of the program [67, 68].
Our quasi-experimental study suggested that education is a successful method of improving vaccination acceptance and willingness to pay for vaccination among elderly people in an aging society. Education intervention is necessary for implementation of vaccination programs, particularly in a developing country such as Thailand due to poor knowledge about vaccination, negative attitudes and low influenza vaccination coverage. An educational video is a helpful tool for health education for older people. It is simple and has both pictures and sound, which may help older people to pay attention, particularly if they have impaired eyesight and less concentration when reading . This indicates that an educational video is an alternative educational tool for older adults that may lead to the sustainability of immunization programs.
A limitation of our study is that the potential factors that showed correlation with acceptance of or willingness to pay in the cross-sectional part of the study do not allow causality to be identified due to the lack of temporal sequence between factors and outcomes.
Influenza vaccination acceptance is associated with having a positive attitude. Therefore, strategies to improve attitudes toward vaccination should be implemented to improve immunization coverage. Furthermore, TV programs are the most common media for conveying information on influenza infection to older adults. However, access to information on influenza vaccination is currently restricted to health care centers. Consequently, sources of vaccination information should be expanded to include TV, radio, newspapers, and leaflets promoting the influenza vaccination program. Health education using an educational video demonstrated a significant impact on acceptance of and willingness to pay for influenza vaccination as well as improving knowledge and attitude in elderly Thais. Health education is particularly important if the vaccination is not available for free. In addition, educational videos provided by the government through tele-media/TV program during influenza vaccination campaigns may improve immunization coverage, particularly in older adults who have low vaccination acceptance. Increasing knowledge and positive attitudes about vaccination will lead to sustainability of the influenza immunization program in low to mid income older populations worldwide.
The Centers for Disease Control and Prevention
The World Health Organization
Thai National Statistical Office
The Bangkok Metropolitan Administration Ethics Committee for Human Research
Bangkok Metropolitan Administration
Thompson WW, Shay DK, Weintraub E, Brammer L, Bridges CB, Cox NJ, et al. Influenza-associated hospitalizations in the United States. JAMA. 2004;292(11):1333–40.
Zhou H, Thompson WW, Viboud CG, Ringholz CM, Cheng PY, Steiner C, et al. Hospitalizations associated with influenza and respiratory syncytial virus in the United States, 1993–2008. Clin Infect Dis. 2012;54(10):1427–36.
Simmerman JM, Chittaganpitch M, Levy J, Chantra S, Maloney S, Uyeki T, et al. Incidence, seasonality and mortality associated with influenza pneumonia in Thailand: 2005–2008. PLoS One. 2009;4(11):e7776.
National Health Security Office. Influenza vaccination program in Thailand 2009. http://www.cdc.gov/flu/about/disease/high_risk.htm. Accessed 22 Jan 2013.
National Health Security Office. Influenza vaccination program in Thailand 2011. http://thaigcd.ddc.moph.go.th/uploads/file/pat%201/%E0%B9%81%E0%B8%99%E0%B8%9A%2B%E0%B9%91%2B%E0%B9%81%E0%B8%99%E0%B8%A7…pdf. Accessed 25 Jan 2013.
Centers for Disease Control and Prevention. What You Should Know and Do this Flu Season If You Are 65 Years and Older. http://www.cdc.gov/flu/about/disease/65over.htm. Accessed 21 May 2015.
Barker WH, Borisute H, Cox C. A study of the impact of influenza on the functional status of frail older people. Arch Intern Med. 1998;158(6):645–50.
U.S. Food and Drug Administration. FDA approves vaccines for the 2012–2013 influenza season. http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm315365.htm. Accessed 25 Jan 2013.
World Health Organization. WHO/Europe recommendations on influenza vaccination during the 2011–2012 winter season. http://www.euro.who.int/__data/assets/pdf_file/0008/152297/EURO_2011_2012-flu-vacc-rec_V2.pdf?ua=1. Accessed 22 Jan 2013.
Looijmans-Van den Akker I, Verheij TJ, Buskens E, Nichol KL, Rutten GE, Hak E. Clinical effectiveness of first and repeat influenza vaccination in adult and elderly diabetic patients. Diabetes Care. 2006;29(8):1771–6.
Wang IK, Lin CL, Chang YC, Lin PC, Liang CC, Liu YL, et al. Effectiveness of influenza vaccination in elderly diabetic patients: a retrospective cohort study. Vaccine. 2013;31(4):718–24.
de Diego C, Vila-Corcoles A, Ochoa O, Rodriguez-Blanco T, Salsench E, Hospital I, et al. Effects of annual influenza vaccination on winter mortality in elderly people with chronic heart disease. Eur Heart J. 2009;30(2):209–16.
Wang IK, Lin CL, Lin PC, Liang CC, Liu YL, Chang CT, et al. Effectiveness of influenza vaccination in patients with end-stage renal disease receiving hemodialysis: a population-based study. PLoS One. 2013;8(3):e58317.
Maciosek MV, Solberg LI, Coffield AB, Edwards NM, Goodman MJ. Influenza vaccination health impact and cost effectiveness among adults aged 50 to 64 and 65 and older. Am J Prev Med. 2006;31(1):72–9.
Nichol KL, Nordin J, Mullooly J, Lask R. Influenza vaccination of the elderly is associated with direct medical care cost savings. Int Congr Ser. 2004;1263:658–60.
Rittirong J, Chuanwan S, Prasartkul P. Population Ageing in ASEAN (in Thai). In: Thaweesit S, Sanpuwan M, Chuanwan S, editors. Population and Society in ASEAN: Challenge and Opportunity. Nakhornpathom: Population and Society Publisher, Institute for Population and Social Research, Mahidol University; 2013. p. 7-26.
European Centre for Disease Prevention and Control. Seasonal influenza vaccination rates across Europe. 2014. http://ecdc.europa.eu/en/press/news/_layouts/forms/News_DispForm.aspx?List=8db7286c-fe2d-476c-9133-18ff4cb1b568&ID=937. Accessed 20 April 2015.
Owusu JT, Prapasiri P, Ditsungnoen D, Leetongin G, Yoocharoen P, Rattanayot J, et al. Seasonal influenza vaccine coverage among high-risk populations in Thailand, 2010–2012. Vaccine. 2015;33(5):742–7.
Praditsuwan R, Assantachai P, Wasi C, Puthavatana P, Kositanont U. The efficacy and effectiveness of influenza vaccination among Thai elderly persons living in the community. J Med Assoc Thai. 2005;88(2):256–64.
Wongsurakiat P, Maranetra KN, Wasi C, Kositanont U, Dejsomritrutai W, Charoenratanakul S. Acute respiratory illness in patients with COPD and the effectiveness of influenza vaccination: a randomized controlled study. Chest. 2004;125(6):2011–20.
Wongsurakiat P, Maranetra KN, Gulprasutdilog P, Aksornint M, Srilum W, Ruengjam C, et al. Adverse effects associated with influenza vaccination in patients with COPD: a randomized controlled study. Respirology. 2004;9(4):550–6.
Plasai VLS, Viputsiri OA, Pongpanich S, Panichpathompong U, Tarnmaneewongse V, Baron-Papillon F, et al. Influenza vaccination among the elderly in Bangkok. Southeast Asian J Trop Med Public Health. 2006;37 Suppl 3:140–4.
Lee KC, Han K, Kim JY, Nam GE, Han BD, Shin KE, et al. Socioeconomic status and other related factors of seasonal influenza vaccination in the South Korean adult population based on a nationwide cross-sectional study. PLoS One. 2015;10(2):e0117305.
Pena-Rey I, Perez-Farinos N, Sarria-Santamera A. Factors associated with influenza vaccination among elderly Spanish women. Public health. 2004;118(8):582–7.
van Essen GA, Kuyvenhoven MM, de Melker RA. Compliance with influenza vaccination. Its relation with epidemiologic and sociopsychological factors. Arch Fam Med. 1997;6(2):157–62.
Damiani G, Federico B, Visca M, Agostini F, Ricciardi W. The impact of socioeconomic level on influenza vaccination among Italian adults and elderly: a cross-sectional study. Prev Med. 2007;45(5):373–9.
Abramson ZH, Cohen-Naor V. Factors associated with performance of influenza immunization among the elderly. Isr Med Assoc J. 2000;2(12):902–7.
Lindley MC, Wortley PM, Winston CA, Bardenheier BH. The role of attitudes in understanding disparities in adult influenza vaccination. Am J Prev Med. 2006;31(4):281–5.
Moran WP, Nelson K, Wofford JL, Velez R, Case LD. Increasing influenza immunization among high-risk patients: education or financial incentive? Am J Med. 1996;101(6):612–20.
Chunharas S. Annual report Situation of elderly Thais 2010. 2012. http://www.oppo.opp.go.th/info/Report_OlderSituation53-TH.pdf. Accessed 20 Feb 2015.
Arthur AJ, Matthews RJ, Jagger C, Clarke M, Hipkin A, Bennison DP. Improving uptake of influenza vaccination among older people: a randomised controlled trial. Br J Gen Pract. 2002;52(482):717–8. 2002.
Coady MH, Galea S, Blaney S, Ompad DC, Sisco S, Vlahov D. Project VIVA: a multilevel community-based intervention to increase influenza vaccination rates among hard-to-reach populations in New York City. Am J Public Health. 2008;98(7):1314–21.
Kececi A, Bulduk S. Health Education for the Elderly. InTech; 2012. http://cdn.intechopen.com/pdfs-wm/29304.pdf. Accessed 25 Feb 2013.
Minor DS, Eubanks JT, Butler Jr KR, Wofford MR, Penman AD, Replogle WH. Improving influenza vaccination rates by targeting individuals not seeking early seasonal vaccination. Am J Med. 2010;123(11):1031–5.
Payaprom Y, Bennett P, Alabaster E, Tantipong H. Using the Health Action Process Approach and implementation intentions to increase flu vaccine uptake in high risk Thai individuals: a controlled before-after trial. Health Psychol. 2011;30(4):492–500.
Roca B, Herrero E, Resino E, Torres V, Penades M, Andreu C. Impact of education program on influenza vaccination rates in Spain. Am J Manag Care. 2012;18(12):e446–52.
Stockwell MS, Westhoff C, Kharbanda EO, Vargas CY, Camargo S, Vawdrey DK, et al. Influenza vaccine text message reminders for urban, low-income pregnant women: a randomized controlled trial. Am J Public Health. 2014;104 Suppl 1:e7–12.
Prachuabmoh V. Annual report Situation of elderly Thais 2012. 2013. http://www.oppo.opp.go.th/info/Report_OlderSituation55-TH.pdf. Accessed 22 Feb 2015.
Rukmanee N, Yimsamran S, Rukmanee P, Thanyavanich N, Maneeboonyang W, Puangsa-art S, et al. Knowledge, attitudes and practices (kap) regarding influenza A (H1N1) among a population living along Thai-Myanmar border, Ratchaburi Province, Thailand. Southeast Asian J Trop Med Public Health. 2014;45(4):825–33.
Payaprom Y, Bennett P, Burnard P, Alabaster E, Tantipong H. Understandings of influenza and influenza vaccination among high-risk urban dwelling Thai adults: a qualitative study. J Public Health (Oxf). 2010;32(1):26–31.
Raftopoulos V. Beliefs, knowledge and attitudes of community-dwelling Greek elders towards influenza and pneumococcal vaccination. J Epidemio. 2007;4.
Nichol KL, Lofgren RP, Gapinski J. Influenza vaccination. Knowledge, attitudes, and behavior among high-risk outpatients. Arch Intern Med. 1992;152(1):106–10.
Mok E, Yeung SH, Chan MF. Prevalence of influenza vaccination and correlates of intention to be vaccinated among Hong Kong Chinese. Public Health Nurs. 2006;23(6):506–15.
Frew PM, Painter JE, Hixson B, Kulb C, Moore K, del Rio C, et al. Factors mediating seasonal and influenza A (H1N1) vaccine acceptance among ethnically diverse populations in the urban south. Vaccine. 2012;30(28):4200–8.
Wong LP, Sam IC. Factors influencing the uptake of 2009 H1N1 influenza vaccine in a multiethnic Asian population. Vaccine. 2010;28(28):4499–505.
Nichol KL, Mac Donald R, Hauge M. Factors associated with influenza and pneumococcal vaccination behavior among high-risk adults. J Gen Intern Med. 1996;11(11):673–7.
Eastwood K, Durrheim DN, Jones A, Butler M. Acceptance of pandemic (H1N1) 2009 influenza vaccination by the Australian public. Med J Aust. 2010;192(1):33–6.
Lau JT, Yeung NC, Choi KC, Cheng MY, Tsui HY, Griffiths S. Factors in association with acceptability of A/H1N1 vaccination during the influenza A/H1N1 pandemic phase in the Hong Kong general population. Vaccine. 2010;28(29):4632–7.
Liu S, Yuan H, Liu Y, Du J, Zhang X, Wang J, et al. Attitudes of seasonal influenza vaccination among healthcare worker and general community population after pandemic influenza A/H1N1 in Hangzhou. Human vaccines. 2011;7(10):1072–6.
Fiebach NH, Viscoli CM. Patient acceptance of influenza vaccination. Am J Med. 1991;91(4):393–400.
Pearson DC, Thompson RS. Evaluation of Group Health Cooperative of Puget Sound's Senior Influenza Immunization Program. Public Health Rep. 1994;109(4):571–8.
Sypsa V, Livanios T, Psichogiou M, Malliori M, Tsiodras S, Nikolakopoulos I, et al. Public perceptions in relation to intention to receive pandemic influenza vaccination in a random population sample: evidence from a cross-sectional telephone survey. Euro Surveill. 2009;14(49).
Kwon Y, Cho HY, Lee YK, Bae GR, Lee SG. Relationship between intention of novel influenza A (H1N1) vaccination and vaccination coverage rate. Vaccine. 2010;29(2):161–5.
Yi S, Nonaka D, Nomoto M, Kobayashi J, Mizoue T. Predictors of the uptake of A (H1N1) influenza vaccine: findings from a population-based longitudinal study in Tokyo. PLoS One. 2011;6(4):e18893.
Gaygisiz U, Gaygisiz E, Ozkan T, Lajunen T. Why were Turks unwilling to accept the A/H1N1 influenza-pandemic vaccination? People's beliefs and perceptions about the swine flu outbreak and vaccine in the later stage of the epidemic. Vaccine. 2010;29(2):329–33.
Huang JH, Miao YY, Kuo PC. Pandemic influenza H1N1 vaccination intention: psychosocial determinants and implications from a national survey, Taiwan. Eur J Pub Health. 2012;22(6):796–801.
Lin Y, Huang L, Nie S, Liu Z, Yu H, Yan W, et al. Knowledge, attitudes and practices (KAP) related to the pandemic (H1N1) 2009 among Chinese general population: a telephone survey. BMC Infect Dis. 2011;11:128.
Honkanen PO, Keistinen T, Kivela SL. Factors associated with influenza vaccination coverage among the elderly: role of health care personnel. Public health. 1996;110(3):163–8.
van Essen GA, Kuyvenhoven MM, de Melker RA. Why do healthy elderly people fail to comply with influenza vaccination? Age Ageing. 1997;26(4):275–9.
Vaux S, Van Cauteren D, Guthmann JP, Le Strat Y, Vaillant V, de Valk H, et al. Influenza vaccination coverage against seasonal and pandemic influenza and their determinants in France: a cross-sectional survey. BMC Public Health. 2011;11:30.
UNWTO Tourism Highlights, 2014 Edition. 2014. http://dtxtq4w60xqpw.cloudfront.net/sites/all/files/pdf/unwto_highlights14_en_hr_0.pdf. Accessed 26 Apr 2015.
Moody HR, Sasser JR. Aging Concepts and Controversies. Los Angeles: Sage; 2012.
Lin-Fu JS. Population characteristics and health care needs of Asian Pacific Americans. Public Health Rep. 1988;103(1):18–27.
Yee BWK. Health and health care of Southeast Asian American elders. http://web.stanford.edu/group/ethnoger/southeastasian.html. Accessed 20 Apr 2015.
Hermalin AI, Ofstedal MB, Tesfai R. Future characteristics of the elderly in developing countries and their implications for policy. 2006. http://www.psc.isr.umich.edu/pubs/pdf/ea06-62.pdf. Accessed 20 Apr 2015.
Ferguson PE, Jordens CF, Gilroy NM. Patient and family education in HSCT: improving awareness of respiratory virus infection and influenza vaccination. A descriptive study and brief intervention. Bone Marrow Transplant. 2010;45(4):656–61.
Bosompra K, Ashikaga T, Ruby A. Attitudes, perceived norms, and intentions: a needs assessment study of the influenza immunization intentions of elderly citizens in Vermont. J Rural Health. 2004;20(2):125–30.
Wong LP, Sam IC. Knowledge and attitudes in regard to pandemic influenza A(H1N1) in a multiethnic community of Malaysia. Int J Behav Med. 2011;18(2):112–21.
This project was funded by the Mahidol University Research Fund. We would like to thank the study participants, the staff of the Health Centers run by the Bangkok Metropolitan Administration and the staff of the Vaccine Trial Center of Faculty of Tropical Medicine, Mahidol University, for their support with providing a venue and assistance with data collection.
To the authors’ knowledge, no conflicts of interest, financial or other, exist.
RW assisted in the design of the study, developed materials, and collected and analyzed the data. WW assisted in recruiting participants and provided support with data collection. KO assisted in design and data analysis. AS assisted in the design of the study. JD assisted in the design of the study and data collection. PP supervised all aspects of study implementation; design of the study, development of materials, collection and analysis of data. All authors read and approved the final manuscript.
About this article
Cite this article
Worasathit, R., Wattana, W., Okanurak, K. et al. Health education and factors influencing acceptance of and willingness to pay for influenza vaccination among older adults. BMC Geriatr 15, 136 (2015). https://doi.org/10.1186/s12877-015-0137-6