Nurses are better equipped for evidence-based practice following postgraduate study

The authorities expect the health service to base decisions concerning treatment, prevention and care on reliable knowledge about the outcome of interventions (1). In the Health&Care21 strategy (2) and the follow-up action plan (3), evidence-based practice (EBP) is described as an instrument for promoting quality, greater patient safety and effectiveness in health and social services.

EBP entails basing decisions on the patient’s wishes and needs in the given situation as well as recommendations from the best research-based and experience-based knowledge available. EBP is presented as a six-step process:

Step one is about reflecting on own practice and step two concerns the framing of a good clinical question. Step three relates to finding research-based knowledge and step four entails the critical appraisal of research literature. Step five is about integrating valid and applicable research-based knowledge with experience-based knowledge and the user’s preferences. Step six involves evaluating own practice (4).

EBP in practice

Melnyk et al. (5) point out that nurses’ EBP-related knowledge, skills and attitudes impact on how research is applied in practice. Despite the increasing focus on evidence-based health services, many Norwegian nurses report that they seldom read research findings or apply them to their own practice (5–7).

One reason for this may be that not all nurses have the knowledge or skills to search for, read, critically understand or apply research (7, 8). In order to achieve the goal of an evidence-based health service, Sandvik, Stokke and Nortvedt (7) suggest that nurses need to be trained in EBP.

EBP in education

Systematic reviews (9, 10) suggest that training in EBP is most effective when it is based on principles for how adults learn. Good information, clear learning goals and specific work tasks with the opportunity to exercise practical skills are indicated as having a positive effect. This also applies to variation in the use of lectures and interactive methods, as well as interaction between participants in the form of small group discussions and group work.

In addition, Khan and Coomarasamy (9) and Young et al. (10) emphasise that the EBP training is most effective when it is based on real issues from practice. This is also the case when students receive training in critical assessment and when they are given regular feedback.

Several studies have shown that EBP training makes nurses more positive towards EBP (7, 11, 12). However, we do not know of any Norwegian surveys that chart nurses’ EBP-related attitudes and behaviour at the start and end of postgraduate study in EBP. The purpose of this study was to map whether postgraduate study in EBP helped change nurses’ attitudes and behaviour with regard to EBP.

Method

Sample and data collection

We conducted a survey of nurses from the specialist health service and primary health service who had completed a postgraduate study programme in EBP at Molde University College in the period 2009–2012 (n = 62, distributed in three cohorts). At the start (pretest) and end (posttest) of the postgraduate study, academic administrators distributed two questionnaires that respondents answered anonymously within 15 minutes.

Postgraduate study

The postgraduate study programme consists of seven seminars over two semesters. The content and teaching methods are based on the recommendations in the book Jobb kunnskapsbasert(4) about evidence-based practice, and the recommendations by Khan and Coomarasamy (9) and Young et al. (10). The systematic reviews indicate that training in EBP is most effective when the participants find it useful for their own practice.

The students therefore use a problem from their own practice as a basis for acquiring knowledge and skills about the first four steps in EBP. At the seminars, the teaching methods alternate between lectures, small group discussions, skills training and the use of interactive methods such as the online course at kunnskapsbasertpraksis.no. devised by Western Norway University of Applied Sciences and the Norwegian Knowledge Centre for the Health Services. The purpose is to teach healthcare personnel to search for, critically appraise and use research-based knowledge in practice.

Between the seminars, the students work on specific course requirements linked to the steps. These requirements are solved in groups of three to five students and are assessed by the teacher. At the final seminar, each group presents a summary of completed course requirements. The groups receive feedback from fellow students and teachers. They then bring together the course requirements in a written assignment, which is the home exam.

Fifteen ECTS credits are awarded to students who pass the exam. Students are encouraged to present the exam assignment at their workplace. The postgraduate study programme does not have a stated goal of training students to implement EBP, but the intention is that student-based methods contribute to the knowledge and skills that are considered to be transferable to practice after completion of the programme. 

Measuring instruments

We used the Norwegian translations of the Evidence-Based Practice Beliefs Scale (13) and Evidence-Based Practice Implementation Scale (13) to measure self-reported changes in attitudes and behaviour with regard to EBP. The scales were translated to Norwegian in 2008 by a research group at the Centre for Evidence-Based Practice, Western Norway University of Applied Sciences in collaboration with the original US authors.

The translation was in line with the World Health Organization’s (14) principle for forward and back translations. In the Norwegian translation, the Evidence-Based Practice Beliefs Scale was called KBP holdningsskalaand the Evidence-Based Practice Implementation Scale became KPB implementeringsskala. We were given permission to use the scales by Nina Rydland Olsen at the Centre for Evidence-Based Practice. 

The EBP Beliefs Scale consists of 16 items that measure self-reported attitudes and confidence in the respondents’ own knowledge and skills in connection with EBP (Table 1). The responses were scored on a five-point Likert scale, ranging from 1 (strongly disagree), 2 (disagree), 3 (neither agree nor disagree), 4 (agree) to 5 (strongly agree). The total score for the scale varies from 16 to 80. A high total score indicates positive attitudes towards EBP.

The EBP Implementation Scale consists of 18 items that measure self-reported activity related to EBP (Table 2). The response alternatives are 0 = 0 times, 1 = 1–3 times, 2 = 4–5 times, 3 = 6–8 times and 4 = > 8 times. The total score for the survey varies from 0 to 72. A high total score indicates increased self-reported practice of EBP. National and international reliability analyses show that the scales have good internal consistency (12, 15, 16).

Research ethics considerations

The Dean of the Faculty of Health Sciences and Social Care granted permission to conduct the survey. At start-up, the nurses received written and oral information about the survey. Participation was voluntary, and those who agreed to participate signed a letter of consent. In order to ensure the confidentiality and anonymity of participants, we did not collect any demographic data.  

Statistical analyses

In line with recommendations by Melnyk et al. (15), the values for the two negatively formulated items in the EBP Beliefs Scale were recoded into reversed content (items 11 and 13 in Table 1) before we conducted the statistical analyses.

The element of missing data for the entire sample was < 3 per cent for all questions. Questions that a participant had omitted to score were given a value that corresponded to the participant’s mean score on the other questions. Mean scores were used as a measure of central tendency in each section of the EBP Beliefs Scale.

In the EBP Implementation Scale, median values were used as a measure of the mean value in each section, since all the score steps above 0 (= 0 times) had more than one specific value (1 = 1–3 times, 2 = 4–5 times, 3 = 6–8 times and 4 = > 8 times). The one-sample Kolmogorov-Smirnov test (17) was used to establish whether the distribution of the scores in the EBP Beliefs Scale and the EBP Implementation Scale were approximately normal.

Since none of the score distributions were normal, we used non-parametric tests in further analyses. Anonymisation prevented us from making a direct (matched) comparison of the individual participant’s scores at pretesting and posttesting. These comparisons were therefore done with statistical tests for dependent samples.

We used the Wilcoxon signed-rank test (17) to test differences between pretesting and posttesting for each section of both scales, and to test differences in the sample’s total scores at pretesting and posttesting. We measured the internal consistency reliability in the EBP Beliefs Scale and the EBP Implementation Scale using Cronbach’s alpha. The study was based on a conventional significance level of p < 0.05.

Since repeated tests in the same sample can give measurement errors, whereby differences that are not real are shown to be statistically significant (type 2 errors), we used the Bonferroni procedure. The Bonferroni procedure involves the use of a more stringent criterion: p < 0.001, for significant findings. We analysed the data using the Statistical Package for Social Sciences (SPSS) version 22.

Results

At pretesting, 56 of 62 nurses (90.3 per cent) completed both scales. At posttesting, 55 of 62 nurses (88.7 per cent) completed both scales. Gender was the only demographic variable available, but since there was only one man in the sample, it was not possible to analyse any gender disparities. The internal consistency of the distribution of responses we measured using Cronbach’s alpha was 0.80 in the EBP Beliefs Scale and 0.89 in the EBP Implementation Scale.

Attitudes towards evidence-based practice

Table 1 shows that the average total score in the EBP Beliefs Scale at pretesting was 52.84, standard deviation (SD) = 5.75 with a range from 41 to 65. At posttesting, the mean total score was 60.80, standard deviation (SD) = 5.41 with a range from 51 to 73. The change in mean total score from pretesting to posttesting for the EBP Beliefs Scale was statistically significant, p < 0.001.

The results in Table 1 show that the change between pretesting and posttesting for items 2, 3, 10, 14 and 15 in the EBP Beliefs Scale was statistically significant (p < 0.001). The pretest results show that nurses were unsure about their own EBP-related knowledge and skills at the start of the postgraduate study (items 2, 10, 14 and 15 in Table 1). A higher mean score at posttesting may indicate increased confidence in their own EBP-related knowledge and skills at the end of the postgraduate study.

At pretesting, the nurses reported that they did not know the steps in EBP (item 2 in Table 1), while at posttesting they reported that they were familiar with these. The nurses also reported that they were sure they could implement EBP both at the start and the end of their postgraduate studies (item 3, Table 1). In connection with knowledge about how they could measure the effect of clinical practice (item 10, Table 1), the results show that they did not know this at pretesting and were unsure about it at posttesting.

With regard to their skills in using EBP, it was found at pretesting that the nurses were not confident in their own ability to implement EBP (item 15 in Table 1) or how EBP could be used to bring about changes in practice (item 14). A higher mean score at posttesting may indicate that a change has taken place. 

A high mean score for items 1, 5, and 9 (Table 1) shows that nurses were confident that EBP could improve their work when they started their postgraduate studies. The posttest results suggest that their studies reinforced their positive attitudes. It was also found that the nurses were confident that critical appraisal of research-based knowledge is an important step in EBP (item 4 in Table 1) at both measurement times.

With regard to items concerning whether their practice is evidence-based (item 16, Table 1), the nurses report that they were unsure about this at both measurement times. 

Behaviour in relation to EBP

At pretesting, median values < 1 indicate that the response alternative ‘0 times’ is the response that the nurses give when answering how many times they have performed various activities related to EBP during the last eight weeks (Table 2). Only for the item about whether they have collected clinical information about a patient problem in the last eight weeks (item 5, Table 2) does the median value indicate that they have done so one to three times (median value > 1).

Table 2 shows that the mean total score in the EBP Implementation Scale at pretesting was 4 (SD = 5.79), with a range from 0 to 25. At posttesting, the mean total score was 20 (SD = 8.92), with a range from 6 to 43. The change in mean scores in the EBP Implementation Scale from pretesting to posttesting is statistically significant, at < 0.001. 

The results show that the nurses report increased activity from pretesting to posttesting for all items except item 7. In addition, the change is statistically significant in 10 out of 18 items, at p < 0.001 (Table 2).

This relates to items about increased activity due to, within the last eight weeks, applying knowledge from different sources, critically appraising a research study, framing a clinical question in a standardised manner and discussing a research study informally with a colleague, presenting research-based knowledge in a report or to a colleague, telling a colleague about clinical guidelines based on research, disseminating knowledge from a research study to an interdisciplinary group, reading and critically appraising a clinical research study, using the Cochrane database and promoting the use of EBP among colleagues.

Discussion

To the best of our knowledge, this study is the first to evaluate whether Norwegian nurses’ EBP-related attitudes and behaviour change between the start and end of postgraduate study in EBP. The main findings in the study show that the nurses gained more confidence in their knowledge of the steps in EBP and in the importance of EBP in the health service. The study also showed that the nurses reported undertaking evidence-based activities to a greater extent after they had completed their postgraduate studies.

A high mean total score on the EBP Beliefs Scale at both pretesting and posttesting may indicate that the nurses have a positive attitude towards EBP and agree that clinical practice based on research improves the quality of the health service. The change in total scores from pretesting to posttesting is significant. Variations in education programmes, differing measurement times and analytical methods make it difficult to draw direct comparisons with results in previous Norwegian studies, but overall the studies reinforce the belief that EBP produces the best treatment outcome for patients (5, 7, 11, 12, 18). 

Attitudes, abilities and knowledge changed 

A review article (19) examining how individual factors impact the use of research in clinical practice suggests that a positive attitude is crucial to whether nurses read and use research in their practice. In our study, we observed that the nurses’ attitudes and assessments of their own ability to use an evidence-based approach in their work were more positive at the end of their postgraduate studies than at the start.

The biggest change was in item 2 (Table 2), where the nurses went from disagreeing that they had knowledge of the steps in EBP, to reporting that they were familiar with the steps at posttesting. The result suggests that the nurses learned more about the steps in EBP in their postgraduate studies. This result concurs with a Norwegian (12) and an international study (19). These studies reinforce the assumption that training in EBP is crucial to meeting the health authorities’ expectations for an evidence-based health service (20).

A low mean total score at pretesting of the EBP Implementation Scale shows that the nurses reported low EBP-related activity before starting their postgraduate studies, which may suggest that they had little focus on evidence-based activities in practice.

Barriers to implementing EBP

The increase in mean total scores from 4 at pretesting to 20 at posttesting indicates that the nurses reported more EBP-related activity after they had completed their postgraduate studies. The result is still low, but is somewhat higher than that found in, for example, the study by Snibsøer et al. (12). According to Snibsøer et al. (12), a low mean total score on the EBP Implementation Scale may be associated with barriers such as lack of allocated time, resources, knowledge and authority to change practice.

In order to successfully implement EBP, it is therefore important to map barriers in the specific context (21). Dahlheim et al. (22) indicate that skills and knowledge in EBP can help reduce such barriers. They found that nurses with evidence-based skills more often read research than those who do not have such skills (22). Increasing research-related activity can lead to better patient safety and job satisfaction (23).

Evidence-based activities in practice

In post-testing, the results showed a statistically significant change in the nurses’ EBP-related activity level in 10 out of 18 items after the Bonferroni correction. The nurses reported that they had used knowledge from different sources, critically appraised a research study and framed clinical questions in a standardised manner to a greater extent at posttesting than at pretesting. They had also discussed research with colleagues and disseminated and presented it to them, used Cochrane Library’s database and promoted the use of EBP among colleagues.

Although the plan was for the nurses to work with the steps in EBP based on a clinical problem from their own practice (9), it may be that the education only had an ‘academic’ effect. The nurses’ uncertainty about whether their practice was evidence-based both at the start and end of their postgraduate studies may indeed indicate this.

Comparable studies (11, 12, 18) show that nurses report limited evidence-based activity in practice despite their EBP training. Further research on what is needed to ensure greater implementation of EBP in practice is therefore necessary.

The understanding of research needs to improve

A surprising result in the study was statistically significant changes in items about EBP that were not thematised in the postgraduate study, e.g. items 6, 8 and 10 (Table 2), which deal with presenting research to colleagues. Greater understanding and discussion of research results in practice can help more nurses to recognise the value of using research-based knowledge in practice.

According to a literature review by Sandvik et al. (7), healthcare personnel’s ability to read and understand research is crucial to being able to implement research in practice. Researchers must therefore take responsibility for presenting research results in a way that healthcare personnel understand. Another surprising result showed that, during their postgraduate studies, the nurses indicated that they had become more sure about how to measure the impact of clinical practice. The postgraduate study programme did not focus on impact assessment, but emphasised learning activities aimed at increasing the nurses’ critical appraisal abilities.

Reading and critically appraising more research

At both measurement times, the nurses agreed that critically appraising research-based knowledge (item 4, table 1) is an important step in EBP. The change between pretesting and posttesting showed a significantly greater emphasis on this (p < 0.005). Because repeated significance tests within the same sample can give unreliable findings, we used Bonferroni-corrected p-values. In order to identify a significant difference, the p-value would have to be at least 0.001.

Despite this, the finding suggests that there was a statistical tendency for the postgraduate study to contribute to the greater emphasis on critically appraising research. The change between pretesting and posttesting in items 2 and 11 in the EBP Implementation Scale (Table 2) shows that reading and critically appraising research were two of the activities that had increased most. This partly explains the growth in support for EBP.

Other Norwegian studies (6, 24) show that Norwegian nurses think that critically appraising research is a challenge. It is therefore positive that the nurses in our study report obtaining more knowledge and skills in this area. The ability to think critically is found to be crucial to being able to use research in practice (24, 25).

Everyone should have competence in EBP

Whether nurses should learn EBP in undergraduate or postgraduate studies is open to discussion. Today’s nursing students learn about the steps in EBP in their undergraduate education, but not all nurses who graduated in past years received such training. Nurses with many years of experience often have an informal authority to set the standard in practice.

Efforts to ensure that everyone has competence in EBP are therefore necessary to safeguard the quality in the future health service. Health policy makers are therefore proposing the mandatory training of healthcare personnel in EBP as one of several instruments for improving the quality in the health service (2). Nurses are the largest healthcare profession in the health service, and their ability to apply EBP therefore impacts on the quality of the entire health service.

More knowledge about the steps in EBP can make nurses more likely to search for and discuss research with colleagues and nursing students. We believe that establishing a culture in which it is natural to read and discuss research can make a positive contribution to implementing EBP in practice.

Some of the items in the study also showed no significant change between pretesting and posttesting. These results can reflect themes where the postgraduate study programme has potential for improvement. Results from the EBP Beliefs Scale suggest that more emphasis should be placed on the positive effects of EBP on clinical practice.

Results from the EBP Implementation Scale indicate that postgraduate study should have a greater focus on measures that can reduce barriers to implementing EBP in practice. Postgraduate study programmes should also focus on the patient’s right to research-based information. According to Snibsøer et al. (12), the results of training in evidence-based practice can differ depending on whether it takes place in an academic institution or in a clinical nursing context.  

Strengths and limitations of the study

Although the results are in keeping with national and international studies, the sample size in the study limits our ability to generalise the results. On the other hand, generalisation is aided by the high response rates at both measurement times and the participation of respondents from different parts of the specialist health service and the primary health service.

The reliability of the survey is also strengthened since the internal consistency reliability was good for both scales. A high internal consistency in the scores of a tool means that the respondents have understood the different elements of the tool in the same way and that the difference between the participants’ scores is not due to systematic measurement errors. With a conventional p-value of 0.05, there would have been 8 significant differences in the EBP Beliefs Scale and 14 in the EBP Implementation Scale.

Using a more stringent significance level (p < 0.001) gave fewer significant differences, but increased the probability of the differences found between pretesting and posttesting being real. We believe that the increased reliability in the study’s result strengthens the study’s internal validity. 

A methodical limitation of the study was that the academic administrator for the postgraduate study programme carried out the study. Since the scales are based on self-reporting, this may have contributed to systematic errors in scores if the respondents adapted the answers to what they thought the researchers expected to find. That participants’ responses do not necessarily reflect reality will always be a weakness in such surveys (26). However, the fact that the questionnaires were answered anonymously may reduce this source of error.

Ideally, the students should have been randomised into a group that had studied EBP and a control group that received the corresponding amount of teaching in other thematic content. However, such a randomised design was not feasible given the eight-month survey period. The choice of a naturalistic pre- and post-design means we can steadfastly assert that only postgraduate study in EBP contributed to the result.

Another limitation is that the anonymous nature of the study meant we were unable to collect demographic information about the respondents and analyse the possible impact of such factors. However, the results from two systematic literature reviews (27, 28) showed no significant correlation between socio-demographic variables and attitudes and behaviour in connection with EBP among nurses.

The danger of using the same scale at pretesting and posttesting is that the respondents learn something from the first measurement that affects the result of the subsequent measurement. Since pretesting and posttesting were more than eight months apart, it is unlikely that the respondents remembered exactly how they responded in the pretest. Another limitation may be that some respondents answered the EBP Implementation Scale at pretesting based on learning activities in the postgraduate study programme and not on their own practice.

Since the scales were developed in and for a US context, the applicability of the scales needs to be studied in the Norwegian health service context. Furthermore, that our study could have given different results if it had been carried out today cannot be ruled out.

Implications for further research

There is a great need for studies that use methods other than self-reporting of attitudes, knowledge and skills in order to evaluate training in EBP. Future research should instead focus on process outcome metrics that measure whether practices change as a result of training in EBP.

Conclusion

Postgraduate study strengthened the nurses’ positive attitudes towards EBP and their belief in the value of using evidence-based knowledge in practice. In addition, the nurses reported an increase in evidence-based activities. Since we lack empirical evidence on whether nurses actually changed their own practices, we cannot conclude that postgraduate study leads to greater implementation of EBP in practice.

Self-reporting of attitudes and practices is important, but it is the actual implementation of EBP and the impact this has on patients that is most crucial for the quality in the nursing profession. 

References

1.         Meld. St. nr. 11 (2014–2015). Kvalitet og pasientsikkerhet 2013. Oslo: Helse- og omsorgsdepartementet; 2014.

2.       Helse- og omsorgsdepartementet. HelseOmsorg21: et kunnskapssystem for bedre folkehelse. Oslo: Helse- og omsorgsdepartementet; 2014.

3.       Helse- og omsorgsdepartementet. Regjeringa sin handlingsplan for oppfølging av HelseOmsorg21-strategien: forsking og innovasjon i helse og omsorg (2015–2018). Oslo: Helse- og omsorgsdepartementet; 2015.

4.       Nortvedt MW, Jamtvedt G, Graverholt B, Nordheim LV, Reinar LM. Jobb kunnskapsbasert!: en arbeidsbok. Oslo: Akribe; 2012.

5.       Berland A, Gundersen D, Bentsen SB. Evidence-based practice in primary care. An explorative study of nurse practitioners in Norway. Nurse Education in Practice. 2012;12(6):361–5.

6.       Hommelstad J, Ruland CM. Norwegian nurses’ perceived barriers and facilitators to research use. AORN Journal. 2004;79(3):621–34.

7.       Sandvik GK, Stokke K, Nortvedt MW. Hvilke strategier er effektive ved implementering av kunnskapsbasert praksis i sykehus? Sykepleien Forskning. 2011;6(2):160–7. Available at: https://sykepleien.no/forskning/2011/05/hvilke-strategier-er-effektive-ved-implementering-av-kunnskapsbaser-praksis-i(downloaded 27.04.2017).

8.       Mahoney JS. Evidence-based practice and research scholars programs: Supporting excellence in psychiatric nursing. Bulletin of the Menninger Clinic. 2009;73(4):355–71.

9.       Khan KS, Coomarasamy A. A hierarchy of effective teaching and learning to acquire competence in evidenced-based medicine. BMC Medical Education. 2006;6:59–67.

10.     Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care? Overview of systematic reviews. PloS One. 2014;9(1):1–13.

11.     Bratberg GH, Haugdahl HS. Opplæring i kunnskapshåndtering – nytter det? Sykepleien Forskning. 2010;3(5):192–9. Available at: https://sykepleien.no/forskning/2010/10/opplaering-i-kunnskapshandtering-nytter-det(downloaded 27.04.2017).

12.     Snibsøer AK, Olsen NR, Espehaug B, Nortvedt MW. Holdning og atferd knyttet til kunnskapsbasert praksis. Sykepleien Forskning. 2012;3(7): 232–41. Available at: https://sykepleien.no/forskning/2012/10/holdning-og-atferd-knyttet-til-kunnskapsbasert-praksis(downloaded 27.04.2017).

13.     Melnyk BM, Fineout-Overholt E. Evidence-based practice in nursing & healthcare: A guide to best practice. Philadelphia: Lippincott Williams & Wilkins; 2005. Norsk oversettelse til KBP holdningsskala og KBP implementeringsskala av NR Olsen. Bergen: Høgskolen i Bergen, Senter for kunnskapsbasert praksis; 2008.

14.     WHO. Process of translation and adaption of instruments. 2011. Available at: http://www.who.int/substance_abuse/research_tools/translation/en/(downloaded 12.02.2015).

15.     Melnyk BM, Fineout-Overholt E, Mays MZ. The evidence-based practice beliefs and implementation scales: Psychometric properties of two new instruments. Worldviews on Evidence-Based Nursing. 2008;5(4):208–16.

16.     Thorsteinsson HS. Translation and validation of two evidence-based nursing practice instruments. International Nursing Review. 2012;59:259–65.

17.     Field A. Discovering statistics using IBM SPSS statistics: and sex and drugs and rock 'n' roll. 4. ed. Los Angeles: SAGE; 2013.

18.     Stokke K, Olsen NR, Espehaug B, Nortvedt MW. Evidence based practice beliefs and implementation among nurses: a cross-sectional study. BMC Nursing. 2014;13(8):1–10.

19.     Squires JE, Estabrooks CA, Gustavsson P, Wallin L. Individual determinants of research utilization by nurses: a systematic review update. Implementation Science 2011;6:1.

20.     Meld. St. nr. 16 (2010–2011). Nasjonal helse- og omsorgsplan 2011–2015. Oslo: Helse- og omsorgsdepartementet; 2011.

21.     Kajermo KN, Bostrom AM, Thompson DS, Hutchinson AM, Estabrooks CA, Wallin L. The BARRIERS scale: the barriers to research utilization scale: a systematic review. Implementation Science. 2010;5:32.

22.     Dalheim A, Harthug S, Nilsen R, Nortvedt MW. Factors influencing the development of evidence-based practice among nurses: a self-report survey. BMC Health Services Research. 2012;12:367.

23.     Melnyk BM, Gallagher-Ford L, Long L, Fineout-Overholt E. The establishment of evidence-based practice competencies for practicing registered nurses and advanced practice nurses in real-world clinical settings: Proficiencies to improve healthcare quality, reliability, patient outcomes and costs. Worldviews on Evidence-Based Nursing. 2014;11(1):5–15.

24.     Wangensteen S, Johansson IS, Bjørkstrøm ME, Nordstrøm G. Critical thinking dispositions among newly graduated nurses. Journal of Advanced Nursing 2010;66(10):2170–81.

25.     Horsley T, Hyde C, Santesso N, Parkes J, Milne R, Stewart R. Teaching critical appraisal skills in healthcare settings. Cochrane Database of Systematic Reviews. 2011;(11):1–35.

26.     Polit DF, Beck CT. Nursing research: generating and assessing evidence for nursing practice. Philadelphia: Wolters Kluwer Health; 2012.

27.     Squires JE, Estabrooks CA, Gustavsson P, Wallin L. Individual determinants of research utilization by nurses: a systematic review update. Implementation Science. 2011;6(1):1–20.

28.     Estabrooks CA, Floyd JA, Scott-Findlay S, O'Leary KA, Gushta M. Individual determinants of research utilization: a systematic review. Journal of Advanced Nursing. 2003;43(5):506–20.