Midwifery practices in the prevention of prolonged labour: a clinical audit
Few midwives followed the recommendations for defining active labour in the electronic medical records. It is uncertain how many women in labour were continuously monitored by a midwife.
Background: To facilitate normal childbirth, the World Health Organization recommends minimising the use of clinical interventions, in line with the principles of perinatal care. In Western countries, the tendency is for pharmacological interventions to be initiated, or for labour to be monitored using advanced medical equipment that is not necessarily required. This practice may be more likely to cause harm than to improve the neonatal outcome. Figures from the Medical Birth Registry of Norway show that the most common reason for medical interventions is that labour is progressing more slowly than expected. The Obstetrics Guidelines for healthcare professionals recommends the use of non-clinical interventions to prevent slow progress.
Objective: To identify the extent to which midwives complied with the recommendations in the Obstetrics Guidelines on preventing slowly progressing labour. We also planned to devise and implement quality improvement measures if necessary, and subsequently evaluate whether midwifery practices had changed.
Method: We conducted a criteria-based clinical audit at a women’s clinic in Norway. Electronic maternity patient records were reviewed retrospectively and documented practice was measured against four criteria based on recommendations in the Obstetrics Guidelines (audit). The criteria related to the use of electronic tools in the medical record system, continuous monitoring by the same midwife, initiation of non-pharmacological interventions and documentation of assessments before pharmacological interventions were initiated. After the audit, we identified the barriers to modifying practice and devised a customised implementation strategy. We implemented quality improvement measures that were primarily aimed at the use of electronic medical record tools, and then conducted a reaudit.
Results: We found a lower level of compliance than in the set standard of 100 per cent for all criteria in the audit and reaudit. The reaudit (n = 92) showed a statistically significant higher level of compliance in the use of electronic medical record tools (p <0.001). Non-pharmacological interventions were initiated in 97 per cent of the records, both in the audit and the reaudit
Conclusion: After quality improvement measures were implemented, there was still a discrepancy between the standard and measured levels of compliance. There was a significant change in the use of electronic medical record tools – the criterion that received the most attention during the implementation period. The way we implemented the quality improvement strategy, the formulation of the criteria, and the standard we set are all factors that may have influenced the results.
In normal births, midwives are responsible for monitoring the patient and delivering the baby. A normal birth is defined as a full-term delivery with spontaneous onset of labour with no known risk factors, where no deviations from the normal process are found or occur during or immediately after delivery (1).
If such deviations are identified, the midwife is required to provide the patient with access to medical care. Despite the fact that one of the midwife’s main tasks is to facilitate a normal birth, there is a growing tendency in the Western world to implement clinical or technological interventions without any evidence that these are required (2).
Recommendations in the field
The World Health Organization (WHO) recommends that clinical interventions are only used in normal births if they are needed to prevent an unwanted outcome (3). This demedicalisation of the normal birth process is an important principle in maternity care.
A systematic review of the research literature conducted in 2014 (2) shows that adhering to the principle of demedicalisation both facilitates health promotion and saves resources. Technological interventions to monitor labour and medication that is given ‘to be on the safe side’ or before the need arises are more likely to cause harm than to improve the neonatal outcome (2).
The Norwegian Directorate of Health has identified a need to improve the quality of midwifery services, and emphasises the importance of carrying out systematic quality improvement work and implementing evidence-based recommendations in the field of practice (4).
If the field of practice is not aware of, or does not adhere to evidence-based recommendations, a gap could arise between theory and practice.
Recommendations for slowly progressing labour
Slow progress is the most common reason for initiating intrapartum clinical interventions in Norway (5). The maternity units’ internal guidelines on intrapartum monitoring are based on the recommendations in the Norwegian Obstetrics Guidelines (6).
In summary, the recommendations on slowly progressing labour are as follows: define the onset of labour and slow progress, use the partogram in the maternity patient record system and the alert and action line tool, and implement non-pharmacological interventions (soft interventions) such as movement, nutrition and safe elimination.
One-to-one midwifery support is also recommended, and medications should only be given where a need has been identified (7).
These recommendations from the Obstetrics Guidelines are also set out in a safe birth project in which a package of measures was introduced as part of the patient safety programme Pasienttryggleik 24/7 (8). The purpose of the package of measures is to reduce the incidence of serious damage to the fetus and fetal death.
One of the measures is to facilitate a more targeted approach to treatment with oxytocin for slowly progressing labour. The use of oxytocin is controversial, and when used incorrectly, is associated with serious harm to both mother and child (9).
Studies from Norway (10) and Sweden (11) show that oxytocin is also given where slowly progressing labour has not been diagnosed. Use of oxytocin in a normal birth process violates the principle of demedicalisation.
Objective of the study
The Obstetrics Guidelines include recommendations on intrapartum monitoring and what interventions can be initiated to expedite the birth (7). It is not known what proportion of maternity patients are monitored in accordance with these recommendations.
Using the clinical audit method, the objective of this project was 1) to identify the extent to which midwives complied with the recommendations in the Obstetrics Guidelines on facilitating normal progress in labour, and 2) if necessary, initiate quality improvement measures and subsequently investigate whether the proportion of maternity patients who were monitored in line with the recommendations changed.
We conducted a criteria-based clinical audit as per the model of the Healthcare Quality Improvement Partnership (HQIP) (12).
In a clinical audit, current practice is mapped and measured against predefined criteria based on the best available knowledge from evidence-based guidelines and/or quality-assessed systematic reviews (audits) (13).
Where there are discrepancies between the desired, ideal practice (the criteria and standard) and the measured practice (actual practice), quality improvement measures are devised and implemented before the practice is then measured again (reaudit).
A successful clinical audit can help close the gap between theory and practice and improve the quality of the health services offered to patients (12).
The project was carried out at a women’s clinic in Norway, consisting of three maternity wards that in total deliver approximately 5000 babies a year. The safe birth project was part of the regional health authority’s patient safety programme. The maternity ward’s internal guidelines for intrapartum monitoring were drawn up in accordance with the recommendations in the Obstetrics Guidelines.
Criteria, standard and knowledge base for the audit
On the basis of the question ‘How can midwives facilitate progression in the normal birth process?’ we conducted a systematic search for guidelines and research papers in the following databases: the Guidelines International Network (G-I-N), the Cochrane Database of Systematic Reviews, MEDLINE, Maternity and Infant Care, and UpToDate.
Our criteria were based on research literature that was critically assessed using checklists adapted to the research design.
We assessed the quality of the recommendations in the Obstetrics Guidelines using the Appraisal of Guidelines for Research and Evaluation (AGREE) II (14), which is a tool for assessing clinical guidelines.
We formulated four process criteria and an associated standard (Table 1). All criteria were in accordance with the clinic’s internal guidelines. We considered the criteria to be accomplishable by the staff and acceptable to the patients.
The standard was set at 100 per cent for all four criteria because the research literature did not indicate circumstances that warranted exceptions to the recommendations.
In accordance with the knowledge base for this audit, the safe birth project recommends monitoring the alert line and the action line at four-hour intervals.
Activation of these lines indicates that the woman is in the active phase of labour, defined here as regular, painful contractions and cervical dilation of more than four centimetres.
If there is a break in the alert line, this may be an indication that labour is progressing slower than expected.
In such cases, the knowledge base for the audit recommends targeted, soft interventions to facilitate a normal birth process, such as offering the patient food and drink, encouraging movement and standing positions, and ensuring that the patient has emptied her bladder (7, 15, 16).
If labour shows no signs of progress after four hours, a break will appear in the action line. The recommendation in this event is a thorough vaginal examination. Medication must then be given if this is indicated by the examination (7, 15, 16).
One of the quality indicators for maternity care is that the patient is given one-to-one midwifery support in the active phase of labour (17).
This means that when the criteria are met and the alert and action lines have been set, the patient must have as much access as she needs to the same midwife per shift. One-to-one midwifery support is also recommended in the knowledge base for the audit (7, 15).
The samples in the audit and the reaudit consisted of maternity patient records retrieved from the extended delivery protocol in Natus from September 2015 (audit) and October 2016 (reaudit).
The criterion for inclusion was that the maternity patient record referred to a nulliparous or multiparous woman with no previous caesarean delivery, with cephalic presentation, spontaneous onset of labour, and gestational age from 37 weeks and 0 days to 41 weeks and 6 days.
We excluded all records showing conditions that, according to the clinic’s procedures, would require clinical intervention with oxytocin. Records of patients who gave birth less than two hours after arrival were also excluded.
The literature that provided the knowledge base for the audit did not contain any audit tools that were suitable for collecting data on our criteria. We therefore devised our own audit tool for obtaining data and demographic variables, such as the women’s age, origin, number of previous deliveries and maternity ward.
We held a brainstorming session with two midwives from the clinic in order to identify possible variables and practices. Most of the discussion related to how one-to-one midwifery support could be measured using the data we had available.
We decided that criterion 2 (one-to-one midwifery support) should be marked as met if all the medical record entries in the active phase of labour were signed by the same midwife per shift.
We briefly discussed whether it was necessary to use more than one data source to assess whether the other criteria were met.
The newly devised audit tool was assessed for face validity and content validity by two midwives from the clinic (face validity) and two other midwives from the clinic who were also associated with the safe birth project (content validity).
Input from these midwives led us to nuance the response alternatives for some of the criteria, and to collect data for criterion 4 for two events. We also developed a guide for entering information in the audit tool.
Four midwives carried out independent pilot tests of the final audit tool by entering information in the audit tool from ten randomly selected medical records. We assessed the reliability of agreement between the midwives’ entries using Fleiss’ kappa, which ranged from 0.64 to 0.91. This indicated significant agreement (inter-rater reliability) (18).
Data collection and statistical analysis
We collected data retrospectively from the clinic’s electronic maternity information system, Natus (19), and from a written log in the fetal monitoring analysis tool, STAN (20). The sample sizes were calculated using an epidemiological tool from Ausvet (21).
The calculation showed that 240 records should be included – 160 from the audit and 80 from the reaudit – in order to find a change in compliance from 40 per cent (expected level at the audit) to 60 per cent (estimated level for the reaudit) as statistically significant, with a strength of 80 per cent.
All records that met the inclusion criteria were numbered in ascending order. A random sample of records was then extracted using the Random Integer Set tool (22).
All data were recorded and analysed in IBM SPSS Statistics, version 23.0 (23). We used the statistical software package R (24) to calculate the 95 per cent confidence interval (CI) for change in compliance. The significance level was set at 0.05. Odds ratio (OR) values are presented with a CI of 95 per cent.
The project was led by the clinic management, and the field of practice was deemed suitable for conducting a clinical audit based on criteria given in the Audit Project Assessment Tool (12). We devised an implementation strategy in parallel with the mapping of practice.
In order to identify factors that hampered and facilitated changes in practice we held a brainstorming session in collaboration with two midwives from the safe birth project and two other midwives from the clinic. The results were plotted onto a cause-effect diagram.
We then identified several barriers to changing practice. The midwives used different definitions of the active phase of labour and were keen to retain their professional autonomy.
Other barriers were the limited number of midwives on duty, lack of awareness of the recommendations, lack of training and skills, and lack of documentation procedures. The drivers for change were the midwives’ high level of professional commitment and their desire to provide good midwifery support.
We developed the implementation strategy on the basis of the results from the brainstorming session and the results from a literature search for implementation research.
The search was performed in the Cochrane Library’s category ‘Effective practice & health systems’, with the subgroup ‘Implementation strategies’, and in the Epistemonikos and Implementation Science databases.
Due to a change in management at the clinic, and thus also available resources, the planned implementation strategy could not be executed in its entirety. Table 2 gives an overview of the planned and executed implementation measures.
The measures aimed at changing practice were implemented over a one-month period. We then assessed the practice again (reaudit).
The project was approved by the hospital trust’s Data Protection Officer (reference number 2016/4454), and was considered exempt from disclosure by the Regional Committee for Medical and Health Research Ethics (reference number 2016/737). All data was stored on the hospital trust’s quality server and treated confidentially.
We included a total of 228 records: 136 in the audit and 92 in the reaudit. The proportion of medical records for nulliparous women was lower in the audit (43 per cent) than in the reaudit (66 per cent) (p = 0.001).
We also observed a statistically significant change in the distribution of the number of deliveries at the different wards in the audit and the reaudit (p <0.001). For other background variables, the changes were only minor (Table 3).
The audit showed a lower level of compliance than the standard for all criteria (Table 4). The lowest levels were found in relation to setting the alert and action lines (34 per cent) and the same midwife signing the medical records (29 per cent).
The audit showed a lower level of compliance than the standard for all criteria.
The highest measured level of compliance was found in relation to initiating soft interventions in the event of a break in the alert line (97 per cent). There was one record in the audit and one in the reaudit that did not meet this criterion due to delivery taking place less than 15 minutes after the break in the alert line.
There was a statistically significant increase of 26 per cent from the audit to the reaudit in the number of records where the alert and action lines were set (p <0.001). Unadjusted, the odds of meeting this criterion were 2.9 times higher (95 per cent CI: 1.7–5.0) at the reaudit than at the audit.
Adjusting for the number of earlier pregnancies and the maternity ward, we found that the OR was equal to 2.1 (95 per cent CI: 1.1–3.7). The change in the use of the alert and action lines differed significantly between the maternity wards (p = 0.02). We found no statistically significant changes in compliance with the other criteria.
The audit showed that the alert and action lines were activated in accordance with the recommendations in one in every three medical records (criterion 1). The number of records where all entries were signed by the same midwife per shift had decreased (criterion 2).
However, in the cases where the alert and action lines were activated, soft interventions were almost always implemented (criterion 3). The criterion for documentation of the level and position of the fetal head in the birth canal and initiation of oxytocin infusion in the event of a break in the action line was less frequently met (criteria 4a and 4b).
However, in the cases where the alert and action lines were activated, soft interventions were almost always implemented.
After the implementation was completed, the reaudit showed a markedly higher compliance with criterion 1, from 34 per cent to 60 per cent, but only minor changes were seen for the other criteria.
Our strategy should be easy to implement
A synthesis of systematic reviews (29) shows that quality improvement measures must be adapted to the field of practice in order to have an effect. The measures must not be overly complex, and they should be limited in number. We devised a strategy that was easy to implement and that could be implemented using the clinic’s existing resources. However, a change in management ultimately led to a modification in the available resources.
This meant that we could not implement the strategy as planned. We prioritised the use of resources on a single criterion – correct use of the alert and action lines – and implementation of the measures was heavily dependent on motivated opinion leaders from the safe birth project.
The midwife in the audit project implemented the audit and feedback measure in one maternity ward.
The audit results concerning the use of the alert and action lines were used as a basis for giving feedback in informal conversations, in groups or one-to-one settings.
The midwife in the audit project directed the conversation and asked the participants about their practice in relation to the alert and action lines. No one was pressurised into giving specific examples of how they used the tool.
The aim was to stimulate reflection on their own practice in order to increase their awareness of the use of the alert and action lines.
Earlier studies have shown that the audit and feedback measure can have a small but important effect on changing practice (28). The effect seems to depend on who gives the feedback, the setting it is given in, and how it is given.
We had an intensive one-month implementation period, and we carried out the reaudit four weeks after implementation was complete.
The results of the reaudit show that practices changed after periods that entailed a major focus on a project, and where the implementation strategy had been carried out shortly before the reaudit. In order to identify the long-term changes of the implementation strategy, we would have had to continue the audit process.
Unsigned medical record entries were a surprising incidental finding
Some of the findings in our material were not directly related to compliance with the criteria. In both the audit and the reaudit, we found that unsigned medical records made it difficult to determine whether more than one midwife had made entries in the maternity patient record system.
The reaudit showed a higher percentage of medical records where all entries were signed by the same midwife.
Unsigned medical entries were a surprising incidental finding because this practice violates the documentation requirement in the Health Personnel Act (30). The reaudit showed a higher percentage of medical records where all entries were signed by the same midwife, but the change was not statistically significant.
The method we chose to map one-to-one midwifery support could identify who signed the medical record entries. However, it did not necessarily show who had actually made the observations or carried out the documented interventions, or whether the patient had as much access to the midwife as she wished.
Little variation in oxytocin stimulation
In an observational study at a Norwegian maternity ward, Bernitz et al. (10) concluded that nulliparous women in particular were stimulated with oxytocin without an obvious reason.
The percentage of variation in practice between midwives was thus low.
In a cross-sectional study from Sweden (11), 30 per cent of maternity patients without the diagnosis ‘slowly progressing labour’ were stimulated using oxytocin. The incidence of oxytocin stimulation without a break in the action line in the audit and the reaudit accounted for 4 per cent and 1.1 per cent of all records in our sample respectively.
The percentage of variation in practice between midwives was thus low. However, the samples on which these calculations are based are small, and we have not considered the causal factors. The results must therefore be interpreted with caution.
The standard was set at 100 per cent
The standard for compliance with all criteria was set at 100 per cent. No optimal level has been determined for compliance with the criteria in a clinical audit, but it should be achievable in order to motivate change (13).
The reason given for compliance with criterion 3 (initiating soft interventions in the event of a break in the alert line) failing to reach the set standard may be an indication that a standard of 100 per cent was neither realistic nor appropriate for this criterion. Nor does the standard take into account that the patient may want different interventions to those stipulated in the criteria.
It is common to set the standard at 100 per cent in cases where the criterion is considered to be of critical importance to patient treatment (13).
With the exception of criterion 4, which relates to treatment with a high-risk drug, our criteria can be considered important rather than critical. An initially lower and achievable standard for the other criteria may have facilitated a greater degree of compliance.
Strengths and weaknesses of the project
The knowledge base for determining the criteria was based on recommendations in the Obstetrics Guidelines, which were incorporated into the hospital trust’s internal guidelines. We could assume that the recommendations were already known to the midwives, but we also assumed that they did not necessarily feel a sense of ownership to them.
These two assumptions were a good starting point for a brainstorming session and discussion on the barriers to and drivers for the midwives changing their practice and for devising a customised implementation strategy.
However, unforeseen events meant that we were unable to implement the strategy as planned. It is reasonable to assume that these events may have affected our results.
The number of medical records included was lower than we calculated because, after a thorough review, it transpired that some did not meet the inclusion criteria, and therefore had to be excluded.
Nevertheless, our sample was large enough to reveal a statistically significant change in the use of the alert and action lines from the audit to the reaudit based on the methodological conditions we had set.
The audit showed that there was room for improvement in practices. We found a greater degree of compliance with the criterion that received the most attention in the implementation phase, but there was still a discrepancy between the set standard and the measured compliance for all the criteria.
The change in the midwives’ practices from the audit to the reaudit may have been influenced by the design of the project. The improvement measures we chose, and how we implemented them, are two important variables. Our choices of criteria and standard, and the duration of the project are also variables that may have affected the results.
Implications for further quality improvements
Changing the electronic maternity patient record system to require an electronic signature for all record entries may improve the quality of documentation.
Electronic reminders that prompt users to activate the alert and action lines may lead to this tool being used in accordance with evidence-based recommendations.
A more appropriate method should also be developed to map the incidence of one-to-one midwifery support – such as an observational study, a questionnaire or qualitative research interviews of patients and midwives.
The WHO recommendation concerning when the active phase of labour starts changed after we completed the project. Consideration must be given to changing the criteria before continuing the audit process.
Our thanks go to the Norwegian Midwife Association for providing financial support to complete the master’s thesis that forms the basis for this research paper.
1. Den norske jordmorforening. Yrkesetiske retningslinjer for jordmødre. Oslo: Den norske jordmorforening; 2016. Available at: http://www.jordmorforeningen.no/politikk/etikk (downloaded 31.03.2016).
2. Renfrew MJ, McFadden A, Bastos MH, Campbell J, Channon AA, Cheung NF, et al. Midwifery and quality care: findings from a new evidence-informed framework for maternal and newborn care. The Lancet. 2014;384(9948):1129–45.
3. Verdens helseorganisasjon (WHO). WHO recommendations: intrapartum care for a positive childbirth experience. Genève: WHO; 2018. Available at: https://www.who.int/reproductivehealth/publications/intrapartum-care-guidelines/en/ (downloaded 23.08.2019).
4. Helsedirektoratet. Utviklingsstrategi for jordmortjenesten. Tjenestekvalitet og kapasitet. Oslo: Helsedirektoratet; 2010.
5. Folkehelseinstituttet. Medisinsk fødselsregister og abortregister. Statistikkbank. F9/I3: Komplikasjoner under fødselen. Oslo: Folkehelseinstituttet; 2015. Available at: https://www.fhi.no/hn/statistikk/statistikkalenderen/statistikkbanker/ (downloaded 07.08.2020).
5. Folkehelseinstituttet. Medisinsk fødselsregisters (MFR) statistikkbank. Standardstatistikk. F9a: Komplikasjoner under fødselen. Oslo: Folkehelseinstituttet; 2015. Available at: https://www.fhi.no/hn/statistikk/statistikkalenderen/statistikkbanker/ (downloaded 07.08.2020).
6. Norsk gynekologisk forening. Veileder i fødselshjelp. Norsk gynekologisk forening; 2014. Available at: https://legeforeningen.no/Fagmed/Norsk-gynekologisk-forening/Veiledere/Veileder-i-fodselshjelp-2014/ (downloaded 11.12.2014).
7. Eggebø T, Rossen J, Ellingsen L, Heide HC, Muneer S, Westad S. Stimulering av rier. In: Norsk gynekologisk forening: Veileder i fødselshjelp. Norsk gynekologisk forening; 2014. Available at: https://legeforeningen.no/Fagmed/Norsk-gynekologisk-forening/Veiledere/Veileder-i-fodselshjelp-2014/Stimulering-av-rier/ (downloaded 11.12.2014).
8. Helse Vest RHF. Tiltakspakke: Juni 2016, Trygge fødsler. Bergen: Helse Vest RHF; 2016. Available at: https://helse-vest.no/Documents/Kvalitet/Innsatsomr%C3%A5der/Tiltakspakke-trygge-fodsler-juni-2016.pdf (downloaded 30.09.2017).
9. Bugg GJ, Siddiqui F, Thornton JG. Oxytocin versus no treatment or delayed treatment for slow progress in the first stage of spontaneous labour. Cochrane Database of Systematic Reviews. 2011;(7). DOI: 10.1002/14651858.CD007123.pub2
10. Bernitz S, Øian P, Rolland R, Sandvik L, Blix E. Oxytocin and dystocia as risk factors for adverse birth outcomes: A cohort of low-risk nulliparous women. Midwifery. 2014;30(3):364–70.
11. Nystedt A, Hildingsson I. Diverse definitions of prolonged labour and its consequences with sometimes subsequent inappropriate treatment. BMC Pregnancy and Childbirth. 2014;14(1):233.
12. Healthcare Quality Improvement Partnership. Best practice in clinical audit. London: Healthcare Quality Improvement Partnership; 2016. Available at: http://www.hqip.org.uk/resources/best-practice-in-clinical-audit-hqip-guide/ (downloaded 24.05.2017).
13. Ashmore S, Ruthven R, Hazlewood L. Stage 1: Preparation, planning and organization of clinical audit. In: Burgess R, ed. New best principles of best practice in clinical audit. Oxford, New York: Radcliffe Publishing; 2011.
14. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: Advancing guideline developement, reporting and evaluation in healthcare. CMAI. 2013;182:E839–42.
15. National Institute for Health and Care Excellence. NICE guideline 190, Intrapartum care: care of healthy women and babies. Oxford: NICE; 2014. Available at: nice.org.uk/guidance/cg190 (downloaded 26.04.2016).
16. Ehsanipoor RM, Satin AJ. Overview of normal labor and arrest disorders. UpToDate; 2015. Available at: http://www.uptodate.com (downloaded 25.10.2015).
16. Ehsanipoor RM, Satin AJ. Overview of normal labor and arrest disorders. UpToDate. 2015. Available at: https://www.uptodate.com/contents/overview-of-normal-and-arrest disorders. UpToDate; 2015. (downloaded 25.10.2015).
17. Helsedirektoratet. Et trygt fødetilbud. Kvalitetskrav til fødselsomsorgen. Oslo: Helsedirektoratet; 2010.
18. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.
19. CSAM Health AS. IMATUS Natus fødesystem. CSAM Health AS; 2016.
20. Scan-Med Norway. Neoventa Stan S31, CTG-enhet for fosterovervåking. Scan-Med Norway; 2014.
21. Sergeant E. Epitools epidemiologicl calculators. Ausvet; 2016. Available at: http://epitools.ausvet.com.au (downloaded 23.03.2016).
22. Random.org. Integer Set Generator. Dublin: Random org.; 2016. Available at: https://www.random.org/company/ (downloaded 18.03.2016).
23. IBM Corp. IBM SPSS Statistics for Windows, versjon 23.0. Armonk, New York: IBM Corp.; 2015.
24. The R-Foundation. The R Project for Statistical Computing. The R-Foundation; 2015. Available at: https://www.r-project.org/foundation/ (downloaded 28.03.2016).
25. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science. 2013;8(1):35. Available at: https://doi.org/10.1186/1748-5908-8-35 (downloaded 16.01.2018).
26. Flodgren G, Parmelli E, Doumit G, Gattellari M, O'Brien MA, Grimshaw J, et al. Local opinion leaders: effects on professional practice and health care outcomes (Review). Cochrane Database of Systematic Reviews. 2011;(8). DOI: 10.1002/14651858.CD000125.pub4
27. Cheung A, Weir M, Mayhew A, Kozloff N, Brown K, Grimshaw J. Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior. Systematic Reviews. 2012;1(1):36. DOI: 10.1186/2046-4053-1-36 (downloaded 16.01.2018).
28. Ivers NM, Grimshaw JM, Jamtvedt G, Flottorp S, O’Brien MA, French SD, et al. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. Journal of General Internal Medicine. 2014;29(11):1534–41. DOI: 10.1007/s11606-014-2913-y (downloaded 16.01.2018).
29. Squires JE, Sullivan K, Eccles MP, Worswick J, Grimshaw JM. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals' behaviours? An overview of systematic reviews. Implementation Science. 2014;9(1):152. DOI: 10.1186/s13012-014-0152-6 (downloaded 16.01.2018).
30. Lov 2. juli 1999 nr. 64 om helsepersonell m.v. (helsepersonelloven). Available at: https://lovdata.no/dokument/NL/lov/1999-07-02-64 (downloaded 13.03.2019).