New form is suitable for assessing clinical nursing education

In clinical nursing education, it is a common challenge internationally to ensure that the learning outcome is achieved, and several methods of assessing this have been tested (1–3). The Bologna declaration is a process that began in 1999, and the Qualifications Framework for Higher Education was introduced in Norway in 2005.

Preparation of a qualifications framework (EQF, European Qualifications Framework) for all levels of the educational system began in 2006, and in 2009 Norway decided to endorse this instrument (5). The introduction of the framework affected both education and learning in clinical nursing education, as well as assessment and tools used for assessment.

There has been an ongoing discussion within bachelor-level nursing education about where students should learn, how learning can best take place, and how the learning outcome should be assessed (2, 6, 7). Universities Norway (UHR) recommends focusing on research on education in health and social care subjects at university level, with special emphasis on work methods and learning outcomes in clinical education (8).

This article focuses on students and preceptors in clinical practice and their experience with and views on a recently developed assessment tool for clinical education.

Previous research

International research shows that it is challenging to assess students in all health-related professions that include clinical education as a core component (1, 9). In addition, research shows that preceptors in clinical education are seeking comprehensible standards to use in their assessments and that a lack of standards and ambiguously formulated learning outcomes make it difficult to give students a failing mark (10, 11).

It is a well-known problem that educational institutions’ documents may be difficult to understand for those who are required to use them (2, 12). Research shows that far fewer students pass theoretical examinations than clinical education programmes. There is a call for processes that further develop assessment criteria that are relevant, manageable and robust enough to distinguish between the competence level of the various students in clinical education (10, 11).

Some of the problem lies in ambiguously formulated learning outcomes in the assessment document (11). Research shows that a lack of clear criteria for satisfactory completion of clinical education is an international problem, and more studies have described the problems associated with assessing clinical nursing competence than have proposed solutions (3, 13).

Assessments

Assessment entails a normative or value-related judgment based on expectations, requirements and criteria (14). It could be argued that assessing students in clinical education has a great deal in common with assessing the achievement of goals. In fact, the learning outcomes in the student’s educational plan are the goals that the student is expected to achieve and that he or she is assessed in relation to. Assessing achievement of goals requires clear criteria regarding what is needed in order to reach the goals (15).

The aim of a summative assessment is to compare the student’s knowledge, skills and general competence with standards that the student is expected to achieve. The assessment may also be formative and put greater emphasis on the student’s progress during and throughout the clinical education programme (16).

A formative assessment must also clarify how the student can achieve the expected learning outcome, thus promoting more in-depth learning and motivation and fostering self-regulated learning. Feedback is a key aspect of formative assessment (14, 16, 17).

Form for assessing learning outcomes

The Assessment for Clinical Education (AssCE) was developed in Sweden as a result of the Bologna declaration (1, 18). After 2000, it was further developed into an assessment form that is used in countries such as Norway, Sweden and Finland. The form was developed to ensure that students are assessed on the basis of their learning outcomes in the clinical education portion of the bachelor’s degree in nursing.

The AssCE form contains 21 different factors or questions divided among five areas in which the student is assessed. The five areas are ‘Communication and teaching’, ‘The nursing care process’, ‘Examinations and treatments’, ‘Management and cooperation’ and ‘Professional approach’.

Examples of factors are ‘Participates by carrying out examinations and treatments’ and ‘Administers medications’, which are factors under the area ‘Examinations and treatments’. Each factor is assessed on a three-level scale: ‘Inadequate achievement of goals’, ‘Good achievement of goals’ and ‘Very good achievement of goals’.

The assessment is conducted in accordance with the description of learning outcomes in the student’s educational plan. From the middle to the end of the clinical education programme, the teacher places a check next to each factor, which highlights the student’s progress within the various factors. The content and design of the AssCE form are the same for all clinical education (1, 19).

Use of the form

When the students use the AssCE form, they must complete a self-assessment and place themselves on the assessment scale in preparation for the performance review. A major study describes experiences of using the AssCE form after the researchers analysed the questionnaire given to preceptors and nursing students at two Swedish universities.

The study concludes that each factor which clarifies achievement of goals should have a better description (1). Clear criteria for achievement of goals makes it possible to distinguish and discuss the students’ progress within the various areas of clinical education. The study also shows that information about use of the form is essential for being able to use it (1).

Alternative assessment form

In the spring of 2015, a university college in Norway created an alternative assessment form based on the AssCE form (1). Unlike the AssCE form, this form contains clearly defined learning activities to clarify what is expected of the student. The aim is to achieve consensus and validity in the assessment (3, 11).

The learning outcomes in the educational plan have been formulated to encompass what the student is expected to do. These learning outcomes in the plan show the progression throughout the course of study so that clarification of the learning outcomes of required learning activities will ensure progression.

For example, the learning outcome under ‘Skills’ in the educational plan is described as follows: the student ‘can apply knowledge about crises and crisis response, and demonstrate insight into the experiences and reactions of patients and their families in the event of acute and critical illness’. Table I shows an example of what achievement of goals implies for the learning outcome for ‘Skills’, as well as the scale for the assessed level of competence.

This form is different from the AssCE form (1) in that it has different content for the various periods of clinical education. The form’s assessment scale uses a five-level Likert scale from ‘Unsatisfactory’ to ‘Highly satisfactory’ for each criterion related to achievement of goals.

The students prepare for their performance review in the same manner as when the AssCE form is used (1). The form provides space for more extensive comments about special circumstances or relevant tasks that the student should work with to satisfy the criteria for achievement of goals.

Critique of methods

The literature describes various methods that have been developed and tested for assessing students’ clinical competence, but these methods have been criticised for giving too much latitude to the preceptor’s subjective interpretation of competence and assessment of students (7, 20, 21). Research shows that it is difficult to find clear assessment criteria that are interpreted in the same way by students, preceptors and teachers (7, 21).

Consequently, there is a need for an assessment form for clinical nursing education that uses clear, unambiguous descriptions of learning outcomes and criteria for achievement of goals to ensure that the learning outcome is achieved (22).

Objective of the study

The objective of our study is to evaluate students’ and preceptors’ views on a new assessment form for clinical nursing education that is based on the principles of the AssCE form.

The research questions in this study are as follows:

• Does the assessment form clearly show the student’s progression in the clinical education programme?
• Does the new assessment form satisfy the requirement for clear criteria for how the students can achieve the learning outcome goals?
• Does use of a scale with descriptions of level facilitate discussion in the performance review?
• Is enough information given on the assessment form to enable its use?

Method

Questionnaire

This study has a descriptive design based on a questionnaire which was developed by Löfmark and Thorell-Ekstrand to assess the AssCE form (1), and which has been adapted to the assessment form in our study. To ensure face validity, three teaching colleagues read and assessed the questionnaire, and found it to be relevant.

The same version of the questionnaire was used for students and preceptors so that we could compare their responses in the survey. The preceptors were asked additional questions about how long they had worked as nurses, the number of times they had served as a preceptor, and any preceptor training they had undergone (1).

A total of eight Likert scale responses from 1 = ‘Strongly disagree’ to 5 = ‘Strongly agree’ are shown in Table 2.

In addition to the eight statements, the questionnaire consisted of four questions. These questions were adapted to students and preceptors, respectively, and one of the questions asked if the student had completed a self-assessment on a separate form prior to the performance review. The response options were ‘Yes’ and ‘No’.

One question was whether the student or preceptor had incorporated views from various preceptors in the assessment, and another asked whether the assessment form could affect the quality of the health services. These two questions had four response options: ‘Always’, ‘Almost always’, ‘Seldom’ and ‘Never’. The final question asked whether anything in the form should be developed – with the opportunity to write a free response.

Sample

In the autumn of 2016, we invited all the students (n = 158) in their first hospital training period in their second year of studies, as well as their preceptors in clinical education (n = 18), to take part in the survey after they had used the assessment form. The clinical education spaces were divided amongst four hospitals in Norway – both medical and surgical departments.

The students took part in clinical education in two subsequent groups. Some of the clinical education spaces had head preceptors, which means that they were responsible for supervising and assessing all the students (2 –14) in the relevant department. Other students had contact nurses with supervisory responsibility for one student.

Data analysis

We used Microsoft Excel to analyse the data. The data were analysed using descriptive statistics which reveal the frequency-related and summative features of the students’ and preceptors’ assessments of the assessment form that was used (23). We reported data from students and preceptors in both percentages and numbers to facilitate comparison (Table 2).

Not all the respondents answered all the questions, but there was no systematic drop-out. The responses to the open-ended questions consisted of individual statements that we did not believe constituted material sufficient for a content analysis, but we read through the statements and used them to support our findings.

Ethical considerations

The study falls outside the mandate of the Regional Committees for Medical and Health Research Ethics (REK), as it does not involve information related to the participants’ own or others’ health. The Norwegian Centre for Research Data (NSD) found the study to be ‘not subject to notification’ with the explanation that all the information is anonymous and individuals cannot be identified.

All the participants signed a declaration of consent in which they agreed to take part in the survey. In addition, they agreed to the use and publication of the information that they provided, and they could withdraw from participation without giving a reason. The survey was carried out after the final assessment to prevent the uneven power relationship between students and teachers from affecting the students’ responses.

Students who did not pass the clinical education programme were excluded from the survey, as they were in a situation that could influence their responses. The participants’ responses were anonymous and confidentially was ensured by placing the completed questionnaires in an envelope that was collected by teachers at the university college.

Results

We invited a total of 158 students to participate in the survey. Seven students were not asked to take part because they did not pass the clinical education programme. Thus, we have based our results on the fact that 129 of 151 students responded and that all 18 preceptors responded. Not all the students responded to all the statements and questions, resulting in fewer than 129 responses to the four questions. There was only one question that not all the preceptors answered (Table 2).

The 18 preceptors had nursing experience ranging from a few months to 45 years, and 15 preceptors had more than ten years of experience. Ten of them had undergone preceptor training. The number of times they had served as a preceptor varied from one to 40.

Can use the form as a model

An overview of responses from the students and preceptors about use of the assessment form with an emphasis on progression, learning outcome and criteria for establishing the level of competence is shown in Table 2. Altogether 73 per cent of the students said they agree (score 4 or 5) with the statement that the assessment form can be used as a model in various clinical education programmes in nursing education. About the same percentage said that the scale makes it possible to show the student’s progress.

Slightly more than 60 per cent of the students agreed with the statements about learning outcome and the assessment form. About the same percentage of students agreed with the statements about criteria for establishing the level of competence, and 64 per cent of the students said that the information about the assessment form was sufficient.

When we analysed the preceptors’ responses to the questions about progression, more than 90 per cent generally agreed that the assessment form can be used as a model in various clinical education programmes in nursing education and that the scale makes it possible to show the student’s progress.

Roughly 80 per cent of the preceptors generally agreed with the statements about learning outcome and the assessment form, and about 90 per cent gave a score of 4 or 5 on the statement about criteria for establishing the level of competence. Almost all the preceptors regarded the information about the assessment form as sufficient.

Preceptors gave higher scores

The average score on all statements was slightly higher for the preceptors than for the students. Almost all the students and preceptors responded ‘yes’ to the question about whether the students had prepared for the assessment. Half of the preceptors said that the views of other preceptors had influenced their assessment of the students, which may indicate that many believe that student assessment is a joint responsibility.

Some open-ended questions at the end of survey were included to capture new thoughts and ideas. The responses from the students and preceptors dealt with difficulties in understanding some of the phrases used in the questionnaire. Examples of such phrases were ‘the overall description’ and ‘achievement of goals of the assessed level of competence’.

Other comments indicated that it was important for the university college teachers to have a common understanding of how they should use the assessment form. Some students would have liked to receive comments with positive feedback.

Discussion

The difference between the AssCE form and the assessment form in this study lies in the clarity of the criteria for achievement of goals, i.e. the specific actions that students are expected to perform and that form the basis for the assessment.

Both students and preceptors found that it was helpful to make learning outcomes more specific, which is consistent with Benner’s proposed changes for improving nursing education (6). Benner also stresses the importance of making knowledge more concrete and referring to clinical examples (6).

The high average scores on the responses regarding progression may indicate that both students and their preceptors believe that progression was clearly defined in the assessment form (Table 2) and show a concurrent positive response. The response to the statement about progression is slightly higher in this survey than in the response to the statement about progression in the study conducted by Löfmark and Thorell-Ekstrand (1).

Distribution of responses to statements

The responses to the statements about learning outcome and the assessment form have a wider distribution than those in the other groups of statements. In Löfmark and Thorell-Ekstrand’s study, about 50 per cent of both the students and preceptors agreed that it worked well to combine achievement of goals for the various assessed levels of competence in the assessment form with the learning outcome for the period (1).

In this study, two-thirds of the students and almost 90 per cent of the preceptors agreed with this statement, which shows a more positive response than in Löfmark and Thorell-Ekstrand’s study (2). A possible explanation for this difference may be that the AssCE form requires the teacher to clarify what is expected of the student, while the assessment form used in this study provides a more specific description of what is expected.

Likes to use the scale for assessing the level of competence

Regarding the criteria for assessing the level of competence, the responses show that the participants generally agree that the descriptions of the levels clarify and enable a discussion of the levels of knowledge, skills and general competence. The responses also show that when the scale is used, the discussion regarding assessment can be improved.

We regard the responses about assessment of the level of competence as an indication that a scale with levels is viewed as a useful tool in the assessment process. A strength of the assessment form is that the students can actively participate in their own assessment, something which is desirable and can promote learning (17). This finding is consistent with findings in the study of the AssCE form (1).

It could be argued that use of the scale for assessing the level of competence aligns with principles for a formative assessment in that the student’s progress is emphasised and highlighted and that it clarifies what is expected in order for the students to achieve the learning outcome (14).

Received enough information to be able to use the form

The assessment form also contains a summative assessment, as the final assessment is an assessment of whether the students’ knowledge, skills and general competence meet expectations (14).

The information in the assessment form was sufficient, as more than 80 per cent of the participants agreed that enough information was provided to be able to use the form. In Löfmark and Thorell-Ekstrand’s study, about half of the preceptors agreed that they had received enough information, and therefore the researchers concluded that more information was needed about how both students and preceptors should use the AssCE form (1).

The finding that the assessment form in this study provided sufficient information indicates that the form is successful, as research shows that forms should as far as possible be self-explanatory (10, 11).

Discussion of method

We clearly defined the inclusion criteria in the study by having all the students in their first hospital training period and their preceptors take part in the trial. Only students who did not pass the clinical education programme were excluded from the study. The students who participated were a strategic sample in the sense that they were the ones who had this period of clinical education in autumn 2016 in their course of study (n = 158). Thus, they can be regarded as a representative sample of nursing students in their first hospital training period. All the preceptors were nurses whose task was to assess students in autumn 2016 (n = 18).

A systematic bias in this study may be the differences within the group of preceptors, as some had undergone preceptor training and others had not. This may have affected their responses since those with preceptor training are more knowledgeable about what is important in the preceptor role. In addition, the preceptors had different lengths of preceptor experience (1–40 times), which gave breadth to the sample. Gender and age in the sample were not taken into account, as these variables were not deemed to have an effect on the responses.

Conclusion and implications for clinical practice

The main finding of this study was that nursing students and their preceptors in clinical education appear to have a concurrent, positive view that the new assessment form can be used as a model for assessing students in clinical education. Our study shows that an assessment form with operationalised learning outcomes and assessed levels of competence is regarded as a good tool to use in the assessment process for students and preceptors and that the information about use of the assessment form was instructive.

The form can be said to contain a description of relevant, clinical learning situations, and can thus be an important contribution to quality assuring the work involved in identifying students who have not achieved the expected learning outcome. This can be illustrated by the statement of a newly hired assistant professor: ‘The student’s weaknesses showed up very clearly’.

References

1.          Löfmark A, Thorell-Ekstrand I. Nursing students' and preceptors' perceptions of using a revised assessment form in clinical nursing education. Nurse Education in Practice. 2014;14(3):275–81.

2.          Bjerknes MS, Christiansen B. Praksisveiledning med sykepleierstudenter. Oslo: Gyldendal Akademisk; 2015.

3.          Yanhua C, Watson R. A review of clinical competence assessment in nursing. Nurse Education Today. 2011;31(8):832–6.

4.          NOU 2000:14. Frihet med ansvar – Om høgre utdanning og forskning i Norge. Oslo: Statens forvaltningstjeneste, Informasjonsforvaltning; 2000. Available at: https://www.regjeringen.no/contentassets/b27a1ef056e64f9d85e3a93460710ac4/no/pdfa/nou200020000014000dddpdfa.pdf(downloaded 05.12.2018).

5.          Kunnskapsdepartementet. Nasjonalt kvalifikasjonsrammeverk for livslang læring (NKR). Oslo; 2011. Available at: https://www.regjeringen.no/globalassets/upload/kd/vedlegg/kompetanse/nkr2011mvedlegg.pdf(downloaded 05.12.2018).

6.          Benner P, Heggen K. Å utdanne sykepleiere: behov for radikale endringer. Oslo: Akribe; 2010.

7.          Boud D, Garrick J. Understanding learning at work. London og New York: Routledge; 2012.

8.         Universitets- og høgskolerådet. Kvalitet i praksisstudiene i helse- og sosialfaglig høyere utdanning: Praksisprosjektet. Oslo; 2016. Available at: https://www.uhr.no/_f/p1/i0311e40a-4465-4a77-a3f3-565762627e15/2016-praksisprosjektet_sluttrapport.pdf (downloaded 25.08.2017).

9.          Cleland JA, Knight LV, Rees CE, Tracey S, Bond CM. Is it me or is it them? Factors that influence the passing of underperforming students. Medical Education. 2008;42(8):800–9.

10.        Hunt LA, McGee P, Gutteridge R, Hughes M. Assessment of student nurses in practice: A comparison of theoretical and practical assessment results in England. Nurse Education Today. 2012;32(4):351–5.

11.        Hauge KW. Veilederes beveggrunner for å unnlate å gi ikke-bestått i praksisstudier til tross for at læringsutbyttene ikke er nådd. Nordic Journal of Nursing Research. 2015;35(1):20–8.

12.        Alvsvåg H, Førland O. Sykepleierutdanningen i lys av nyutdannedes yrkeserfaringer. Vård i Norden. 2006;26(3):34–8.

13.        Wangen MG, Torjuul K, Sorlie V. Hva praksisveiledere vektlegger i vurderingen av sykepleierstudentenes kliniske kompetanse-en kvalitativ intervjuundersøkelse. Nursing Science & Research in the Nordic Countries. 2010;30(4):43.

14.        Eriksson BG, Karlsson PÅ. Att utvärdera välfärdsarbete. Stockholm: Gothia; 2008.

15.        Sverdrup S. Evaluering : faser, design og gjennomføring. Bergen: Fagbokforlaget; 2002.

16.        Taras M. Assement-summative and formative – some theoretical reflections. British Journal of Educational Studies. 2005;53(4):466–78.

17.        Raaheim A. Læring og undervisning. Bergen: Fagbokforlaget; 2011.

18.        Löfmark A, Thorell‐Ekstrand I. An assessment form for clinical nursing education: a Delphi study. Journal of Advanced Nursing. 2004;48(3):291–8.

19.       Engstrøm M, Löfmark, Ugland Vae KJ, Mårtensson G. Nursing student's perceptions og using Clinical Education Assessment tool AssCE and their overall perceptions of the clinical learning environment. A cross-sectional correlational study. Nurse Education Today. 2017;51(1):63–7.

20.        Haddeland K, Söderhamn U. Sykepleierstudenters opplevelse av veiledningssituasjoner med sykepleiere i sykehuspraksis – En fenomenologisk studie. Nordisk sygeplejeforskning. 2013;3(01):18–32.

21.        Taras M. Student self-assessment: processes and consequences. Teaching in Higher Education. 2010;15(2):199–209.

22.        Hauge KW, Brask OD, Bachmann L, Bergum IE, Heggdal WM, Inderhaug H, Grønvik CKU, Julnes SG. Kvalitet i praksisstudier i sykepleier- og vernepleierutdanning. Nordisk tidsskrift for helseforskning. 2016;12(1):19–33.

23.        Bjørndal A, Hofoss D. Statistikk for helse- og sosialfagene, eds. Oslo: Gyldendal Akademisk; 2004.