Patient experiences following day surgery – translation and validation of the Quality of Recovery-15 (QoR-15nor) questionnaire

An increasing number of complicated operations can be performed as day surgery thanks to advancements in anaesthesiology and surgical techniques (1). Day surgery is defined as elective surgery among hospitalised patients who are discharged the same day.

The typical day-surgery patient remains in hospital for four to six hours, while more complicated procedures require a somewhat longer hospital stay. Nationally as well as internationally, political and medical spheres are focused on increasing the percentage of patients whose operations are performed as day surgery. The purpose is to reduce the need for overnight hospitalisation. Twenty years ago, day surgery comprised 25 per cent of all operations, while today it accounts for 60 per cent of all elective procedures (2, 3).

The goal for the period following day surgery is for patients to return as soon as possible to the same physiological condition they were in before the procedure. As a result, day-surgery patients need encouragement to get started quickly with nutritional intake and mobilisation so they can be discharged as soon as possible.

Early discharge can present challenges related to handling the negative effects of the operation, such as pain and postoperative nausea and vomiting (PONV), and to preventing complications (4). For example, a recently published Norwegian study found that 16 per cent of day-surgery patients experience PONV (5).

Patient experience

Patient experience adds another dimension of clinical outcome measures, such as health-related quality of life, presence of infection or rehospitalisation, and is a recognised parameter for measuring health service quality.

The terms ‘satisfaction’, ‘preferences’ and ‘perception’ are used as well, although they include other aspects of patients’ perspectives on treatment and care. Preferences entail what patients prefer or expect, while satisfaction is described as the gap between ‘expectations’ and ‘experiences’ (6).

Patient-reported experiences are regarded as less subjective than reported satisfaction (6–8). Low patient satisfaction following day surgery has been associated with 30 days of rehospitalisation as well as with various postoperative complications (9–11).

Many studies of day-surgery patients have been conducted in recent decades. These have shown that 95 per cent of day-surgery patients are satisfied, both at discharge and after 30 days. However, only 75 per cent of patients are completely satisfied at discharge, and this decreases to 62 per cent after 30 days (10).

Patient-reported outcome measures

Another aspect related to patient experiences is patient-reported outcome measures, such as physical and psychological well-being. Several different questionnaires for exploring patient-perceived quality have been developed, including the Postoperative Quality of Recovery Scale (12).

The most recognised instrument is the Quality of Recovery-40 items (QoR-40), a questionnaire developed by Myles et al. that originally contained 40 questions for exploring postoperative patient symptoms (13).

The QoR-40 has been translated into and validated in several languages, including Turkish (14), Japanese (15), Danish (16) and Swedish (17). Stark et al. subsequently shortened the form to 15 questions, known as the QoR-15. The questionnaire shows high validity and reliability, also in the abbreviated version (18).

The QoR-15 contains 15 questions divided into two dimensions: ‘physical well-being’ and ‘psychological well-being’. Patients score their experience on a scale from 0 (= none of the time) to 10 (= all of the time), where negative statements are reversed. This gives a maximum score of 150.

The systematic literature review of the QoR-15 conducted by Kleif et al. (19) found that the questionnaire has high validity and internal consistency (Cronbach’s alpha = 0.836) as well as an intraclass correlation of 0.989. The study is the first systematic review of the QoR-15 that is in accordance with the ‘Consensus-based Standards for the Selection of Health Measurement Instruments’ checklist (COSMIN).

The COSMIN checklist is used to evaluate the methodological quality of studies of the measurement properties of outcome measurement instruments. Kleif et al. have concluded that the QoR-15 meets the requirements for instruments that measure patient outcomes in clinical trials.

Norwegian questionnaires

The Norwegian Knowledge Centre for the Health Services designed the questionnaire titled ‘User experiences with outpatient surgery, adults’ (20), which contains 24 questions divided into five categories: overall assessment, accessibility and reception, treatment and care, information and the period following discharge.

The questionnaire does not cover areas such as physical and psychological well-being. The QoR-15 could provide answers about other aspects of patient experiences following day surgery, but it had not been translated into Norwegian.

Objective of this article

The objective of this article is to present the translation and validation of the Quality of Recovery-15 items (QoR-15) questionnaire in Norwegian.

Method

Translation and validation were carried out as part of a larger study of patient experiences following day surgery. The main study was a quantitative, longitudinal, multi-centre study that included three different hospitals.

Setting and sample

Table 1 presents an overview of the hospitals’ service region and level of day-surgery activity in 2017.

All the patients at the respective hospitals who were scheduled for gynaecological surgery under general anaesthesia were invited to take part in a questionnaire survey. The inclusion criteria were patients over 18 years of age who were able to give their verbal and written consent to participate.

The exclusion criteria were psychiatric conditions or cognitive decline documented with a diagnosis, rehospitalisation within 72 hours of the operation, and diagnosed neuromuscular disease. Data were collected on the first postoperative day and four weeks following the operation.

Guidelines on questionnaire validation recommend a participant-to-question ratio of 5:1, 10:1 and 30:1, which implies a variation from 75 to 450 participants in such studies (21). In this validation study, we included the answers from the respondents as of August 2019 in the analysis (N = 197).

Translation

There is no consensus about which method should be used to translate instruments or questionnaires to other cultural settings, but most researchers agree that merely translating and then using them is inadequate. We used a forward and backward translation process in line with theoretical recommendations (22):

1. Two nurse anaesthetists whose first language is Norwegian translated the English version of the QoR-15 to Norwegian independent of each other. The researchers in the main study (11 nurse anaesthetists) compared the two translations and combined them to create a Norwegian version of the QoR-15.
2. Next, a nurse anaesthetist whose first language is English translated the Norwegian version of the QoR-15 back to English. 
3. Finally, three independent nurse anaesthetists compared the Norwegian and English versions, focusing on correspondence between the two versions with respect to semantics and concepts.

We used a forward and backward translation process in line with theoretical recommendations.

An expert panel comprised of 11 healthcare professionals with many years of experience in postoperative care and treatment assessed whether the questions in the form were relevant, logical and comprehensible. The expert panel gave written feedback to the researchers, who then discussed the matter until consensus was reached on the final version of the QoR-15nor. The discussions took place via email and in meetings.

Pilot testing

In the next phase, we conducted a pilot test of the Norwegian version of the QoR-15 (QoR-15nor) that focused on face and content validity. Ten randomly selected patients, who were recruited from the user sample and in the respective departments, were asked to assess whether the questionnaire was adequate, suitable and comprehensible in terms of language, instructions and scoring. They gave verbal feedback to the nurses in the study (21).

Data collection – main study

The patients received written information about the study well in advance of the day of surgery and brought with them a signed consent form to participate in the study when they were admitted to hospital. The nurses in the study administered the questionnaire in a telephone interview on the first postoperative day and entered the responses into an online form. The responses were transferred directly to a database in the Services for Sensitive Data (TSD), which requires two-factor authentication in order to log in.  

The patients received an email with a link to the questionnaire about four weeks after the surgery. These responses were also transferred directly to the database in TSD. Patients who did not have or use email could answer the questions on a paper form and were given a pre-stamped envelope to take home with them.

The questionnaire consisted of three parts:

1. Socio-demographic variables such as age, gender, living situation (live alone, live with others), civil status (married/domestic partner, single, widowed, in a relationship), work situation (employed, not employed) and educational background (compulsory school, upper secondary, higher education).  
2. The validated EuroQoL 5-dimension 3-level questionnaire (EQ-5D-3L), a form for reporting self-rated health. The questionnaire is divided into five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression, and patients score their experience as 0 = no problems, 1 = some problems, 2 = severe problems. The EQ-5D-3L is a recognised instrument that gives a descriptive profile of self-rated health, which the health services use in clinical and financial evaluations. The form also consists of a visual analogue scale, where 0 = ‘the worst health you can imagine’ and 100 = ‘the best health you can imagine’. The score is calculated as an EQ-5D index score (23).
3. Norwegian version of the Quality of Recovery-15 (QoR-15nor) questionnaire (see Figure 1).

After four weeks, we also included the Norwegian questionnaire ‘User experiences with day surgery, adults’ (20).

Analysis

We used descriptive statistics, mean/median and standard deviation (SD) to describe the sample. The Kruskal-Wallis test was used to investigate any demographic differences among the sample and between the hospitals. Inter-item correlations between the questions were measured using Spearman’s correlation coefficient (rho).

Internal consistency was measured with Cronbach’s alpha, where a score ≥ 0.9 = excellent, 0.8–0.89 = good, 0.7–0.79 = acceptable and < 0.7 = poor. We assessed the test–retest using Cohen’s unweighted kappa, where < 0.2 = poor, 0.2–0.4 = acceptable, 0.4–0.6 = moderate and 0.6–0.8 = good agreement.

Ethics and privacy protection

The main study was approved by the Regional Committees for Medical and Health Research Ethics (REK), project number 2018/985, and by the Norwegian Centre for Research Data (NSD), project number 416326.

The study was conducted in accordance with the principles set out in the Declaration of Helsinki and on voluntary, informed, written consent. We used the Services for Sensitive Data (TSD) at the University of Oslo for secure data storage.

Results

Translation

Feedback from patients (n = 10) showed that the questionnaire was adequate, relevant with regard to the explored symptoms, and comprehensible in terms of language, instructions and scoring. The questionnaire did not need to be changed after pilot testing.

The questionnaire did not need to be changed after pilot testing.

Participants in the analysis

Table 2 provides an overview of the participants included in the validation study.

The table shows that the average age of the participants ranged from 42.9 to 50.4 years. Most were married and had a higher education. Additionally, the majority of patients were assessed as being in ASA Class II. There were no significant differences among the participants at the three hospitals.

Validity and reliability

Tables 3 and 4 show the inter-item correlations between the questions in the QoR-15nor for the first postoperative day and four weeks after surgery, respectively.

Inter-item correlations of the QoR-15nor showed acceptable to good reliability in most areas. For questions 1–10, the correlation varied between 0.17 and 0.69 on the first postoperative day and between 0.38 and 0.88 four weeks after surgery. The correlations between these questions were also significant.

A low correlation was shown in areas with no logical connection, e.g. between ‘been able to enjoy food’ or ‘feeling rested’ and having ‘moderate/severe pain’, ‘nausea or vomiting’ or ‘feeling worried or anxious’. The tables show that the correlations were stronger four weeks after surgery than on the first postoperative day.

Internal consistency measured with Cronbach’s alpha was 0.56 on the first postoperative day and 0.89 four weeks after surgery. Test–retest reliability measured with Cohen’s unweighted kappa statistic was 0.41.

Discussion

This article presents the process of translating and validating the Norwegian version of the Quality of Recovery 15 questionnaire. Direct content validity among participants of the pilot test suggests that the questionnaire is comprehensible and relevant and that the scoring is logical.

We found low internal consistency measured with Cronbach’s alpha on the first postoperative day (0.56). Four weeks after surgery, however, the questionnaire’s internal consistency was good (0.89). This result contradicts the findings of previous studies (17, 19), which may be related to the data collection method.

This may indicate that self-reporting produces the highest internal consistency within the QoR-15nor.

On the first postoperative day, the nurses in the study phoned the patients and filled in the questionnaire. Four weeks after surgery, the patients themselves marked the boxes on the form. This may indicate that self-reporting produces the highest internal consistency within the QoR-15nor.

Comparison with previous studies

The inter-item variations differed both between questions and between the first postoperative day and four weeks after surgery. Previous studies have also shown variation in inter-item correlations (24, 25).

In previous studies, test–retest reliability was measured by filling in the questionnaire after 24 and 48 hours, respectively (17). Our test–retest reliability was measured between the first postoperative day and four weeks after surgery, but it was nonetheless acceptable to moderate.

Feedback on the use of the QoR-15nor

The nurses in the study reported that the questionnaire was easy to use after some training. They said the challenge was that the patients often talked about things other than what they were asked about.

We used both the EQ-5D-3L and the QoR-15nor on the first postoperative day, and the patients said they felt some of the questions overlapped, such as those about pain and feelings of sadness or depression. In hindsight, we might have considered using a different scale for self-assessed health.

The QoR-15nor questionnaire works well

Our findings suggest that the QoR-15 is a relevant, logical and comprehensible questionnaire that can be used to explore postoperative symptoms following day surgery. The QoR-15 has been translated into and validated in a number of languages, including the Scandinavian languages of Swedish and Danish (16, 17). All the published studies of the questionnaire have concluded that it is valid, reliable and suitable for exploring postoperative symptoms.

The QoR-15 is a relevant, logical and comprehensible questionnaire that can be used to explore postoperative symptoms following day surgery.

The validation studies have explored the associations between other factors, such as type of operation, duration of the procedure, the amount of time spent in the postoperative ward, and the length of hospitalisation. We plan to extract this data later in our main study.

More findings to come

In the main study, we want to investigate whether factors such as gender, age, ASA classification, height/weight, comorbidity, self-assessed health, medications administered (analgesics, anti-nausea) and the amount of time spent in the postoperative ward can be associated with the QoR score.

A study from 2011 found that gender and perceived anxiety were significantly associated with reported QoR (26). Other factors shown to have a negative effect on patient experiences are the type of surgery, younger people and those out of work, duration of the operation, women and laparoscopic cholecystectomy (27–29).

Pain has been shown to be one of the most common reasons that patients contact the health service after they return home, followed by the need for additional information (28, 29). In the main study, we have also explored the entire clinical pathway of the patients and compared this with any complications to gain more insight into day-surgery patients as of 2019–2020.

When studying patient experiences, it is important to be aware of what data is actually being sought. Patient experiences with waiting time, finding their way to the right department, and information before and after the operation generate a different type of information than patient-reported symptoms. This information is included in the questionnaire developed by the Norwegian Knowledge Centre for the Health Services (20).

The information we gain from using the QoR-15nor could also be linked to the administration of analgesics and anti-nausea medications, the type of operation, surgical techniques and other factors more specific to the anaesthesia and postoperative ward, and this is something we want to investigate in the main study.

Conclusion

Until now, there have been no Norwegian questionnaires that explore patient-reported postoperative symptoms following day surgery. Findings in this article show that the QoR-15nor has good face validity, internal consistency and reliability. Our conclusion is that the QoR-15nor questionnaire is suitable for charting patient experiences following day surgery.

References

1.      Quemby D, Stocker M. Day surgery development and practice: key factors for a successful pathway. Cont Educ Anesth Crit Care Pain. 2014;14(6):71442.

2.      Lieng M, Busund B, Ræder J, Iversen T. Innsatsstyrt finansiering og dagkirurgi. Tidsskr Nor Legeforen. 2013;133:974–6.

3.      Helse- og omsorgsdepartementet. Samdata spesialisthelsetjenesten 2011. Oslo: Helse- og omsorgsdepartementet; 2012. Available at: https://ekstranett.helse-midt.no/1001/Presentasjoner/Samdata%20spesialisthelsetjenesten.pdf (downloaded 18.11.2020).

4.      Fehrman K, Mathews C, Stocker M. Day surgery in different guises: a comparison of outcomes. J One Day Surg. 2007;19:39–42.

5.      Stjernberg M, Rustøen T, Ræder J. Få pasienter opplever postoperativ kvalme etter dagkirurgi med multimodal kvalmestillende behandling. Sykepleien Forskning. 2018;13(71442):e-71442. DOI: 10.4220/Sykepleienf.2018.71442

6.      Salisbury C, Wallace M, Montgomery A. Patient experience and satisfaction in primary care: secondary analysis using multilevel modelling. BMJ. 2010;341:5004.

7.      Cleary P, Edgman-Levitan S, Roberts M, Moloney T, McMullen W, Walker J, et al. Patients evaluate their hospital care: a national survey. Health Aff. 1991;10(4):254–67.

8.      Grøndahl VA, Karlsson I, Hall‐Lord ML, Appelgren J, Wilde‐Larsson B. Quality of care from patients’ perspective: impact of the combination of person‐related and external objective care conditions. J Clin Nurs. 2011;20(17–18):2540–51.

9.      Jaensson M, Dahlberg K, Eriksson M, Nilsson U. Evaluation of postoperative recovery in day surgery patients using a mobile phone application: a multicentre randomized trial. BJA. 2017;119(5):1030–8.

10.    McGrath B, Elgendy H, Chung F, Kamming D, Curti B, King S. Thirty percent of patients have moderate to severe pain 24 hr after ambulatory surgery: a survey of 5,703 patients. Can J Anesth. 2004;51(9):886–91.

11.    Quemby D, Stocker M. Day surgery development and practice: key factors for a successful pathway. Cont Educ Anaesth Crit Care Pain. 2014;14(6):256–61.

12.    Royse C, Newman S, Chung F, Stygall J, McKay R, Boldt J, et al. Development and feasibility of a scale to assess postoperative recovery: the post-operative quality recovery scale. Anesthesiology. 2010;113(4):892–905.

13.    Myles P, Hunt J, Nightingale C. Development and psychometric testing of a quality of recovery score after general anesthesia and surgery in adults. Anesth Analg. 1999;88(1):83–90.

14.    Karaman S, Airici S, Dogru S, Karaman T, Tapar H, Suren M, et al. Validation of the Turkish version of the Quality of Recovery-40 questionnaire. Health Qual Life Outcomes. 2014;12(8). DOI: 10.1186/1477-7525-12-8

15.    Tanaka Y, Wakita T, Fukuhara S, Nishiwada M, Inoue S, Kawaguchi M, et al. Validation of the Japanese version of the quality of recovery score QoR-40. J Anesth. 2011;25(4):509–15. 

16.    Kleif J, Edwards H, Sort R, Vilandt J, Gögenur I. Translation and validation of the Danish version of the postoperatove quality of recovery score QoR-15. Acta Anesthesiol Scand. 2015;59(7):912–20.

17.    Lyckner S, Böregård I, Zetterlund E, Chew M. Validation of the Swedish version of Quality of Recovery score-15: a mulitcentre, cohort study. Acta Anesthesiol Scand. 2018;62(7):893–902.

18.    Stark P, Myles B, Burke J. Development and psychometric evaluation of a postoperative quality of recovery score: the QoR-15. Anesthesiology. 2013;118(6):1332–40.

19.    Kleif J, Waage J, Christensen K, Gögenur I. Systematic review of the QoR-15 score, a patient-reported outcome measure measuring quality of recovery after surgery and anaesthesia. BJA. 2018;120(1):28–36.

20.    Holmboe O, Sjetne I, Groven G, Bjertnæs Ø. Støtte til gjennomføring av lokale brukerundersøkelser ved dagkirurgiske sentre. Oslo: Nasjonalt kunnskapssenter for helsetjenesten; 2010.

21.    Tsang S, Royse C, Terkawi A. Guidelines for developing, translating and validating a questionnaire in perioperative and pain medicine. Saudi J Anaesth. 2017;11(1):80–9.

22.    Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee A, et al. Task force for translation and cultural adaption of good practice for translation and cultural adaption process for patient-reported outcomes (PRO) measures: Report of the ISPOR Task Force for translation and cultural adaption. Value Health. 2005;8(2):94–104. DOI: 10.1111/j.1524-4733.2005.04054.x

23.    EuroQol Group. EuroQol – a new facility for the measurement of health-related quality of life. Health Policy (Amsterdam, Netherlands). 1990;16(3):199–208.

24.    Gornall B, Myles P, Smith C, Burke J, Leslie K, Pereira M, et al. Measurement of quality of recovery using the QoR-40: a quantitative systematic review. BJA. 2013;111(2):161–9.

25.    Chazapis M, Walker E, Rooms M, Kamming D, Moonesinghe S. Measuring quality of recovery-15 after day case surgery. BJA. 2016;116(2):241–8. DOI: 10.1093/bja/aev413

26.    McIntosh S. Anxiety and quality of recovery in day surgery: a questionnaire study using hospital anxiety and depression scale and quality of recovery score. Int J Nurs Pract. 2011;17(1):85–92.

27.    Stessel B, Fiddelers A, Joosten E, Hoofwijk D, Gramke H, Buhre W. Prevalence and predictors of quality of recovery at home after day surgery. Medicine. 2015;94(39):e1553. DOI: 10.1097/MD.0000000000001553

28.    Brix L, Bjørnholdt K, Thillemann T, Nikolajsen L. Pain-related unscheduled contact with healthcare services after outpatient surgery. Anaesthesia. 2017;72(7):870–8. DOI: 10.1111/anae.13876

29.    Boissard M, Crenn V, Noailles T, Campard S, Lespagnol F. Recovery after shoulder arthroscopy: inpatient versus outpatient management. Orthopaedics & Traumatology: Surgery & Research. 2018;104(1):39–43. DOI: 10.1016/j.otsr.2017.10.010