Patient-Satisfaction Measures in Anesthesia: Qualitative Systematic Review

PATIENT satisfaction is an important measure of the quality of health care. Satisfaction with anesthesia is used as an outcome measure in clinical trials,¹  and patient satisfaction is considered to be an integral part of service quality.²  Its measurement is also required to fulfill performance improvement and revalidation agendas for healthcare professionals.³  However, clinical experience tells us that appropriately developed or validated instruments are not widely used in any of these settings.

Pascoe⁴  defined patient satisfaction as the patient’s reaction consisting of a “cognitive evaluation” and “emotional response” to the care they receive. It, therefore, seems prudent to ensure that patients are involved in the development of satisfaction tools, particularly because it is also subject to the sociodemographic, cultural influences, and cognition of the patients.⁵  The Picker inpatient survey⁶  is a well-known tool used in Europe to measure “patient experience,” however, there have been many flaws detected in its design, including the lack of patient involvement in the development stage.⁷  This has been compared with the Hospital Consumer Assessment of Healthcare Providers and Systems survey used by Press Ganey in the United States, which has been extensively developed.⁸ 

The development of a patient-satisfaction tool requires a step-wise psychometric process and subsequent validation in practice, and due to the multidimensional and complex nature of satisfaction, questionnaires should use multiple items to investigate specific events.⁹  The steps generally involved in the psychometric development of a questionnaire are described in table 1. In the “satisfaction” field there is no “definitive standard” to compare with (criterion validity), so to guarantee validity of the questionnaires, a thorough item-generation process is required to ensure content and face validity. Results can then be correlated with other factors suspected to be associated with the topic, known as construct validity. Measuring the internal consistency of the questionnaire may also enhance the validity. Items within a dimension should correlate, and the individual dimensions should have a Cronbach α greater than the overall result.¹⁰ 

Quality of recovery¹¹  is sometimes joined with patient satisfaction and quality of life to provide “patient-centered” outcomes.⁵  Previous work has comprehensively reviewed the literature on quality-of-recovery scores^12,13  and found there to be at least two suitable instruments available. However, systematic evaluations of instruments used to measure patient satisfaction after anesthesia, have been limited to two particular clinical settings: ambulatory anesthesia¹⁴  and regional anesthesia;¹⁵  both reviews demonstrated a paucity of appropriately validated tools. To our knowledge, there is no published evidence synthesis of instruments used to measure patient satisfaction with anesthesiology in general. Given the importance of using validated outcome measures, and the increasing focus on patient-centered outcomes in both research and clinical practice, this represents an important gap in the literature. Therefore, we have undertaken a qualitative systematic review, to answer the question: “What instruments have been psychometrically developed to measure patient satisfaction with anesthesia, and what is their validity?” The purpose of this review is to qualitatively appraise the literature and provide guidance about the strengths and limitations of patient-satisfaction tools that may be used for quality improvement and research purposes.

We have adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement standards in this article.¹⁶ 

We searched the online databases MEDLINE and Embase and ISI Web of Science (all database search) for articles published between January 1, 1980 and March 1, 2012 without language exclusion, but limited to human studies. The search strategy included snowballing of references and manual searching of citation lists, which is detailed in appendix 1.

For the purposes of this review, a “patient-satisfaction questionnaire” was defined as an instrument that was developed using psychometric techniques, and that consisted of at least two distinct dimensions. We included all studies that used a questionnaire developed in this way to assess patient satisfaction with some aspect of anesthesia: these included studies of pediatric patients and parental satisfaction, satisfaction with general anesthesia, local anesthesia, ambulatory anesthesia, and regional anesthesia. In order to avoid repeating previously published work, we have focused on measures of “patient satisfaction” and therefore, have excluded studies describing the development or validation of “quality of recovery” indicators. We also excluded questionnaires that were developed to measure satisfaction with sedation or satisfaction solely with pain management.

We reported the characteristics and quality of every article by extracting the following information: year and country of origin, number of patients recruited into study, number of dimensions within the score, number and nature of the items within each dimension, the response format, the type of anesthesia and surgery being evaluated, and the results of the study as reported by the authors.

For every satisfaction measure we identified, we evaluated the rigor of the original psychometric construction and evaluation process by assessing how the authors reported the questionnaire development process, pilot testing, and the validity, reliability, and acceptability of each instrument. The criteria we have used for assessing validity is based on methodological descriptions of thorough item generation as well as authors claims. We were unable to find a published system for comparing the quality of the psychometric development processes for questionnaires in a structured and objective manner. Therefore, we have reported our evaluation of the psychometric development reported in each article, by dividing the process into three phases: (1) item generation and pilot testing, (2) validation and reliability, and (3) acceptability to patients, including response rate and completion time. Each questionnaire was then scored on a scale of 0 to 2 in each category, with a maximum achievable score of 6. Although this scoring system was not previously validated, it gives an indication of the depth of psychometric development and testing behind each questionnaire.

The search identified 18,665 studies. Two authors independently screened the titles and abstract, and 15,454 articles were excluded. Three authors reviewed the full texts of the remaining 3,211 articles; manual searching of reference lists (snowballing) revealed a further 58 articles. Articles that excluded were 3,118 as they did not describe instruments that met our definition of a patient-satisfaction questionnaire. Of the remaining 150 articles, 79 were excluded as they did not use a questionnaire which met our criteria for psychometric development. Therefore, our final analysis consists of 71 articles describing a total of 34 patient-satisfaction scores, developed and evaluated using psychometric testing (fig. 1). Questionnaires meeting our inclusion criteria were not published before 1990, however, 6 were from the 1990s, and 28 were between 2000 and 2012 March.

Our description of the original articles developing each of these 34 patient-satisfaction tools is listed by clinical specialty in tables 2–7. We have reported the details of the psychometric evaluation process and scored the presence of item generation, validity and reliability, and acceptability for each of these studies in table 8. A list of studies which have subsequently used any one of these 34 questionnaires is provided in appendix 2. Below, we report a summary of the overall results and descriptions of the highest quality studies in each category.

We found three studies, which used questionnaires that had been psychometrically developed to measure maternal satisfaction with obstetric care: two were used following cesarean section, and one assessed maternal satisfaction after neuraxial blockade for labor analgesia. Of these, one¹⁷  involved patients in the questionnaire design and development process and two did not.^18,19  Morgan et al.¹⁷  used a clearly defined psychometric development and evaluation process, a 22-item questionnaire, which they named the Maternal Satisfaction Scale for Cesarean Section. Hobson et al.²⁰  validated the Maternal Satisfaction Scale for Cesarean Section using a different distribution format to the original development article; Sindhvananda et al.¹⁸  used the most objectively robust development and validation process (scoring 5 out of 6 on our assessment); however, their report was published in 2002,²¹  and their questionnaire has not subsequently been used in any other published studies.

Although there were many studies which included satisfaction with general and regional anesthetics, we could find only one French article, which used a psychometric development and evaluation process, to construct a questionnaire measuring satisfaction with regional anesthesia in the nonobstetric setting.²²  Despite a growing literature evaluating the efficacy and outcomes of regional anesthesia, this instrument has subsequently been used in only one other study.²³  This lack of validated tools for measuring satisfaction with regional anesthesia was also reported by Wu et al.¹⁵  in their systematic review of this field of practice.

The American Society of Anesthesiologists defines Monitored Anesthetic Care as the delivery of local anesthesia together with sedation and analgesia for a planned procedure. Themost referenced instrument assessing satisfaction with Monitored Anesthetic Care is the Iowa Satisfaction with Anesthesia Scale (ISAS), consisting of 11 questions;²⁴  this scored highly (6 out of 6) in our objective appraisal of the development process.

We found a further 17 studies using the ISAS to assess satisfaction. Eight of these used the ISAS for satisfaction with ophthalmology procedures;^25–32  only one of these studies²⁸  performed further validation of the scale within their patient cohorts. The remaining studies used the ISAS to assess satisfaction with Monitored Anesthetic Care for other procedures and surgery.^{33–37,38–40}

We identified six tools used in pediatric anesthesia, which had undergone psychometric development.^41–46  Kain et al.⁴⁴  developed an 11-item questionnaire using a three-step approach starting with validity testing in the form of items grouping using input from anesthetists, surgeons, psychologists, play specialists, and nurses. A rigorous protocol and psychometric evaluation was recently undertaken when Schiff et al.⁴⁶  constructed a “Pediatric Perianesthesia Questionnaire.” This comprised 37 questions and demonstrated extensive item generation, content, and convergent and discriminant validity with excellent internal consistency for all five dimensions. The questionnaire developed by Iacobucci et al.⁴³  is notable for being one of two we identified, which attempted to assess the child’s satisfaction with the anesthetic experience. Although they reviewed the literature, they did not undertake any formal item generation or pilot testing for their questionnaire assessing parental (6 questions) and child (9 questions) satisfaction. They assessed construct validity by comparing parental satisfaction with the child’s reported anxiety, and they tested reliability with test-retesting on 18 parents and 11 children a day after the intervention. They demonstrated good internal consistency (Cronbach α 0.86), with response rates of 84% for parents and 52.3% for children, respectively. This instrument was modified by Lew et al.⁴⁷  to assess satisfaction with pediatric sedation, rather than anesthesia.

We found 23 original articles that developed and validated patient-satisfaction measures with perioperative anesthetic care. Within this cohort, these tools have been used to evaluate satisfaction with preoperative assessment conducted by anesthetists, regional anesthesia, and/or general anesthesia. We have summarized these preoperative assessment instruments in table 6 and perioperative instruments in table 7; the details of the most rigorously developed and subsequently validated measures are described in the following sections on preoperative assessment and perioperative care.

Snyder-Ramos et al.⁴⁸  developed their measure in order to evaluate the quality of the anesthetist’s preoperative visit. Thetool was divided into two parts: evaluation of satisfaction with the preoperative visit; and the information the patient gained as a result of the visit. This was a German study and its validity and suitability when translated into other languages is yet to be established; however, a recent study, looking at the use of a preanesthetic information form, used some questions from this original tool.⁴⁹  The Consultation and Relational Empathy questionnaire⁵⁰  is a 10-question modification of a tool that had been previously developed and validated to assess patient satisfaction with consultations in primary care. The Patient Liaison Group of the United Kingdom Royal College of Anesthetists, discussed the tool to establish validity where generalized reliability, interrater reliability (using G-coefficient, similar to Cronbach α), and internal consistency were calculated. This resulted in a reliable and internally valid tool to assess patients’ views on anesthetists’ interpersonal communication skills.

Nineteen questionnaires measuring patient satisfaction with perioperative care are included in our review. Of these, 10 sought patient advice in the development process.^51–60  When Auquier et al.⁵¹  initially constructed their 25-item Evaluation du Vecu de l’Anesthesie questionnaire, they conducted a pilot study on 742 patients who underwent procedures under general anesthetic.⁵¹  They concluded that the Evaluation du Vecu de l’Anesthesie questionnaire is valuable in assessing patients’ opinions on the perioperative period,⁶¹  and went on to develop the Evaluation du Vecu de l’Anesthesie Generale questionnaire,⁶²  consisting of 26 questions, which was rigorously psychometrically developed and validated. Both these questionnaires used patient input in the development processes.

Bauer et al.⁶³  looked primarily at measuring satisfaction with anesthesia and secondarily, comparing a 15-item written questionnaire with face-to-face interviews. A robust item-generation process was undertaken and content validity was assured by using anesthetists, nurses, and a literature review in the development of questions; however, no patients were consulted at this initial item stage. Pilot testing, question streamlining, and test–retest reliability were conducted and internal consistency measured (Cronbach α 0.84). This tool has been used once subsequently, to measure satisfaction after carotid endarterectomy.⁶⁴ 

Caljouw et al.⁵⁶  developed the 39-question Leiden Perioperative care Patient Satisfaction questionnaire, using the Evaluation du Vecu de l’Anesthesie questionnaire by Auquier et al.⁵¹  as their basis for items generation. The English adaptation of the Lieden Perioperative care Patient Satisfaction questionnaire was validated by Jlala et al.⁵⁷  Pilot and follow-up studies found this tool to be acceptable (response rate >90% for all questions) and reliable (Cronbach α 0.94).

Capuzzo’s pilot study⁵²  generated 10 items for a new questionnaire, using a panel of doctors, nurses, experts, and interviews with patients who had recently received an anesthetic. Reliability and internal consistency were evaluated, and construct validity was assessed based on an assumption that young patients would have a lower satisfaction than older patients, and that a significant relationship between the items and satisfaction would be found. This tool has been used in two further studies.^65,66 

Another rigorous protocol was used in the development and validation of the 29-item patient-satisfaction questionnaire by Heidegger et al.⁵³  They concluded that a psychometric questionnaire for satisfaction with anesthesia care must include areas related to information, involvement in decision-making, and contact with the anesthetist. This tool has been used in three studies since this initial study.^67–69 

During a 5-yr period, Hüppe published three studies evaluating a new perioperative questionnaire now known as the Anesthesiological Questionnaire. The initial study described the development and initial evaluation.⁷⁰  The result was a two-part questionnaire with 66 items; part 1 assessing the postoperative period and the patients’ symptoms, and part 2 more concerned with satisfaction with anesthetic care, perioperative care, and postoperative recovery. The questionnaire was then modified to 46 items and a further study was performed to test its reliability and validity.⁷¹  Finally, the authors adapted it for use in cardiac anesthesia with further psychometric evaluation in this cohort of patients.⁷²  TheAnesthesiological Questionnaire was also used by Reurer et al.⁷³  to assess satisfaction after elective surgery.

Le May et al.⁵⁴  also addressed patients’ perceptions of cardiac anesthesia services, developing the Scale of Patients’ Perceptions of Cardiac Anesthesia Services scale. This included 17 Likert-type questions with 10 sociodemographic and 3 open-ended questions. Of importance, this trial addressed a very homogenous group of cardiac patients and therefore, this specific questionnaire is not necessarily a valid tool for more generalized patients.

In 2008, Schiff et al.^55,74  published two studies and developed the 38-item Heidelberg perianesthetic questionnaire to assess perioperative satisfaction for quality improvement and benchmarking purposes. They also used this tool in a study of the anesthetic preoperative evaluation clinic⁷⁵  along with another group of questions addressing the preanesthetic consultation.⁴⁸  The Heidelberg questionnaire has been used by another research group to psychometrically assess patients’ suitability for local anesthesia for carotid endarterectomy.⁷⁶ 

This systematic review identified a large number of questionnaires that have been psychometrically developed to measure patient satisfaction with anesthesia in a variety of clinical specialties and settings. However, of more than 3,000 articles using patient satisfaction as an outcome measure, only 71 used patient-satisfaction measures that were multidimensional and had undergone some sort of psychometric development process. Our qualitative appraisal of the tools used in different areas of anesthesia practice leads us to make recommendations about the tools researchers and clinicians may choose to use for measuring patient satisfaction in different settings. For “Monitored Anesthetic Care,” the ISAS²⁴  is robust, with high patient and clinician acceptability. For the perioperative assessment of satisfaction, the questionnaires by Capuzzo et al.⁵²  and Bauer et al.⁶³  are short, yet well developed and may be suitable for use in quality-improvement projects. However, the more lengthy questionnaires, such as the English adaption of the Leiden Perioperative care Patient Satisfaction questionnaire⁵⁷  and Heidelberg perianesthetic questionnaire,⁵⁵  are also acceptable to patients, and therefore, may be suitable for research purposes. These recommendations are listed in table 9.

Our study has some limitations. This is not the first systematic review of patient-satisfaction measures in anesthesia; however, previous publications have focused on specific areas of practice, such as ambulatory or regional anesthesia.^14,15  We believe that this is the first systematic review to cover instruments measuring satisfaction with each and every element of the anesthetic experience (including preoperative assessment and postoperative recovery) and every patient group (for example, pediatrics and maternity). We have attempted to minimize bias by not restricting our search on the basis of language; however, we did limit the search to articles published from 1980 onward, as our intention was to provide the reader with information on questionnaires that would be relevant to current practice. Finally, although we have attempted to locate all relevant articles by using a robust search methodology, it is possible that with a review of this size, some relevant articles may have been missed.

The need for a summary of the literature in this field has been demonstrated by our finding that only a small proportion of studies that use patient satisfaction as an outcome, use a multidimensional validated questionnaire to measure it. Within this systematic review we have differentiated “patient satisfaction” questionnaires from “quality of recovery” questionnaires. A poor recovery may delay discharge from the postanesthetic care room or hospital, which has obvious resource implications.⁷⁷  Yet, there is evidence that incomplete recovery from various postoperative recovery domains does not always influence patient satisfaction.⁷⁸ 

Psychometrically developed questionnaires are important for the reliable measurement of patient satisfaction with anesthesia care for a number of reasons. First, patient-reported satisfaction with anesthesia is generally high, both in studies and clinical practice; a single question or visual analog scale is likely to lead to this result,¹  therefore providing limited information to enable service evaluation or quality improvement. Second, it is not unusual for patients to have limited knowledge regarding anesthesia and the role of the anesthetist; these issues may skew data collection, as questions may be answered with a focus on the “perioperative experience” and not the specific anesthetic care.¹⁵  Finally, a poorly constructed survey instrument can lead to a bias toward the investigators who designed it; this may result in the reporting of misleading outcomes in clinical studies. During the development process, involving patients in item generation can ensure a patient-focused approach and help to address patient expectations.⁵² 

Although our review may prove helpful to clinicians and researchers in the future, by summarizing the available measures, there are still unanswered questions in this field. For example, the generalizability of questionnaires across different settings is unclear: it is not necessarily right to assume that a questionnaire is valid outside its country of origin as there may be disparities in health care and patient expectations between nations and healthcare systems. Furthermore, we identified a number of the questionnaires that were developed in countries that did not have English as the first language; their validity after translation has not been established.^{18,22,48,58,71,72,79}  Only one instrument developed in a non–English-speaking country (the Leiden Perioperative care Patient Satisfaction questionnaire) has been validated after translation into English.⁵⁷ 

The optimal timing for completing a satisfaction questionnaire for patients undergoing anesthesia is also not clear. A dilemma exists, as within the acute recovery period, the patient may still be under the influence of anesthesia and yet, with the implementation of enhanced recovery programs, many patients are not in hospital for extended periods of time. Patient demographics also require consideration: there is evidence that women have lower satisfaction levels for up to 3 days postoperatively,⁸⁰  and also that patients having major and minor surgery will have differences in their recovery profile and, therefore, in their responses to satisfaction surveys.¹¹  Therefore, the optimal timing (and therefore method) of administration of a patient-satisfaction survey may be different depending on the surgical specialty and the extent of the surgical procedure.

These issues may in turn have an impact on the answers that patients provide and also, on the response rates. Patient responses may be biased in order to please the hospital staff to avoid negative repercussions,¹  and equally satisfaction may be dominated by relief that the operation was a success.⁶³  In theory, in order to avoid the phenomenon of transference and countertransference, a questionnaire should lead to less bias than an interview.⁸¹  However, Bauer et al.⁶³  found that their standardized interview identified more patients reporting lower degrees of satisfaction and was, therefore, superior in detection of anesthetic quality; however, the resource and cost implications of interviews rule out this method as a means of recording patient satisfaction outside the research setting. In contrast, using a postal questionnaire some time after the patient episode of interest may impact on the number of responses received. Perhaps, surprisingly, there is some evidence that postal questionnaire response rates may be higher than those achieved by questionnaires administered at the hospital.⁸²  However, this is not consistent with evidence from within the setting of anesthesia satisfaction surveys, where response rates have been shown to be significantly lower at 9 weeks compared with 1 week and 5 weeks after an anesthetic.⁶⁸ 

When choosing a questionnaire to use in clinical practice or for research purposes, there are a number of considerations must be taken into account. Successful completion of a satisfaction questionnaire with minimal missing data is an indication of the clinical acceptability of the tool, thereby supporting its use in practice. Although the optimal length of time to complete an assessment is not clear, a shorter questionnaire that maintains a good level of validity and reliability with simple and easy-to-understand vocabulary is likely to be less of an imposition for patients who are asked to complete it.⁷⁹  A validated yet brief questionnaire will be more suitable for audit and quality-improvement purposes, whereas more detailed questionnaires, providing more information, may be more valuable as outcome measures in clinical trials. In areas of anesthesia practice, where there is a range of well-developed tools to choose from, we have made recommendations based on instruments that may be used in either the quality-improvement or research settings, based on the quality of the psychometric development process. However, there are many branches of anesthesiology where further work is required on the development and/or validation of satisfaction measures is required.

Regional anesthesia is gaining popularity, partly due to improvements in safety and success attributed to ultrasound-guided techniques.⁸³  Our review identified only one tool developed for measuring patient satisfaction after regional anesthesia;²²  further evaluation of this measure would be of value. Satisfaction surrounding the birth of a child is a complex and emotive subject; for this reason, a tool specifically assessing maternal satisfaction with the anesthetic care would be invaluable. Although our review found three original questionnaire designs, the two most robustly developed and validated instruments measured satisfaction after cesarean section.^17,20  There is, therefore, an unmet need for a survey, which can be used to measure the quality of anesthesia care in obstetric patients who do not have operative deliveries, or at least a requirement for further evaluation of the two existing published tools.^17,20  Pediatric anesthesia, where satisfaction measurement is complicated by the parent–child unit, is another area where an evidence-based process for developing satisfaction measures is important. Children may not evaluate their treatment in the same way as adults; memory at a young age may not be reliable, the power of suggestion should not be overlooked, and there is currently no research to fully elucidate whether a parent can accurately judge their child’s satisfaction with anesthesia.⁴⁶  The Pediatric Perianesthesia Questionnaire, which is answered by the patient and parent together, was the most robustly developed measure in this field. Although it is lengthy and complex, the high response rate in its development study indicates that it is acceptable to parents, although reducing its complexity may improve its feasibility even further. However, it is only with further evaluation in multiple centers that the true acceptability of this tool can be ascertained.

It is reassuring that our study has found a large number of well-developed tools to measure satisfaction with perioperative anesthesia care. However, we have also been able to highlight areas where further work would be of benefit. Perhaps our most significant finding is that the vast majority of anesthesia-related studies do not use validated tools to measure satisfaction, where this outcome is thought to be of importance. This omission may lead to biased and misleading results in studies of clinical effectiveness. As well as focusing on further evaluation of existing measures, and development of new tools where necessary, there is a need to encourage clinicians and researchers to incorporate validated measures into everyday practice and in clinical studies. This qualitative appraisal of the literature should provide a guide to anesthetists, reviewers, and editors on the measures that are available and valid, and therefore, assist in increasing the standards of outcome reports in academic studies, and quality improvement in clinical practices.