Anesthesia Safety: Model or Myth?: A Review of the Published Literature and Analysis of Current Original Data

The Medline and HealthStar databases were searched using subject keywords “anesthesia AND mortality” from 1966 to 2000 and from 1975 to 2000, respectively. Publications were included in this review if their titles or abstracts were available in English and suggested a perioperative mortality rate related to anesthetic management in a general patient population over a specified period of time based on original data. Publications were excluded if the anesthetic management was limited to a particular technique, or the patient population was limited to a particular procedure, associated disease state, or age group. Each publication was then reviewed as needed to identify author(s), study period, data source, perioperative mortality rate, anesthesia-related mortality rate, mortality rate for which anesthesia was solely responsible, and preventable anesthetic mortality rate as defined by each study.

Anesthesia-related mortality rates taken from the literature review were plotted against each study's midpoint on an “attribute P chart.” Attribute P charts reflect the number of defectives (anesthesia-related deaths) as a proportion of variable sample size. 7Studies with the same midpoint of the time period under investigation were combined into one sample, so that one data point could be generated for a single point in time. Upper control limits (three standard deviations from the average proportion defective) were established based on a binomial distribution. Systems were considered “out of control” if a point fell outside of the control limits or a “run” or “trend” was detected. A run  is defined as a succession of seven points that are above or below the average. A trend  is defined as a succession of seven points that are rising or falling. In a system without special causes for variation, a run or trend has approximately the same probability of occurring as a point outside a control limit, or .005. 7 

All cases of perioperative mortality, defined previously as death during or within two postprocedure days, 8,9which occurred at a suburban university hospital network between January 1, 1992 and December 31, 1994, were referred to the Department of Anesthesiology for peer review. Similarly, all cases of perioperative mortality, which occurred at an urban university hospital network between January 1, 1995 and December 31, 1999, were also referred for departmental peer review. A standardized model of peer review was used at both institutions under the author's supervision during these sequential time periods. 9 

Multiple referral sources were used, including the anesthesiologists, other clinical personnel using hospital incident reports, a follow-up phone call by nursing staff to ambulatory surgery patients, and concurrent chart reviewers. Contact was made with the anesthesiologist involved, and/or the medical record was reviewed, so that an abstract could be prepared by a preliminary group of two to four anesthesiologists for presentation to the respective departmental peer review committees. The peer review committees consisted of all available clinical members of the Departments of Anesthesiology for both suburban and urban hospital networks (approximately 25 staff anesthesiologists and 15 residents per committee for each). Each committee met on a monthly basis to participate in a structured peer review 9of the cases presented. The peer review process determined whether the mortality was solely the result of “system error” or whether there was a “human error” contribution. The American Society of Anesthesiologists (ASA) Physical Status of all patients undergoing anesthesia for operative procedures was only recorded at the suburban hospital network during 1994 and the urban hospital network from January 1, 1992 through December 31, 1999. Therefore, data collected during these years was used to extrapolate the distribution of ASA Physical Status over the two study populations. The Risk Management Departments of the respective institutions were also queried to determine whether legal action, previously defined as a letter of intent, claim, or closed claim, 10was initiated within 1 yr of any procedure that resulted in a perioperative death determined by peer review to be due, at least in part, to human error by an anesthesia practitioner.

The principle underlying the peer review process is that all adverse outcomes are the result of “error,” either human or system. 9Nominal definitions for subcategorizing these two types of errors were used to add structure and increase the objectivity of the peer review process. 9,11,12Error here was defined as an act that through ignorance, deficiency, or accident departs from or fails to achieve a desired outcome. 13Human errors included failing to perform a technique properly, misuse of equipment, disregarding available data, failing to seek appropriate data, and responding incorrectly to available data due to a lack of knowledge. These human errors were considered deviations from the standard of care.

System errors, on the other hand, result in adverse outcomes that might otherwise be considered unavoidable and ordinarily dropped from the peer review process. 14,15System errors included accidental occurrences resulting from performing a technique properly, equipment failure despite proper use, missed communication while following established protocol, inability to diagnose a disease process due to limitations of currently available screening and monitoring standards, inability to treat a disease process due to limitations in current standards of care, and inability to meet the demands for resources of equipment or personnel. Supervision by an attending anesthesiologist working with more than one resident or nurse anesthetist was viewed as a unique resource whose limitations were recorded separately from other resources. The peer review processes used by both Departments of Anesthesiology considered human errors on the part of nonanesthesia practitioners to be system errors if they were outside the control of the anesthesia provider. This would not prevent nonanesthesia practitioners from categorizing these adverse outcomes as human errors during their independent departmental reviews, but this process was unrelated to the anesthesiology peer review process. The error categories used by the Departments of Anesthesiology are summarized in tables 1 and 2along with common examples of each. 9,10 

Anesthesia-related mortality was defined as perioperative death to which human error on the part of the anesthesia provider, as defined by the peer review process, had contributed. If the departmental peer review committees determined that human error had contributed to an adverse outcome, they judged the degree to which the anesthesia care provider had contributed to that outcome. Contribution was graded on a three-point Likert scale ranging from minor to major. All determinations of the peer review committees were based on consensus.

The Medline and HealthStar database queries identified 3,566 and 525 publications respectively. Review of the published abstracts that met inclusion criteria identified 23 anesthesia-related mortality rates determined between 1955 and 1992 by 21 different investigators. The results can generally be summarized in four major categories: (1) overall perioperative mortality ranged from 1 death in 53 anesthetics to 1 in 5,417 anesthetics, (2) anesthesia-related mortality ranged from 1 in 1,388 anesthetics to 1 in 85,708 anesthetics, (3) anesthesia considered solely responsible for perioperative death ranged from 1 in 6,795 anesthetics to 1 in 200,200 anesthetics, and (4) preventable anesthetic mortality ranged from 1 in 1,707 anesthetics to 1 in 48,748 anesthetics. Results of the literature review are summarized in table 3and plotted on an attribute P chart in figure 1.

One hundred fifteen perioperative deaths occurred at the suburban university hospital network between January 1, 1992 and December 31, 1994. Two hundred thirty-two perioperative deaths occurred at the urban university hospital network between January 1, 1995 and December 31, 1999. Anesthetic caseloads were 37,924 for the suburban hospital network and 146,548 for the urban hospital network during those time frames. Calculated perioperative mortality rates were 30.3/10,000 anesthetics (1:332) and 15.8/10,000 (1:632) anesthetics respectively with an overall perioperative mortality of 18.9/10,000 anesthetics (1:532) as summarized in table 4. When broken down by ASA Physical Status the perioperative mortality rates in the suburban hospital network were 0 for 8,210 class I patients, 2 for 15,625 class II patients, 8 for 10,877 class III patients, 34 for 2,939 class IV patients, and 71 for 273 class V patients. Similarly, in the urban hospital network, perioperative mortality rates were 4 for 35,025 class I patients, 22 for 67,851 class II patients, 53 for 34,146 class III patients, 67 for 9086 class IV patients, and 86 for 440 class V patients. Perioperative mortality rates as a function of ASA Physical Status are shown in figure 2.

Peer review determined that human error by an anesthesia practitioner contributed to 3 of the 115 (2.6%) perioperative deaths at the suburban university hospital network and 11 of the 232 (4.7%) perioperative deaths at the urban hospital network. Anesthesia-related mortality, defined as a perioperative death to which human error by an anesthesia practitioner contributed, occurred at a rate of 0.79 per 10,000 (1:12,641) anesthetics in the suburban setting and 0.75 per 10,000 (1:13,322) in the urban setting as summarized in table 4. When broken down by ASA Physical Status the anesthesia-related mortality rates in the suburban hospital network were 0 for 8,210 class I patients, 0 for 15,625 class II patients, 2 for 10,877 class III patients, 0 for 2,939 class IV patients, and 1 for 273 class V patients. Similarly, in the urban hospital network, anesthesia-related mortality rates were 0 for 35,025 class I patients, 3 for 67,851 class II patients, 2 for 34,146 class III patients, 6 for 9086 class IV patients, and 0 for 440 class V patients. Overall anesthesia-related mortality rate as a function of ASA Physical Status throughout the study period is shown in figure 3.

Of the 14 anesthesia-related deaths overall, four were the result of major contributions from the anesthesia personnel (1:46,118 anesthetics), but only three resulted in legal action in the form of a letter of intent, claim, or closed claim (1:61,490 anesthetics). Of the three resulting in legal action, only one occurred in a patient with an ASA Physical Status of 1 or 2 (1:126,711 anesthetics in ASA Physical Status 1 to 2 patients).

Our current original data suggest an overall perioperative mortality rate of approximately 1/500 anesthetics. This is consistent with the literature review, but the medical literature review offers a wide range of values. The wide range of perioperative mortality rates offered by the literature may be caused by differences in operational definitions and reporting sources. This is best illustrated by Pedersen (table 3), who described markedly different perioperative mortality rates in the same population depending on the timing of the patients’ deaths. 16Our current data are consistent with the perioperative mortality rate recorded by the JCAHO (approximately 1/300 anesthetics), which used the same definition and similar mandatory reporting for participating hospitals. ‡

Our overall mortality rate (1:532) is also similar to the perioperative mortality rate reported by Vacanti et al.  for 68,388 elective and emergency surgeries performed in 11 U.S. Naval Hospitals between 1964 and 1966 (1:25717). There are differences, however, when these perioperative mortality rates are stratified by ASA Physical Status. For example, Vacanti et al.  17reported a perioperative mortality rate of 1:1179 versus  our rate of 1:10,809 for ASA Physical Status 1, and 1:11 versus  our rate of 1:4.5 for ASA Physical Status 5. These differences may be caused by the small number of patients in both studies and our need to extrapolate the distribution of ASA Physical Status from a limited data set. Another possibility is differences in the application of ASA Physical Status designations. For example, by definition, ASA Physical Status 5 patients are not expected to survive 24 h with or without their planned operative procedure, thus our higher mortality figures for this class of patients seem more credible. It is also possible that not all of the perioperative deaths were captured. Our use of mandatory reporting to a nonpunitive peer review process, multiple reporting sources, and the severity of the outcome assure a high capture rate for this occurrence 18and may also account for some of the difference between our data and that of Vacanti. Vacanti's study was excluded from our literature review because it did not include an anesthesia-related mortality rate.

Twenty-three anesthesia-related mortality rates were identified, however, by our literature review. Wide variation in anesthesia-related mortality rates reported over the last 35 years may also be due to differences in operational definitions. Definitions varied from intraoperative deaths 19–22to deaths occurring within 30 days 23or prior to discharge from the hospital. 16In some studies, perioperative “death” also included patients who failed to regain consciousness after anesthetic management, 24–26or who died any time during their hospital stay following an intraoperative cardiac arrest. 27This author used the 1992 JCAHO (Joint Commission on Accreditation of Healthcare Organizations) definition of perioperative mortality. “Death of patients during or within two postprocedure days” was one of the JCAHO Perioperative Performance Indicators that survived α and β testing for face validity and feasibility of data collection in a broad range of healthcare institutions. 28Anesthesia-related mortality was defined as perioperative death to which human error on the part of the anesthesia provider, as defined by our peer review process, had contributed.

The principle underlying our peer review process is that all adverse outcomes are the result of “error” defined as an act that through ignorance, deficiency, or accident departs from or fails to achieve a desired outcome. 13This definition of error is consistent with the IOM definition:

“Failure of a planned action to be completed as intended” or “use of a wrong plan to achieve an aim; the accumulation of errors results in accidents.”1 

Both of these definitions allow reviewers to look at the system as critically as they look at each other, thus making peer review less threatening. Katz and Lagasse have shown that anesthesiologists will comply with a system of self-reporting if the process is nonpunitive and can result in real improvements in patient care. 18In addition to its effect on self-reporting rates, peer review can also affect published anesthesia-related mortality rates through the accuracy of its judgments.

Although the accuracy of judgments by a peer group can never be assured, interrater reliability can give some indication of the reproducibility of the data. Several measures have been taken to improve the reliability of our peer review process. Use of multiple reviewers who meet to discuss the case has been shown to markedly increase consensus among reviewers. 11,12,29,30During the course of this study, the faculty of the departments of anesthesiology remained relatively constant so that the members of the peer review groups remained stable. Structured assessment procedures have also been recommended to decrease differences in reviewers’ understanding of their task and thus to increase the objectivity of implicit peer review. 11,12By using nominal definitions for categorizing peer review opinions regarding adverse outcomes, the error analysis was relatively easy to apply so that the errors could be reliably identified and grouped. Structured peer review and a stable pool of reviewers allow the error categories to become more sharply defined over time. 9Shared expertise in a particular area also improves agreement among reviewers. 31All of our reviewers were anesthesiologists or resident anesthesiologists as defined by the composition of each department. Although some investigators have suggested that outcome data be withheld when determining appropriateness of care, 32others have suggested that outcome data are necessary to assure adequate agreement among multiple reviewers. 11A recent study of structured peer review models showed no relation between severe outcomes and subsequent classification as human error. 12This study also showed that the peer review model used in the current study has excellent interrater reliability when used in the manner described.

As pointed out by Keats 33more than two decades ago, high interrater reliability in a peer review process does not assure that judgments are accurate. Judgments are shaped by the knowledge, experience, and current norms of the reviewers, and therefore, may vary over time. Consequently, to make valid comparisons with historical controls, outcomes databases must record the circumstances surrounding each event in addition to the judgments rendered. This allows subsequent peer review at a later date and removes the potential bias of an evolving peer review process. Although the author's database contains abstracts of each event for subsequent review at a future date, one can envision a more complete database that is linked to an archive of objective data recorded by a clinical anesthesia information management system for similar purpose.

The current study defined anesthesia-related mortality as perioperative death to which human error on the part of the anesthesia provider, as defined by our peer review process, had contributed. Although the review mechanisms differed, this was the predominant definition used in the studies reviewed for this article (table 3). Not all investigators, however, adhered to this convention. McKenzie, for example, reported an “anesthesia-associated” mortality rate that was identical to his perioperative mortality rate of 1:388 anesthetics, but went on to report a preventable anesthetic mortality of 1:3030 anesthetics. Preventable anesthesia mortality was defined by McKenzie, and other investigators, as death of patients in whom alternative care on the part of the anesthesia provider was likely to have resulted in patient survival. 16,26,34–36 

Another mortality rate that was reported in the literature reviewed was that in which anesthesia was considered solely responsible. This mortality rate was inconsistently defined as patients suffering a perioperative death as a result of human error on the part of the anesthetist alone, 24patients suffering perioperative mortality as a result of the anesthetics administered, 37,38or patients in whom anesthesia-related deaths were unlikely to be due to patient disease. 22Comparison of anesthesia-related deaths to deaths in which anesthesia was considered solely responsible may have led the IOM to conclude that anesthesia mortality rates have decreased from 2 deaths per 10,000 anesthetics administered in the 1980s to about 1 death per 200,000 to 300,000 anesthetics administered today.

Only one investigator in our literature review reported an anesthesia mortality rate of less than 1 in 200,000 anesthetics. 22In Eichhorn's study, five intraoperative deaths in ASA Physical Status 1 and 2 patients, which were reported to the Harvard malpractice insurance carrier between 1976 and 1988, were considered to be solely attributable to anesthesia. During this time period, anesthetics were administered to 1,001,000 ASA Physical Status 1 and 2 patients, thus yielding a rate of death solely attributable to anesthesia of 1 per 200,200 “healthy” patients.

Although one cannot be certain that the 1976–1988 Eichhorn data 22are the source of the claim by the IOM that anesthesia mortality rates have decreased by an order of magnitude over the past two decades, it is important to put these findings into perspective by comparing them with our current original data. The Eichhorn study relied on data reported to a malpractice insurance carrier, whereas the current data looked at human error as determined by peer review. A recent study showed that malpractice litigation risk as defined by a letter of intent, claim, or closed claim has no relationship to human errors by anesthesiologists that result in disabling patient injuries as determined by peer review. 10In addition, Eichhorn's study only considered intraoperative deaths that occurred in patients with an ASA Physical Status of 1 to 2 in which anesthesia was considered the sole contributor.

Of the 14 anesthesia-related deaths reported in the current original data, only 4 were the result of major contributions from the anesthesia personnel, or 1 per 46,118 anesthetics. Of note, our peer review process has never considered a death to be due solely to anesthetic management. Patient disease or other environmental factors have always been felt to be contributory. Of the four deaths with major contributions by the anesthetist, only one occurred in a patient with an ASA Physical Status of 1 to 2 and resulted in litigation over the 8-year period in which 126,711 anesthetics were performed in these classes of patients. Interestingly, this single death did not occur intraoperatively and would not have met the inclusion criteria of the Eichhorn study. Therefore, our findings are consistent with the Eichhorn study, which may have been the basis of the IOM claim.

Although our literature search casts some doubt on the improvement in anesthesia safety, expressed in terms of anesthesia related mortality, there are some considerable weaknesses. First, it fails to consider anesthesia-related mortality reported prior to 1966 because of the temporal limitations of the Medline database. In 1954, Beecher and Todd reviewed 599,548 anesthetics administered in ten institutions and reported a perioperative mortality rate of 1 in 75. These investigators reported that anesthesia was the primary cause of mortality in 1 in 2,680 cases and was contributory in 1 in 1,560 cases. 39Dornette and Orth reported a total of 108 deaths in 63,105 anesthetics administered between 1943 and 1954. Death was attributed solely to anesthesia in 1 in 2,427 anesthetics and contributory in 1 in 1,343 patients. 40Dripps et al . at the University of Pennsylvania reported 1,285 perioperative deaths within 30 days of the approximately 120,000 anesthetics performed between 1947 and 1957. Anesthesia was considered a definite contributor to death in 1 in 1,580 spinal anesthetics and 1 in 536 general anesthetics. 41Of note, even this 1961 report by Dripps et al.  41showed no deaths in any of the 16,000 patients with ASA Physical Status 1. Although these selected studies suggest improvements in anesthesia safety over the past 50 years, they have not been included in our control chart because it is difficult to assure consideration of all relevant studies predating the Medline database (1966).

Second, a lack of appropriate risk stratification makes it difficult to detect trends in anesthesia safety because study populations may differ, both regionally and historically, with respect to perioperative risk. For example, in 1996, Clergue surveyed French hospitals and found that anesthetic procedures had increased by 120% since a similar survey was conducted in 1980. When corrected for the increase in the French population, his findings showed that the annual rate of anesthetic procedures had increased from 6.6 to 13.5 per 100 people. More importantly, elderly patients and those with higher ASA Physical Status showed the greatest increase. In France between 1980 and 1996, the number of anesthetic procedures for patients with an ASA Physical Status of 1 had increased by 30%, while the number of procedures for patients with an ASA Physical Status of 3 had increased by 268%. 42If this trend were global, it could certainly mask improvements in anesthesia safety.

A trend toward increasing concurrent disease could also make the author's use of extrapolation to estimate the distribution of ASA Physical Status for the current original data less than optimal. In the current study, that could have resulted in an overestimation of patient disease because ASA Physical Status was known in the latter portion of each study period (urban and suburban) and extrapolated to the earlier portions. Overestimation of ASA Physical Status might then have resulted in a mortality rate that is lower than expected, rather than the findings presented here. Also, comparison of the distributions of ASA Physical Status between the urban and suburban hospital networks shows a higher percentage of ASA Physical Status 4 and 5 patients treated before 1995. Clearly this is more likely to be due to regional differences than a trend in perioperative risk.

In addition to patient population, regional differences may also confer variation in practice standards, technological resources, and reporting mechanisms. These variations might be considerably large when the regional differences involve several different countries. This may have resulted in increased variability in the anesthesia-related mortality rates seen in our statistical process control chart and masked trends in safety. Evidence to support this can be seen in the anesthesia-related mortality studies conducted in a single country. For example, in New South Wales there appears to be steady improvement in anesthesia-related mortality. Holland 43demonstrated a decline in anesthesia-related mortality from approximately 1:5,500 in 1960 to 1:10,250 in 1970, and 1:26,000 in 1984. Similarly, Harrison 35,25demonstrated a modest decline in anesthesia-related mortality over a 20-yr period at a teaching hospital in Cape Town, South Africa. Still, these single venue studies do not show dramatic improvements in anesthesia-related mortality rates over the past three decades.

Similar trends in safety have been observed in other industries that rely on high-risk technologies. The safety of airline travel, for example has increased dramatically in this century, but since the 1960s there has been minimal improvement in fatality rates. 44This may be due to the effect that improved safety technology has had on air traffic density. Technology has made it possible to meet production pressures of the commercial airline industry by allowing more takeoffs and landings with less separation between aircraft. With this increased aircraft density comes increased danger, thereby offsetting potential improvements in safety. This may be analogous to the practice of anesthesiology in which improvements in medical technology have led to increased anesthetic management of older patients with significantly more concurrent disease. As we have shown (figs. 1 and 2), the risk of death in these complex patients increases exponentially, as does the risk of death in which human error by an anesthesiologist is deemed contributory.

It is not surprising that our control charts did not detect a trend in anesthesia-related mortality over time. The majority of our data points are more than three standard deviations from the mean anesthesia-related mortality of 2.2 deaths per 10,000 anesthetics, which represents 927 anesthesia-related deaths that occurred in 4,279,177 anesthetics reported in the literature. Clearly, anesthesia-related mortality measured worldwide is not a stable system. A system that is unstable does not have a definable capability. In other words, one cannot detect trends in anesthesia safety in terms of anesthesia-related mortality until special causes of variation have been removed. These special causes of variation may represent real differences in anesthesia safety between the various samples, or just differences in the tools used to measure anesthesia-related mortality (e.g. , definitions, sampling methods).

In summary, current data suggests that the overall perioperative mortality rate for patients having ASA Physical Status 1–5 is approximately 1 per 500 anesthetics. The literature suggest a wide range of perioperative mortality rates, which are probably caused by differences in operational definitions and reporting sources, as well as a lack of appropriate risk stratification. Our current data, however, are consistent with reported perioperative mortality rates using the same definition and similar mandatory reporting for participating hospitals. Our data further suggest that the anesthesia-related mortality rate, as determined by peer review, has been stable over the last decade at approximately 1 death per 13,000 anesthetics. Wide variations based on methodological differences reported in the literature make it impossible to detect trends in anesthesia safety.

Based on these findings, the recommendations are quite simple. It is time to tell the emperor that he is not wearing any clothes. We must dispel the myth that anesthesia-related mortality has improved by an order of magnitude. Science does not support this claim. We must then begin our efforts to standardize our methodology of data collection and analysis so that we can share data worldwide. Large international data pools will allow us to develop risk adjustment models and identify best practices. Only then can anesthesia become a model of safety.