Frequency and Type of Situational Awareness Errors Contributing to Death and Brain Damage: A Closed Claims Analysis

ACCURATE situational awareness is considered a core element for the nontechnical skills of decision-making, team work, and task management.^1–5  Situational awareness can be described as a hierarchical model where the basic level of perception (i.e., detection) describes the degree to which relevant information (e.g., patient history, information provided by monitors such as blood pressure or oxygen saturation) is perceived by the anesthesiologist. Once the information is perceived, the anesthesiologist integrates this information with long-term memory content to comprehend (i.e., diagnose) the patient’s medical condition (e.g., hypovolemia, pneumothorax). At the highest level, projection, the anesthesiologist projects (i.e., predicts) the patient’s state into the near future, allowing for planning ahead and thus for managing resources relevant for optimal treatment.^1,4 

Situational awareness errors in medical care can lead to patient harm. In 2009, the World Health Organization recognized that situational awareness plays an important role in clinical decision-making by physicians.⁶  Situational awareness errors may occur in surgery and anesthesia.^5,7–10  Studies in surgery using observational techniques and review of operative notes suggested that complications of laparoscopic cholecystectomy (e.g., bile duct injuries) and carotid endarterectomy were associated with situational awareness errors, especially involving perception.^5,7,8  In anesthesia, investigators in Australia⁹  and Germany¹⁰  reviewed voluntary critical incidents not resulting in patient injury and found a significant number of situational awareness errors (40 and 82%, respectively). In primary care¹¹  and nursing,^12,13  situational awareness errors were studied qualitatively, but without quantitative data. The role of situational awareness errors in major injury associated with anesthesia has not been previously investigated.

This study investigated the role of situational awareness errors by an individual anesthesiologist in major patient injury (severe brain damage and death) using anesthesia malpractice claims. Due to the lack of validated classification of situational awareness in clinical anesthesia, our first study aim was to develop detailed protocol to operationalize definitions of perception, comprehension, and projection errors based upon the scientific literature.^11,14  To assess the role of cognitive skills involving situational awareness, we focused on claims involving a single anesthesiologist (rather than an anesthesia care team) to avoid misclassification with complex interactions with communication, team work, and different types of training among members of an anesthesia team. We used the Anesthesia Closed Claims Project database to examine trends in permanent brain damage and death. We then studied the frequency and type of situational awareness errors in these malpractice claims where patients died or suffered severe brain damage between 2002 and 2013. For those cases judged as having situational awareness errors, data are provided with respect to the patient and case characteristics, the cause of injury, and associated anesthesia payments.

The Anesthesia Closed Claims Project database is a structured collection of closed anesthesia malpractice claims described in detail elsewhere.^15,16  Briefly, on-site anesthesiologist-reviewers abstracted data from closed anesthesia malpractice claims onto detailed data collection instruments at participating professional liability companies across the United States. The panel of 16 companies (at the time of this study) insured approximately one third of practicing anesthesiologists in the United States. Information was collected from medical records, consultant evaluations, expert witness reports, claims manager summaries, and legal summaries. Data collected included patient demographics, type of surgery, details regarding anesthesia care, patient outcomes, and legal outcomes. The on-site reviewer evaluated the outcome, severity of injury, and cause of injury (i.e., damaging event), and summarized the claim in a brief narrative, including the sequence of events and causes of injury. The Closed Claims Project Investigator Committee reviewed the claims, and any disagreements in assessments were resolved by committee members.

For this study, we used the Anesthesia Closed Claims Project database of 10,546 claims. Inclusion criteria were claims associated with surgical or procedural anesthesia care. Claims associated with obstetric anesthesia (including cesarean section) and pain medicine (acute or chronic) were not included. Additional inclusion criteria were based on the nature and phase of anesthesia care and the type of damaging event leading to injury. Claims with anesthesia-related damaging events that occurred intraoperatively were included; claims with injuries attributed to patient underlying condition, purely surgical events, or ambiguous events that could not be classified as to clinical cause were not included. Claims with damaging events occurring during other phases of anesthesia care (not intraoperative care) or outside the operating room were not included. As the study focused on situational awareness errors in individuals, only claims with care by a single physician anesthesiologist (rather than an anesthesia care team) were included. Analysis of trends in severe injury over time utilized all claims meeting inclusion criteria. In-depth analysis of the role of situational awareness error in severe injury was restricted to claims for death or severe brain damage that occurred in the years 2002 to 2013 that otherwise met inclusion criteria (fig. 1).

The damaging event in each claim was classified according to the primary mechanism that resulted in injury. Damaging events were grouped into the following categories: respiratory, cardiovascular, medication, equipment, regional block-related, or other. Examples of respiratory system events include difficult intubation, inadequate ventilation or oxygenation, and pulmonary aspiration. Examples of cardiovascular events include hemorrhage, electrolyte or fluid derangement, stroke, myocardial infarction, and pulmonary embolism. Medication events include wrong drug, wrong dose, and adverse drug reactions including malignant hyperthermia. Equipment events include cautery fires, central line insertion or management issues, anesthesia gas delivery equipment problems, and airway device injuries. Regional block-related events include intravascular injection or absorption of local anesthetics and high blocks.

A situational awareness error in this study was defined as lack of situational awareness that contributed to patient death or brain damage. Two authors (C.M.S. and A.B.) assessed each claim for whether a situational awareness error occurred and whether it contributed to death or brain damage. Assessments were collected using REDCap (Research Electronic Data Capture)¹⁷  electronic data capture tools through a standardized questionnaire. An acceptable level of reliability (κ = 0.69) was achieved on a single round of assessment of a random sample of 100 cases. Disagreements were resolved by correspondence and/or discussion between these two authors, with resolution of remaining disagreements (n = 8) by a third author (K.B.D.).

If a situational awareness error contributed to death or brain damage, the team members involved in the error (anesthesia vs. surgical or other) were identified. Levels of situational awareness error were assessed for claims with errors by an anesthesiologist. The levels of situational awareness error were defined as perception (failure to perceive or misperception of information), comprehension (improper integration or comprehension of information), or projection (incorrect projection of foreseeable future trends). Given a lack of published operationalized definitions and examples of situation awareness levels in surgical or procedural anesthesia, we extended Singh’s adaptation of Endsley’s model to medical diagnosis to construct definitions and examples of levels of situation awareness in the clinical context of surgical and procedural anesthesia.^11,14  We pretested our protocol (table 1) on a subset of situational awareness error claims from the initial random sample of 100 cases, refining the definitions and examples on additional subsets of claims with situational awareness error until an acceptable level of interrater reliability was achieved. Acceptable reliability (κ = 0.40) for levels of situational awareness error was achieved in round 3. Although this met our a priori study measurement criterion, we decided to further refine the protocol to achieve a better level of reliability (κ = 0.53) with one additional round of coding using another subset of 25 claims.

We defined an error of perception as failure to gather information via history, patient chart, physical exam, diagnostic tests, imaging, or monitors, including absence of monitor(s) (table 1 provides the protocol with clinical examples). Following the Endsley model of situational awareness as a dynamic process, we defined perception as encompassing both the active seeking of information as well as the passive receiving of information.^18,19  Therefore, failure to seek information and failure to receive information were both defined as perception errors. Comprehension errors were defined as failure to understand the significance of information obtained from history, physical exam, diagnostic tests, imaging findings, or monitors; that is, the information was available, but it was not understood or was misunderstood, which led to an incorrect diagnosis. The definition of comprehension errors relied heavily on the model put forth by Singh with its emphasis on diagnosis.¹¹  Projection errors were failure to forecast future events or scenarios based on a high-level understanding of the situation.¹⁸  This included failure to forecast potential outcomes based upon a chosen plan, poor contingency planning, or poor/absent back-up plan. Because an error of perception nearly always leads to errors of comprehension and projection, if there were situational awareness errors at multiple levels, the lowest level of anesthesia situational awareness error was coded.²⁰  The same two authors (C.M.S. and A.B.) assessed the level of situational awareness error, and disagreements (n = 3) were resolved by discussion or the third author (K.B.D.).

The severity of injury in each claim was assigned using the National Association of Insurance Commissioners’ 10-point scale, which ranges from 0 (no apparent injury) to 9 (death).²¹  In this study brain damage was defined as brain damage with permanent disabling injury (score 6 to 8). The severity of injury represents the assessment at the time the claim was closed. A claim with severe brain damage resulting in death before claim closure was classified as death.

Interrater reliability for the assessment of situational awareness error versus no error (binary assessment) and interrater reliability for levels of situational awareness error (classification into one of three response categories) were measured by independent assessment of random samples from the study data set using κ scores calculated on the initial two author judgments before discussion or tie-breaking by the third author. κ values were calculated based on expected versus observed probabilities across response categories. These values may be biased by a large number of response choices (resulting in low expected probabilities and rendering high κ values more likely) or highly uneven use of response choices (resulting in high expected probabilities, rendering high κ values difficult to achieve). With two and three response categories and fairly even distribution of responses in our data, κ scores above 0.4 were considered acceptable.²²  All payments made to the plaintiff were extracted from the database and adjusted to 2015 dollar amounts with the Consumer Price Index.²³  Median and interquartile range were reported for payments because they were not normally distributed. Claims with no payment were excluded from calculation of median and interquartile range. Trends in death and brain damage over time were reported as percentages of all claims per year meeting inclusion criteria. Years before 1980 with fewer than 50 total claims were combined sequentially to form groups with at least 50 claims. All claims from these years have closed and been entered into the database. The database is incomplete for events that occurred since 2004, as many of those claims are still open, and some that have closed have not yet been collected and incorporated into the database; the rates of death and brain damage in 2009 to 2013 were calculated but may change considerably as additional claims from those years are added to the database. The years 2012 to 2013 have been combined due to very small denominators in those years. Characteristics of anesthesia situational awareness error claims were compared to all other claims meeting study inclusion criteria (“other claims”: no situational awareness error, surgical situational awareness error, or insufficient information) using chi-square test, Fisher’s exact test, Student’s t test for equality of means (age), or Mann–Whitney U test (payment amount). All statistical analysis employed SPSS 22 for Windows (IBM Corporation, USA) with P < 0.05 as the criterion for statistical significance and two-tailed tests. No a priori power calculation was conducted; the analyses were based on the available data.

The proportion of claims for death or severe brain damage declined from 73% (n = 37) of 51 claims in 1970 to 1976 to 36% (24 of 67) in 1989, remaining in the 36 to 38% range through 1998. From 1999 to 2008, the proportion of death and severe brain damage varied from 41 to 52% (391 of 850, averaging 46%; fig. 2). In the years 2002 to 2013, 198 (74%) out of 266 claims had a situational awareness error by an anesthesiologist (e.g., errors) that contributed to patient death or brain damage (κ = 0.69). The remaining 68 (26%) claims in 2002 to 2013 included claims with no anesthesia error contributing to death or brain damage (n = 50), surgical errors (n = 10), and claims with insufficient information (n = 8). Among 198 claims with anesthesia errors, errors of perception were most common (n = 83, 42%), whereas errors of comprehension (n = 58, 29%) and projection (n = 57, 29%) were relatively less common (κ = 0.53). There was no difference by year of event in the relative frequency of error claims compared to other claims (P = 0.71).

Patients in claims with errors tended to be younger than patients in other claims by an average of 6 yr, but most were middle-aged (mean 50 vs. 56 yr, P = 0.023; table 2). There were no other associations between patient characteristics and errors. Errors were not more common in emergency versus routine cases (table 2). The primary damaging event differed between error claims and others (P < 0.001; fig. 3). In claims with errors, respiratory events were dominant (56%), whereas the most frequent primary damaging events in claims without errors were cardiovascular (47%; fig. 3). The most common specific respiratory events in claims with errors were inadequate oxygenation or ventilation (24% of errors), difficult intubation (11%), and pulmonary aspiration (10%; table 2). The most common respiratory events in claims without anesthesia errors were pulmonary aspiration (9%) and difficult intubation (6%). Hemorrhage was the predominant cardiovascular system event in claims with errors (10%). In claims without errors, the most common cardiovascular events were hemorrhage (12%), myocardial infarction (6%), and pulmonary embolus (6%; table 2).

Payments made on behalf of the anesthesiologist and/or anesthesia corporation group were significantly more frequent in claims with errors (85%) than in other claims (46%, P < 0.001; table 2). When a payment was made, the size of payment did not differ between claims with errors and other claims (table 2).

Greater than half (60%) of the 83 perception errors involved respiratory system events, whereas 20% were cardiovascular system events. Respiratory events resulting from errors of perception were characterized by an absence of respiratory monitoring resulting in inadequate oxygenation and/or ventilation (n = 37; table 3). Lack of end-tidal carbon dioxide and/or peripheral capillary oxygen saturation monitoring and absent alarms led to missing information and consequently failure to perceive the patient’s declining respiratory status, often until after cardiac arrest occurred. Perception errors also included inadequate preoperative evaluation and missing preoperative information that led to inappropriate anesthesia plans and management (table 3).

Errors of comprehension (n = 58) were equally divided between respiratory and cardiovascular events (n = 23 each). Respiratory system events that were not comprehended appropriately included inadequate oxygenation or ventilation, difficult intubation, bronchospasm, and pulmonary aspiration. In multiple cases of difficult intubation, the anesthesiologists clearly perceived the difficulty as multiple attempts at intubation were described, but the extent of ongoing difficulty was not adequately comprehended as intubation attempts continued without changes in technique while the patient deteriorated. In these elective cases, the patient could have been awakened or a laryngeal mask airway used as a bridge for airway management, but the full difficulty of the airway management was not comprehended, and these steps were not taken. Errors of comprehension of inadequate oxygenation or ventilation were characterized by a similar lack of understanding of the ongoing seriousness of the clinical situation and continued deterioration of the patient. Half (n = 11) of the cardiovascular system events with comprehension errors involved hemorrhage, with a typical example presented in table 3.

Errors of projection (n = 57) most commonly involved failure to anticipate or plan for difficult airway management (n = 15) and pulmonary aspiration (n = 11), which together accounted for approximately half of the projection errors. Difficult airway management projection errors included both lack of plans for intubation on induction and lack of plans for extubation of patients who had been difficult to intubate on induction (table 3). Planning routine anesthetics that did not take into account severe patient comorbidities, often in ambulatory surgery centers or office locations, were also common errors of projection. Other projection errors involved lack of planning for fire prevention in high-risk cases (cautery plus open oxygen plus procedure above the chest).

Our study suggests that situational awareness errors may have contributed to catastrophic outcomes in three quarters of anesthesia malpractice claims in which patients died or sustained permanent brain damage from 2002 to 2013. Consistent with a model describing the crucial role of situational awareness in anesthesia,¹  these findings suggest that lack of situational awareness may play an important role in the genesis of patient harm.

Previously, using all available closed claims, Cheney et al.²⁴  identified a decrease in the proportion of anesthesia malpractice claims for death and brain damage between 1975 and 2000. Our study identified a plateau in the proportion of claims for death and severe brain damage in anesthesia malpractice claims involving intraoperative care by a single anesthesiologist extending through 2013. The reasons for this are not clear, and potential factors such as increasing patient comorbidities were beyond the scope of the current inquiry. However, this finding suggests that further improvements in anesthesia patient safety require new theoretical approaches to analyze adverse events. Situational awareness represents one such approach.¹ 

This is the first study to develop a detailed protocol to assess the role of situational awareness errors in anesthesia-related severe injury. Using a detailed protocol with anesthesia-specific examples (table 1), interrater reliability was substantial with respect to whether an error occurred (κ = 0.69) and acceptable (κ = 0.53) with respect to assessment of the specific level of error. We defined nonuse of anesthesia monitors as errors of perception. Unlike aviation where standard gauges are present in every cockpit, in anesthesia practice the decision to use or not use many monitoring devices is part of the anesthesia plan, based on the patient, procedure, and other factors. Although one could argue that the lack of some monitors such as capnography during monitored anesthesia care is a poor plan and therefore an error on the level of projection, we considered such errors as perception errors, based upon the situational awareness scientific literature.¹⁴  If a monitor is not used, the information is not available, similar to lack of specific preoperative tests that results in absence of information for potential perception. This definition of perception errors is based on the premise that gathering information is an active process, beyond simple vigilance.^1,18,19 

The frequency of situational awareness errors in this study (74%) was consistent with the range in studies that investigated situational awareness errors in anesthesia incident reports (82%)¹⁰  and aviation (59 to 88%).¹⁴  The types of errors in malpractice claims differed in relative frequency from critical incident reports where there was a risk but no actual injury. In incident reports, perception errors were also the most frequent, but projection errors (15%) were almost twofold less frequent than in this study of malpractice claims (29%).¹⁰  The Australian study did not analyze levels of situational awareness.⁹  Different case characteristics, the voluntary character of the incident reporting system, and the subjective view of the reports significantly limit comparability.

Many of the perception errors stemmed from lack of, or lack of attention to, respiratory monitoring. This is consistent with the finding of Bhananker et al.²⁵  that 44% of claims due to oversedation during monitored anesthesia care were judged to be preventable by better use of monitoring or by additional monitoring. Another common perception error was missing preoperative information, often associated with inadequate preoperative evaluation (table 3). This finding speaks to a common anesthesia situation that, although not emphasized in previous analyses of anesthesia malpractice claims, represents an important safety risk to patients. Anesthesiologists may elect to proceed with a routine procedure in the absence of important preoperative history and test results. Although alternative analytic models could be applied to these cases, in the framework of situational awareness, they represent errors at the most basic level: missing information essential to comprehension and planning of a safe anesthetic course.

The most common comprehension errors involved respiratory and cardiovascular events (especially massive hemorrhage). Previous analysis of massive hemorrhage claims revealed team communication and management problems that led to death and brain damage.²⁶  The situational awareness theoretical approach provides new insights into diagnostic error or diagnostic delay of massive hemorrhage. This illustrates the value of different theoretical models for similar events: both models provide important clinical insights into anesthesia patient safety problems potentially demanding differing solutions.

The comprehension errors related to respiratory management most commonly involved lack of comprehension of the ongoing clinical difficulty and the need to change course. Clinical algorithms and emergency manuals may have potential in such situations by limiting the repetition of the same clinical approach when improvement is not forthcoming. This has been shown previously in an analysis of claims for difficult airway management in which claims for difficult intubation on induction of anesthesia after adoption of the American Society of Anesthesiologists difficult airway algorithm had improved outcomes compared to earlier claims.²⁷ 

The projection errors frequently involved anticipation of difficult airways and pulmonary aspiration. In addition, procedure planning in an inappropriate environment (such as very sick patients in offices or outpatient settings) is a liability risk that has not been revealed in previous closed claims analyses using alternative analytic frameworks. Fire risk is not a new finding but significant in the context of the current analysis that was restricted to claims for death and brain damage. Most on-patient fires involve burns, generally of lower severity than the claims included in this study.²⁸  The situational awareness framework provided a unique perspective with increased emphasis on the importance of anticipating fire risks and incorporating risk and prevention planning into the preoperative anesthesia plan and checklist.

Perception errors may be prevented by directing attention to additional or other more relevant sources of information. Regular scanning and processing of all the information available may avoid misdiagnosis based on heuristic reasoning.²⁹  Implementing standards that define a minimum of vital parameters for certain procedures or the World Health Organization surgical safety checklist³⁰  may have positive impact. Accurate assessment and improved availability of preoperative history and test results may prevent perception errors.

Comprehension and projection involve long-term memory content, which may be difficult to access during a critical incident. A “call for help” and use of cognitive aides (e.g., emergency checklists or manuals) may help.^31–34  Assistance from other anesthesiologists may provide additional processing capacity for the integration of basic information. Cognitive aids enhance comprehension and projection.^31,34  The situational awareness framework also offers systematic approaches to reduce errors,^35,36  such as the user-centric design of information and training curricula including simulation.³⁷  Situational awareness-relevant learning objectives are addressed in anesthesia crisis resource management education and include self-checking behavior and specific communication strategies.³⁶  Situational awareness-specific training principles focus, among other things, on strengthening mental models, trainee feedback, and common errors.^35,38 

This study has numerous limitations. The study analysis relied on narratives that were not prospectively collected with the intent to assess situational awareness errors in a standardized manner. Therefore, important information to determine underlying causes was missing. The nonrandom, retrospective collection of data may reflect selection and hindsight bias. In contrast to incident reports, the Anesthesia Closed Claims Project malpractice narratives were created by trained experts using a larger set of information and not just reports from single individuals. Nevertheless, it is likely that the poor outcomes within the claims bias the reviewers toward a very rigorous assessment of standards of care.³⁹  We did not consider the database assessment of appropriateness of care when we made our independent assessment of errors. As all claims in this investigation had poor outcome (death or permanent brain damage), any outcome bias would apply to the entire study sample. Another limitation is selection bias in that only selected companies were willing to provide claims data. Furthermore, there is always uncertainty as to the veracity of the medical documentation and depositions in malpractice claims.

The Anesthesia Closed Claims Project represents a large opportunity sample, and as such, the data do not necessarily reflect the actual quantity of situational awareness errors in U.S. anesthesia malpractice claims. Despite these limitations, using the situational awareness framework for the analysis of anesthesia closed malpractice claims provided information about the potential role of situational awareness in the genesis of patient harm in events that would be difficult to analyze prospectively. Other analytical frameworks such as educational needs assessment or different human factors models might reveal different safety problems and point to different solutions, but that does not negate the lessons learned through application of a situational awareness framework in this study. Although we did not address changes in errors over time, it nonetheless reveals the potential for situational awareness methods to contribute toward reducing catastrophic injury.

Lack of situational awareness may have contributed to a large proportion of recent catastrophic anesthesia malpractice claims, providing further support for the premise that situational awareness is a core element for anesthesia decision-making and patient safety. Operationalized definitions of situational awareness errors were developed and reliably applied to the analysis of real-world anesthesia cases with severe injury. We hope that this novel adaptation of methods from aviation to clinical medicine will stimulate further research and development of strategies to prevent patient harm.

The authors acknowledge the closed claims reviewers from the American Society of Anesthesiologists (Schaumburg, Illinois) and participation of the following liability insurance companies who have given permission to be acknowledged: Anesthesia Service Medical Group, Inc., San Diego, California; COPIC Insurance Company, Denver, Colorado; ISMIE Mutual Insurance Company, Chicago, Illinois; MagMutual Insurance Company, Atlanta, Georgia; Medical Liability Mutual Insurance Company, New York, New York; Midwest Medical Insurance Company, Minneapolis, Minnesota; NORCAL Mutual Insurance Company, San Francisco, California; Physicians Insurance, A Mutual Company, Seattle, Washington; Preferred Physicians Medical Risk Retention Group, Overland Park, Kansas; Risk Management Foundation, Cambridge, Massachusetts; State Volunteer Mutual Insurance Company, Brentwood, Tennessee; The Doctors’ Company, Napa, California; and the University of Texas System, Austin, Texas.

Supported in part by the American Society of Anesthesiologists, Schaumburg, Illinois, and the Anesthesia Quality Institute, Schaumburg, Illinois. All opinions expressed are those of the authors and do not reflect the policy of the American Society of Anesthesiologists or Anesthesia Quality Institute. REDCap (Research Electronic Data Capture) electronic data capture tools hosted at the University of Washington, Seattle, Washington, were provided by the Institute of Translational Health Science through grant No. UL1 RR025014 from the National Institutes of Health National Center for Research Resources, Bethesda, Maryland. Additional support was provided by institutional funding.

The authors declare no competing interests.