A Quality Improvement Initiative to Reduce Adverse Effects of Transitions of Anesthesia Care on Postoperative Outcomes: A Retrospective Cohort Study

This protocol was approved by the Wake Forest University School of Medicine Institutional Review Board (IRB00057263) and deemed exempt from a requirement for written informed consent under category 4, as a medical records study involving no use or disclosure of protected health information. The data extraction and analysis plan was written and filed with the institutional review board before data were accessed. All data elements were extracted from our electronic health record (Epic; Epic Systems, USA) at Atrium Health Wake Forest Baptist Medical Center (Winston-Salem, North Carolina). At our academic institution, anesthesia is provided by a care team model consisting of an attending anesthesiologist medically directing either resident physicians or certified registered nurse anesthetists (CRNAs), variably accompanied by student registered nurse anesthetists or medical students. We followed the Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) guidelines for reporting on quality improvement.⁹  The Strengthening Reporting of Observational Studies in Epidemiology (STROBE) checklist was used in preparing this manuscript.¹⁰ 

The study population included all adult (more than 18 yr of age) patients undergoing noncardiac surgery of at least 1 h anesthesia care duration in the inpatient operating room suite at the main university health sciences campus of Atrium Health Wake Forest Baptist Medical Center with general or regional anesthesia from January 1, 2016, to December 31, 2021. We excluded 3,047 patients with American Society of Anesthesiologists (ASA; Schaumburg, Illinois) Physical Status classification V to VI and 4,449 patients undergoing cardiac surgery. A total of 121,077 patients’ records were included in the analyses, and a flow diagram of the study population is shown in Supplemental Figure 1 (https://links.lww.com/ALN/D367).

We defined an intraoperative handover of care to have occurred if the attending anesthesiologist or medically directed individual changed during the anesthetic, defined as anesthesia start to anesthesia end as documented in the electronic medical record. A departure or change of a student registered nurse anesthetist or medical student was not considered a handover. We did not consider breaks, defined as absence of a directed provider for 35 min or less, to be a handover unless the case ended during this interval. Any change in attending anesthesiologist, without regard to duration, was treated as a handover. We treated handovers as binary events for a given case, either occurring or not occurring; we did not consider the number of handovers for a given case in the analyses.

Our primary outcome was a composite of all-cause mortality in the 30 days after surgery and occurrence of postoperative morbidity (“morbidity composite”), which includes Centers for Medicare & Medicaid Services (Baltimore, Maryland) Patient Safety Indicator 90 (PSI-90) and Hospital Acquired Conditions and other in-hospital morbidities. A list of the conditions triggering coding of this morbidity composite outcome is listed in Supplemental Table 1 (https://links.lww.com/ALN/D370) and is consistent with those included in previous studies of intraoperative anesthesia handovers.^2,5  Secondary and exploratory outcomes included morbidity composite, 30-day all-cause mortality, 1-yr all-cause mortality, safety event (activation of the rapid response team or cardiac arrest team), unplanned intensive care unit admission, hospital readmission within 30 days after discharge, remaining intubated at case end, postoperative reintubation, length of stay index more than 1 (indicating actual length of stay higher than predicted length of stay), postoperative visual or verbal analog pain score greater than 5 of 10, and emergency department visit within 30 days after discharge. Mortality data are regularly validated against the North Carolina death registry and refreshed in the electronic health record. Patients were followed for 1 yr after the index operation, unless death occurred during this interval; these patients were followed for (mean ± SD) 94 ± 108 days.

In the spring of 2019, we conducted a preliminary analysis of the effect of handovers on patient outcomes and concluded that there was likely to be a positive association upon formal analysis (no significance level adjustment was made for this analysis). The institution participates in the Multicenter Perioperative Outcomes Group’s quality arm, the Anesthesiology Performance Improvement and Reporting Exchange. The metric assessing percentage of cases in which a structured handover tool was utilized showed low use compared to peer institutions. On the basis of these findings, the department undertook a quality improvement initiative to promote the use of a structured tool built into the Epic electronic medical record during all permanent handovers of care, with documentation in the record. The intervention began in May 2019 with presentation of the preliminary data and use of the tool. The formal initiative was announced in June 2019 and began in July 2019. Data on use of the handover tool were reported to anesthesiology faculty monthly, and periodically to CRNAs and residents, and universal use of the tool was encouraged. Email appeals for increased use were sent to all medically directed providers in December 2019. The handover tool includes a case summary including visibility of the preoperative note, preoperative medications, airway management, regional anesthetic blocks, intravenous and arterial catheter details, current fluid totals and estimated blood loss, blood availability, free text “quick notes,” anesthesia events and times, and prompts to discuss anticipated changes, immediate patient needs, provider concerns, and postoperative plans including extubation, patient destination, and plans for management of postoperative analgesia and postoperative nausea and vomiting (fig. 1). In addition, the graphical display shows all medications administered with running case totals and patient hemodynamics. An icon on the anesthesia record illustrates the time of the current and any previous handovers, and the presence of such an event was counted as use of the tool.

Patient- and case-level data used in the study as covariates are listed in table 1 and included basic patient demographics (age, sex, body mass index, self-identified race as Caucasian [yes/no], self-identified ethnicity as Hispanic [yes/no]); Charlson Comorbidity Index as measure of patient comorbidity; patient admission and surgical case characteristics (ASA Physical Status, emergent surgery [yes/no; defined by inclusion of the “E” modifier to the recorded ASA Physical Status], whether the surgery was elective [yes/no], day/time of surgery, delay in minutes in starting the surgery from original scheduled time, duration of anesthesia, general anesthesia used in the operation [yes/no], surgical specialty); and intraoperative factors (number of surgeons for the surgical procedure, total surgeon work relative value units, deduced procedural severity score of the principal procedure,¹¹  attending anesthesia service provider concurrency, anesthesia service started by nurse anesthetists [yes/no], attending anesthesia service provider specialty trained, estimated blood loss during the operation and use of vasoactive infusion [phenylephrine, norepinephrine, epinephrine, or vasopressin infusion; yes/no]). Covariate body mass index was missing in 5387 patients (4.23%), and ASA Physical Status was missing in 172 patients (0.14%), and was addressed utilizing fivefold multiple imputation methodology using Multivariate Imputation by Chained Equations¹²  in R Studio, using 14 variables from table 1.

Statistical analysis was performed in R v3.6.1 (R Foundation for Statistical Computing, Austria) using the RStudio environment v1.1.456 (RStudio, USA) and IBM SPSS, version 24 (IBM, USA). All categorical covariates were evaluated initially using the chi-square test to determine their association with the primary outcome at a univariate level. Continuous covariates were initially evaluated for normality using the Shapiro–Wilk normality test and Kolmogorov–Smirnov goodness-of-fit test. Normally distributed data are presented as means with SDs, and data that were not normally distributed are presented as medians with interquartile ranges. Normally distributed continuous covariates were compared at the univariate level using an independent, two-sample, two-tailed t test, while the Mann–Whitney U test was used for comparison of nonnormal data on the preintervention and postintervention data sets. Multicollinearity among potential confounding variables was assessed using variance inflation factor and condition index, with values less than 10 indicating acceptability. No confounders were eliminated based on this criterion.

We deduced the intensity of the procedure by the procedural severity score using the risk quantification index developed by Dalton et al.¹¹  The Current Procedural Terminology code of the primary procedure for the case was extracted from the case billing information. We derived the procedural severity score based on combined risk for mortality and morbidity for each logical Surgical Current Procedural Terminology group.

A set of multiple logistic regression models was used to estimate the associated odds ratio (OR) for the composite primary outcome and additional secondary outcomes associated with intraoperative anesthesia service handover during the entire study period, both unadjusted and adjusted for other potential risk factors. Parameter estimation was done using robust sandwich asymptotic variance estimator for linear regression with the sandwich package in R. The P value for significance level within the primary and secondary outcome analyses was set at 0.004 for each group of analyses after Bonferroni correction for multiple comparisons, based on 12 total comparisons between handover and nonhandover groups.¹³ 

To evaluate the association between intraoperative transfer of care and our primary composite and secondary outcomes, we performed an inverse probability of treatment weighting (IPTW) analysis using the probability that a patient was exposed to a handover as propensity scores. The propensity score was used to calculate weights using generalized boosted regression. Patients with a probability value of 0 or 1 for exposure to intraoperative handover were excluded from the analyses based on the positivity assumption.¹⁴  In addition, extreme weights greater than the ninety-ninth percentile or less than the first percentile were replaced with the value of the ninety-ninth percentile or the first percentile, respectively. Balance in variables between the two groups before and after weighting was evaluated by calculating the standardized mean difference. The covariates used as contributors to the propensity score are shown in table 1 with their respective standardized mean differences before and after weighting. We then calculated the OR and 95% CI of experiencing a handover on the primary and secondary outcomes before and after IPTW.

The effect of the quality improvement intervention for handover was analyzed using a “before-after” approach. This technique, however, has potential biases such as the Hawthorne effect and regression to the mean.¹⁵  To limit these biases, we examined the change in the weighted point estimate of the OR¹¹  of the primary outcome for a handover quarterly, using an interrupted time series analysis with autoregressive integrated moving average modeling.¹⁶  We hypothesized that there would be a change in slope in the OR versus time plot with the gradual uptake of the structured handover tool. Initial autoregressive and moving-average structures were chosen based on examining the autocorrelation and partial autocorrelation of the time series. The resulting model for autoregressive integrated moving average was (1,0,0), indicating autoregression over one lag, no differencing, and no moving average.¹⁷  The model’s stability and fit were evaluated with stationary R² and Ljung–Box statistics.¹⁸  The residuals were examined to ensure correct model choice, and the model with lowest residuals was selected.

Preliminary analysis for the cases during the preintervention period (2016 to 2018) indicated there were 25,061 handover cases and 33,148 nonhandover cases, with 8% and 5% incidences of the composite study outcome, respectively. Assuming the midpoint 6.5% incident rate, the observed sample of size of approximately 121,000 would yield 98% power at the 0.05 significance level to detect an OR of 1.04 or larger.^19,20  We also estimated the post hoc power of the interrupted time series analysis, using two approaches based on observed effect size, number of samples per epoch, and number of epochs.^21,22  We determined the power of the analysis was greater than 80%.

We performed three sensitivity analyses. First, we defined subgroups of patients who experienced an anesthesia handover between attending physicians only (“attending handover”), those intraoperative handovers among resident physicians or CRNAs (“directed clinician handover”), and those in which both the attending physician and the directed practitioner occurred (“both handover”). The association of study outcomes and type of handover was performed using IPTW analysis for each handover type. Second, we investigated the interaction of handovers for different study outcomes using the IPTW and further adjusted for anesthesia duration and anesthesia start hour, which were not fully balanced by the weighted score, number of surgeons, and pre- or postintervention period (neither of which were included in the propensity score). Finally, we conducted multivariable binary logistic regression for various study outcomes using backward stepwise methodology for variable inclusion,²³  where covariates that were strongly associated with outcome (P < 0.05) were applied in a conventional logistic regression model, to validate study findings.

A total of 121,077 surgical cases from January 1, 2016, to December 31, 2021, met the study inclusion criteria. Of these, 48,986 (40.4%) cases experienced a handover. The anesthesia handover frequency during the preintervention period was 41.1%, while during the postintervention period, it was 39.4% (P = 0.435). CRNAs were the initiating directed provider in 81.2% of cases with a handover and 95.2% in nonhandover cases. Handover cases tended to be longer, be delayed and start later in the day, involve a resident physician, utilize general anesthesia, be nonelective, involve more surgeons, have higher blood loss, utilize vasoactive drugs, and have a higher procedure severity score than nonhandover cases (standardized mean difference greater than 0.1; table 1). The demographics and case characteristics are shown in table 1 with the unadjusted and IPTW weighted standard mean differences for each variable. IPTW balanced all potential covariates (standard mean difference less than 0.1) except anesthesia start hour and duration of anesthesia episode in minutes (table 1).

Logistic regression analysis of the weighted cohort showed an increase in the odds of the primary outcome in intraoperative handover cases with OR 1.08 (95% CI, 1.04 to 1.12; P < 0.001). When further adjusted for the two confounders that were not fully balanced by the propensity score (anesthesia start hour and duration of anesthesia episode in minutes), the number of surgeons involved (which might reflect surgical complexity and might influence the composite outcome as well), and an indicator for before/after the intervention, the OR for the composite outcome remained statistically significant (OR, 1.07; 95% CI, 1.03 to 1.09; P < 0.001). The IPTW-adjusted ORs for all secondary outcomes except remaining intubated at case end and 1-yr all-cause mortality were likewise significantly increased for intraoperative handovers (fig. 2; Supplemental Table 2, https://links.lww.com/ALN/D371, and Supplemental Table 3, https://links.lww.com/ALN/D372). The absolute event numbers and rates are shown in figure 2 for handover and nonhandover cases, and the counts and rates of the composite outcome by quarter are shown in Supplemental Table 5 (https://links.lww.com/ALN/D374), illustrating the effect of the declining OR.

Figure 3A illustrates the quarterly autoregressive moving average plot of the fraction of intraoperative handover cases using the structured handover tool; the initial intervention date (quarter 2 2019) is noted. There was a statistically significant increase in fraction of usage of the structured handover tool after the intervention date (t = 4.78; P < 0.001). Quarterly usage for the last year after intervention compared to the first year after intervention showed a statistically significant increase (P < 0.001).

The quarterly plot of the change in the point estimate of the OR for the composite outcome for handover cases versus nonhandover cases is shown in figure 3B. There was a statistically significant decrease in odds after the initial intervention (t = –3.97; P < 0.001). The slope of the OR versus time curve was 0.002 (95% CI, 0.001 to 0.004; P = 0.018) in the preintervention period, declining to –0.011 (95% CI, –0.01 to –0.018; P < 0.001) after the intervention. The confidence limits for the OR for the final four quarters of the study included 1.0.

The quarterly plot of the change in frequency of handover cases is shown in figure 3C. There was a nonsignificant decrease in the fraction of handover cases after the intervention date (t = –0.949; P = 0.355). A similar pattern and statistical result were observed when analyzing the data by month rather than by quarters (Supplemental Fig. 1, https://links.lww.com/ALN/D367, and Supplemental Fig. 2, https://links.lww.com/ALN/D368).

The effect of handover of care by the attending physician only, the medically directed clinician only, or both team members was similar across the composite and secondary outcomes, with a suggestion that handover of both personnel may be associated with a higher risk (Supplemental Table 4, https://links.lww.com/ALN/D373). When forcing anesthesia start hour and duration of anesthesia episode, number of surgeons, and pre- versus postintervention as independent variables into the IPTW logistic regression model, the odds of the primary outcome remained similar (OR, 1.07; 95% CI, 1.03 to 1.09; P < 0.001; table 2), as did the odds of the secondary outcomes (Supplemental Table 3, https://links.lww.com/ALN/D372). The backwards stepwise logistic regression analysis of primary (table 2) and secondary (Supplemental Table 3, https://links.lww.com/ALN/D372) outcomes showed similar results to the IPTW analysis.

A summary of the OR and 95% CI for the three main and one sensitivity models, for the composite outcome and its major components, is shown in table 2.

The results of this investigation confirm the risks associated with intraoperative handovers of anesthesia care as seen in most, although not all, previous investigations.^{1,2,5,24–26}  After controlling for multiple confounding factors by propensity score–based IPTW, handovers were associated with an increase in our primary outcome, a composite of 30-day mortality and major postoperative morbidity. Most secondary outcomes were also significantly associated with handovers of anesthesia care. The risk persisted even after forcing the time of day the case began and the duration of the anesthesia episode into the model, in addition to the IPTW value. In the exploratory analysis of type of handover, change in the attending physician, the medically directed personnel, or both were each associated with adverse outcomes, with a suggestion that changeovers of both personnel were riskier.

Perhaps more notably, institution of a structured handover tool may have ameliorated the risk associated with a change in personnel. Use of such a tool in the Epic electronic health record increased from approximately 30% to more than 90% during the 2 yr after the institution of the quality improvement initiative (fig. 3A). The frequency of handovers overall, however, did not change (fig. 3C). Coincident with this increased use, we observed a significant decline in the OR for the composite outcome (fig. 3B), with the CI of the OR including 1.0 in the final four quarters.

Our observed risk of a handover is consistent with the results of Saager et al.,²  who found an OR of 1.08 (95% CI, 1.05 to 1.10) for in-hospital mortality or major morbidity for one handover of care compared to none. These investigators used multivariate logistic regression to control for a similar range of confounders as in our study. Similarly, Jones et al.¹  observed an increase in 30-day mortality or major morbidity in a province-wide cohort in Ontario, Canada, using IPTW to control for potential confounders. Other investigations have also found adverse effects, using different patient populations and outcomes.^24–26  Conversely, some studies have not found such an association. Terekhov et al.⁵  found that a collapsed composite of hospital mortality and major morbidity was not associated with the number of handovers of anesthesia care. Shorter transitions (breaks) appeared to be associated with improved outcomes. Notably, they observed a much lower frequency of handovers (approximating national average data at the time of publication in 2016 of 6%), whereas Saager et al.²  observed a 39% incidence, similar to our observed frequency. Moreover, Terekhov et al. noted that the institution had recently undertaken an initiative to improve operating room to postanesthesia care unit transitions using a structured handover tool and provider education, possibly biasing their observations toward the null because of learning contamination bias.

It is in this latter arena that our data are most provocative. Several previous investigations have shown improved transfer of information between anesthesia providers when a structured checklist was utilized.^27–29  However, these process improvements have been at best surrogates for improved patient care, and recent reviews and a national consensus panel found no direct such evidence and encouraged an emphasis on finding it as a top priority.^30–32  Recently, implementation of the “I-PASS” structured handover tool in medical and pediatric hospitalist services (accompanied by extensive education and training of personnel) was associated with a reduction of self-reported “handoff-related adverse events.”³³  Our data suggest that in the setting of a high frequency of anesthesia handovers, use of a structured tool may reduce the risk of the change of personnel.

However, our study also has significant limitations. First, the retrospective nature of the study and the temporal correlation of structured handovers with improving outcomes is not a demonstration of causality. Only a randomized trial, most likely in a block design by institution rather than at the individual provider level (to avoid learning contamination bias), could support a causal mechanism. Notably, the only randomized clinical trial of anesthesia handovers failed to demonstrate adverse outcomes, although the lack of information regarding who actually handed over and the impossibility of blinding the affected personnel to whether a handover was part of usual operations versus a randomized event complicates interpretation of this result.^34–36  Second, our observed decline in relative risk was modest, although sustained during 2 yr. A longer-term follow-up, demonstrating persistence of the salutary effect over time, would strengthen our conclusion. Further, we utilized a composite outcome, and it is possible that results for individual components may be fragile. Third, our study, like most in this field, is a single-institution investigation. It is likely that practice style, culture of safety, and staffing and relief models could influence the effects of handovers and structured tools. In particular, the overall frequency of handovers should be investigated as an independent predictor of their effect on outcomes. Validation of our results in other institutions would be helpful in this regard. Fourth, learning contamination bias could diminish the effect of a structured tool over time if clinicians learned to perform a better handover during its original introduction, even if it were subsequently omitted. Moreover, we made no attempt to evaluate handover quality or verify that the tool was actually used appropriately, but handovers in which the tool was activated but not utilized would tend to bias the results to the null, strengthening our conclusions. Finally, we did not differentiate the risks associated between handovers involving resident physicians versus CRNAs, nor did we study a dose–response relationship by analyzing the effect of number of handovers.

In conclusion, we have confirmed an adverse effect of intraoperative handovers of anesthesia care. Promotion of a structured handover tool was temporally associated with a reduction in the apparent risk of such transitions. Further research should be undertaken to validate our provocatively positive findings.

The authors wish to thank Ashley L. Talbott, M.D., Wake Forest University School of Medicine (Winston-Salem, North Carolina), for her inspiration and support for improved anesthesia handovers, and Addie Larimore, B.A., Wake Forest University School of Medicine, for editorial assistance in formatting the manuscript.

Support was provided solely from institutional and/or departmental sources.

The authors declare no competing interests.

Supplemental Figure 1: Study flow diagram, https://links.lww.com/ALN/D367

Supplemental Figure 2: Percentage of handover cases using the structured handover tool by month of the study period, https://links.lww.com/ALN/D368

Supplemental Figure 3: Odds ratio for the composite outcome for handover cases versus nonhandover cases by month, https://links.lww.com/ALN/D369

Supplemental Table 1: Components of the morbidity composite, https://links.lww.com/ALN/D370

Supplemental Table 2: Primary and secondary outcomes after inverse probability of treatment weighting, https://links.lww.com/ALN/D371

Supplemental Table 3: Primary and secondary outcomes in fully adjusted model, https://links.lww.com/ALN/D372

Supplemental Table 4: Primary and secondary outcomes by type of handover, https://links.lww.com/ALN/D373

Supplemental Table 5: Composite event rate by quarter, https://links.lww.com/ALN/D374