Behavioral Modification of Intraoperative Hyperglycemia Management with a Novel Real-time Audiovisual Monitor

HYPERGLYCEMIA is associated with poor outcomes in critically ill patients,^1,2  and tighter control of blood glucose (BG) is an established tenet of clinical management.³  Perioperative hyperglycemia is associated with increased hospital length of stay,⁴  morbidity,^5,6  and mortality after noncardiac general surgery.⁷  There is accumulating evidence that postoperative outcomes are influenced by perioperative hyperglycemia in patients with undiagnosed type 2 diabetes, calling attention on the need for appropriate glycemic control irrespective of diabetic status.^8–11  However, treatment thresholds for intraoperative hyperglycemia have not been defined as a standard of care, and little is known of the interindividual variability of intraoperative management of hyperglycemia. Recent reviews suggest that treatment of a moderate BG target between 150 and 200 mg/dl (8.3 to 10 mM) is associated with decreased postoperative morbidity and mortality.^2,12  The overall effectiveness of tight glucose control is still debatable, with several studies showing lack of benefit in critically ill patients.^13–15  On the other end of the therapeutic margin, intraoperative insulin treatment of hyperglycemia needs vigilant monitoring to avoid fatal hypoglycemia.¹⁶  Additionally, tight glucose control was associated with a significant increase in the incidence of severe hypoglycemia (13.2% vs. 6.2%).¹⁷  Thus, desirable intraoperative BG management includes treatment of hyperglycemia and rechecking BG values after treatment to ascertain treatment response.

Previous studies looking at computerized decision support systems did not show a change in nurse behavior of treatment of hyperglycemia in the intensive care unit.¹⁷  As there are no prior studies of intraoperative behavioral change of insulin treatment with decision support, we intended to perform this as an exploratory study. We tested the hypothesis that real-time visual notification of recent BG values would modify therapeutic thresholds for treatment of intraoperative hyperglycemia.

After obtaining approval from the University of Michigan Medical School Institution Review Board (Ann Arbor, Michigan), we conducted a retrospective analysis of our database between November 1, 2010, and February 26, 2014. The study involved a single site: the University of Michigan Health System. The inclusion criterion was BG at least 200 mg/dl (11.1 mM). Exclusion criteria were procedure less than 1 h and American Society of Anesthesiologists (ASA) physical status 5 and 6.

AlertWatch (AlertWatch Inc., USA) is a Food and Drug Administration–cleared secondary patient monitor for use in operating rooms.¹⁸  AlertWatch pulls historical patient data (risk factors and laboratory results) from multiple networked information systems and combines it with patients’ live physiologic data in a real-time audiovisual display (fig. 1). At our institution, our primary intraoperative automated anesthesia information management system (AIMS) (Centricity, USA) is linked to the AlertWatch system. The AlertWatch display is presented in three parts: patient demographic data on the left, organ system icon display in the middle, and alerts on the right. The organs are displayed in green, yellow, or red depending on the level of alert for each organ system. If an organ (or laboratory value) is outlined in orange, it designates a comorbidity of that organ. The visual alerts of interest in this study are the BG bar in the lower right and a red text alert in the upper right, notifying an out of range/very high BG value. The BG bar is highlighted in orange when the patient has diabetes mellitus, and the bar is filled with red if the BG is in a high range, along with a time stamp of the last BG measurement. Once insulin is documented in the AIMS, the alert disappears and a timer is started to remind providers to check a postinsulin glucose measurement. If a glucose measurement is not noted within 90 min of the documentation of insulin, an alert reminding the provider to recheck glucose appears.

We extracted data from the AIMS to evaluate the usage of AlertWatch in BG monitoring and initiation of appropriate treatment. AlertWatch usage was captured by intraoperative access of AlertWatch, which requires secure login by the clinician. The key study exposure variable was intraoperative access of AlertWatch. AlertWatch was installed in all operating rooms evaluated in this study in a staggered fashion. We assumed, based on initial clinical observations using the census and administrative views of the system, that AlertWatch was accessed during ~50% of all anesthetics during contemporaneous anesthetics between May 1, 2012, and October 31, 2013.

In addition, the data on the following variables were collected from the AIMS: patient age, gender, body mass index, preoperative diabetes diagnosis, home insulin therapy, fasting BG, highest intraoperative BG, and ASA class. Body mass index was categorized based on the World Health Organization’s classification.¹⁹  ASA class was handled as a dichotomous variable for presence or absence of class 3 or 4. Procedural variables included emergent procedure, surgical service, primary anesthetist (resident vs. nurse anesthetist with de-identified unique provider numbers), attending anesthesiologist (with deidentified unique provider numbers), case duration, and procedural complexity measured by the Current Procedural Terminology (CPT) base unit value. To relate this measure in clinical terms, arthroscopy carries a CPT base unit of 3 points, radical neck dissection carries 5 points, and aortic fenestration carries 10 points. In order to evaluate systematic bias in the usage of the system, we graphed a histogram by operating room use of the device to exclude cases from rooms that exclusively used or did not use AlertWatch. Further, we classified major groups of surgical services along the lines of existing groups of operating rooms and included them in the adjusted models. Surgical specialties were collapsed into six categories including general surgery, head–neck, hepatobiliary–transplant surgery, trauma–acute care, orthopedic–plastics, and gynecology–urology. This was performed to also adjust for service-line-specific biases in treatment of hyperglycemia. Anesthesia staff members (attending anesthesiologists) have core responsibilities along these surgical service groupings, but together with anesthetists (residents and Certified Registered Nurse Anesthetists) work randomly across all possible inclusion rooms during various times of the day or week.

The quality of the data was verified by confirming the accuracy against the independent query output of two different programmers. The data fields chosen for this study were either binary electronic chart entries or laboratory results. Specific continuous variables were converted to categories to increase clinical utility and improve the ability to match cases. Case duration and CPT base values were classified into quartiles, with the lowest quartile serving as the baseline in analyses. These variable transformations were performed to highlight differences in treatment behavior at either low or high case duration or complexity.

Preoperative fasting BG is collected from diabetic patients per protocol, and intraoperative BG is collected as part of an arterial blood gas in diabetic or nondiabetic patients and isolated BG analysis in diabetic patients. Arterial blood gas analysis is performed using the Gem Premier 3000 ABG analyzer (Instrumentation Laboratory, USA) located within the same patient care unit. Point-of-care glucometry is performed using Accu-Chek Inform II (Roche Diagnostics, USA). Standard maintenance and calibration of the devices are performed by quality control processes as specified by the manufacturers.

The treatment threshold for hyperglycemia was defined as any intraoperative point-of-care BG measurement or laboratory BG value at least 200 mg/dl (11.1 mM) as this is the device threshold for issuing an audiovisual alert. Since departmental protocols predating the study period^*†‡ specify treatment of BG more than 180 mg/dl (10 mM), we anticipated that all patients included in the study would be treated with insulin intraoperatively. Treatment was identified by searching the AIMS for use of insulin as a bolus or infusion. A previous study of computerized decision support in the intensive care unit reported a higher incidence of hypoglycemia in the tight glucose control group.¹⁷  Device safety was evaluated by comparing the incidence of severe hypoglycemia between the study groups, defined as BG value up to 70 mg/dl (3.9 mM).

The study outcome was defined as documentation of insulin use in response to preoperative (day of surgery) or intraoperative hyperglycemia. The secondary outcome of the study was the frequency of postinsulin BG checks in patients who were treated with insulin. The unit of analysis was set at the level of the operative case, with one or more instances of hyperglycemia or treatment permissible during the case, but counting as a single denominator and numerator, respectively.

Differences in study outcomes between the AlertWatch and no-AlertWatch groups were analyzed using Pearson chi-square tests, and P value of less than 0.05 was considered significant. The continuous variables’ data distributions were tested for normality and data presented as mean and SD for parametric or median with interquartile ranges for nonparametric data. We used a backward Wald logistic regression model to adjust for preoperative patient or procedural characteristics, with insulin treatment of intraoperative hyperglycemia as the dependent variable. Some of the major confounders of studies of behavioral change relate to inherent biases in clinical decision-making around treatment thresholds for hyperglycemia. These biases may be influenced by the following factors that were included as independent variables in the models: AlertWatch usage, preoperative diabetic diagnosis, age, body mass index, ASA class, preoperative insulin treatment, fasting BG value, CPT base unit categories, case duration, emergent procedure, male gender, resident anesthetist, surgical service, and highest intraoperative glucose value. Missing data were handled by first quantifying magnitude of missingness. For variables with more than 5% missing data, initial unadjusted 2 × 2 analyses were performed to evaluate the relationship between missingness and the primary study outcome. Consideration for data imputation was predicated on finding significant unadjusted differences in outcome in patients with more than 5% missing data.

The models were assessed for multicollinearity using condition index threshold of 30 and calibration using the Omnibus test for goodness-of-fit.^20,21  Model discrimination was estimated using the c-statistic.²²  The size of independent associations between study variables and study outcome was evaluated by adjusted odds ratios with 95% CIs.²³  Model validation was planned given the small sample size related potential for confounding.²⁴  In order to establish the feasibility of performing a propensity score–based matched analysis, we initially estimated the discrimination of a model predicting the use or nonuse of AlertWatch, as this was the study exposure of interest. If the resulting model discrimination was average or good (c-statistic > 0.75), we intended to perform a matched analysis. If the model discrimination was poor, we intended to perform internal validation with bootstrapping to validate model performance. Somers Dxy estimates of 0 suggest that the model’s predictions are random and Dxy of 1 indicates perfect discrimination. Overestimation of model discrimination was evaluated by examining the optimism estimates, and values less than 0.05 were indicative of excellent validation. Bias adjusted c-statistics were calculated using the formula (1 + index-corrected Dxy)/2.²⁵ 

All providers involved in the intraoperative care of study patients have a unique protected numeric identifier within the local information database. This is different from the hospital provider identity number or the pager number of these providers. Since intraoperative treatment behaviors possibly have significant variance at the individual provider level, additional analyses using generalized estimating equation modeling with exchangeable correlation matrix were performed to control for clustering either at primary anesthetist or attending anesthesiologist level. Tests of collinearity were performed on all model variables to assess for independence in addition to examination of the magnitudes of the standard errors. Model overfitting was limited by ensuring that more than 10 subjects per independent variable were included in the model. Estimates of adjusted odds ratios obtained by this method were compared with the primary logistic regression model estimates for the key study exposure variable, AlertWatch usage. Next, treatment variance at the primary anesthetist and attending anesthesiologist level was quantified in two separate hierarchical random effects mixed models using the primary model covariates described in the Statistical Analysis section above. Resulting variances were back transformed into the median odds ratio, which is reported as a value from 1 (no variance) to above 1 (increasing variance).^26,27 

Of 31,595 instances of unique patient care during the study period, the AlertWatch system was noted to have been accessed during intraoperative anesthetic care of 16,271 patients (51%). Overall, 2,341 patients (7.4%) had BG at least 200 mg/dl (11.1 mM), of whom 791 (33.5%) had AlertWatch and 1,550 were controls (fig. 2).

The frequency of insulin treatment of BG at least 200 mg/dl (11.1 mM) was 496 of 791 (62.7%) with and 817 of 1,550 (52.7%) without AlertWatch, respectively. The frequency of BG recheck after insulin treatment was 407 of 791 (51.5%) with AlertWatch and 655 of 1,550 (42.3%) in controls. Device safety as defined by the frequency of hypoglycemia was 3.3% (95% CI, 2.2 to 4.8%) with AlertWatch and 3.1% (95% CI, 3.1 to 5.0%) in controls. Table 1 describes the demographic and management variables with and without use of the AlertWatch system. There were no differences between groups for distribution of age, sex, body mass index, preoperative diabetes diagnosis, home insulin therapy, ASA class 3 or 4, and preoperative fasting BG. Emergent surgery was less frequent, and intraoperative BG values were lower in the AlertWatch group. Figure 3 displays the frequency of use or nonuse of AlertWatch across all operating rooms included in the study. Since all operating rooms had varying use of the device, no location-specific case exclusions were deemed necessary. Missing values of more than 5% were present for only one of the independent variables, preoperative fasting BG with 15% missing data, possibly reflecting patients without a preoperative diagnosis of diabetes mellitus. No differences in outcomes were noted in patients with missing values (P = 0.37 for insulin treatment and P = 0.613 for recheck of BG after insulin).

After adjusted analyses, use of AlertWatch was associated with a 55% increase in odds of insulin treatment (adjusted odds ratio [95% CI], 1.55 [1.23 to 1.95]; P <0.001) (table 2). The other significant independent associations of insulin treatment were case duration, procedural complexity, fasting BG value, and home insulin therapy. Preoperative diagnosis of diabetes, ASA physical status, emergent procedure, patient age, surgical service, and primary anesthetist were not associated with variance in insulin treatment. The median CPT base value was 7 (with the lower and higher quartiles at 6 and 10). The c-statistic (95% CI) for the primary outcome model was 0.74 (0.72, 0.76) with good calibration (Hosmer–Lemeshow test P = 0.571) and no evidence of multicollinearity. We were unable to model the use or nonuse of AlertWatch using the variables available with adequate discrimination (c-statistic, 0.6), and therefore, propensity score matched analysis was not performed. Bootstrapping was performed using 1,000 replacements with Somers Dxy optimism estimate of 0.038, suggesting excellent model validation. The bootstrapped bias adjusted c-statistic for the primary outcome model using the index corrected Dxy was 0.73. We addressed the provider level variance by adjusting for provider level clustering in a generalized estimating equation model. The relationship between AlertWatch and study outcome was unaffected by individual provider-level case clustering (adjusted odds ratio, 1.47 [1.21, 1.8]). On random effects mixed modeling, the median odds ratio for anesthetist was 1.04 and attending anesthesiologist was 1.10, reflecting a 4% variance in insulin treatment of hyperglycemia at the anesthetist level and a 10% variance at the level of the attending anesthesiologist in an adjusted model that included the use or nonuse of AlertWatch. Anesthesia providers were also more likely to recheck intraoperative BG values after insulin treatment with AlertWatch (table 3; adjusted odds ratio [95% CI], 1.44 [1.14 to 1.81]; P = 0.002).

In this large retrospective study of intraoperative BG management, we found a clinically relevant change in individual provider response to significant hyperglycemia with use of the AlertWatch system.

Despite controversies in perioperative BG management,¹  there is a large body of evidence suggesting that high BG is associated with poor outcomes after cardiac and noncardiac surgery.^8,10,28  Recent studies showed improved outcomes in treating BG above 150 mg/dl (8.3 mM) during or after surgery.²⁹  The literature on perioperative BG management is evolving, likely reflecting the overall variance in insulin treatment rates seen here.

One of the challenges in patient monitoring is the task of assimilating the large amount of real-time data and integrating this information, a process that largely relies on the operator’s alertness. Continuous data tend to quickly overwhelm human alertness and effectiveness in recognizing critical events. Some monitoring data are considerably slower to change over time and are likely to be missed in complex environments. An effective strategy to prioritize clinical decisions in real time is to create clinically relevant thresholds for generation of audiovisual alerts.^18,30  This “human-in-the-loop” type of process monitoring has been compared to an iceberg, where only a small proportion is visible to the operator. The response of the human body to anesthesia is complex and appropriate decision support with “human-in-the loop” monitoring relies on effectively refining those variables available to make the right clinical decision.^31,32  This is very critical in an operating room environment, as the distractors and time available for effective clinical decision-making and treatment are often limited.³³  Automated real-time alert systems have been shown to be effective when used for therapeutic decision-making, monitoring adequacy, billing, and documentation compliance in addition to other uses.^30,34,35  For instance, automated alert systems have been successfully used in following protective lung ventilation strategies and have been shown to improve behavioral changes in clinical decision-making.³⁴ 

In this study, we report that using an integrated real-time audiovisual alert system reduces variability in management of hyperglycemia. The AlertWatch system influenced glycemic management behavior to a greater extent than individual provider-level treatment variance in this study population. We also quantified the provider-level insulin treatment variance at 4% at the anesthetist level and 10% at the level of the attending anesthesiologist in an adjusted model that included the use or nonuse of AlertWatch. This suggests that despite use of AlertWatch, there is still a clinically important difference in the behavioral responses of anesthesiologists to a hyperglycemic episode. This may reflect significant variance in case complexity and duration, with more complex lengthy procedures carrying greater likelihood of occurrence and treatment of hyperglycemia, as seen in this study. Alternatively, this finding offers opportunities for exploring other novel methods to reduce the variability of intraoperative glycemic therapy.

Despite being a relatively large retrospective study, it is important to highlight some of the limitations of this study. This was undertaken at a single center, and the positive behavioral response may not be generalizable to other settings. We cannot attest to functionality of the AlertWatch system in all instances when the Web site was accessed and cannot confirm that the Web site was actually utilized throughout care. AlertWatch use may be different in different levels of training, as these data are not available. The AlertWatch development site was University of Michigan, so study estimates may not be generalizable to other centers implementing this technology. In addition to these limitations, clinical databases research has inherent challenges related to the use of retrospective data, as the quality of the data cannot be verified directly against independent patient report. We attempted to address these issues by two independent queries of the database and by verifying differences using a manual check process. The statistical methods are limited by the absence of a provider-level change in behavior with and without exposure to the AlertWatch system, owing to sample size limitations. In the absence of this direct comparison of individual provider behavior, the use of median odds ratios quantifies the variance at the level of the providers, independent of the use or nonuse of AlertWatch. The models had modest discrimination and therefore reflect the presence of unmeasured factors that influence treatment behavior. The study was not randomized in design, a weakness that is counterbalanced by the fact that we were unable to model usage of AlertWatch. This suggests either that there are unmeasured factors dictating the choice of use or that the choice is more random than initially assumed. Finally, this is not a patient outcome study as the sample size numbers are limited for a meaningful outcome analysis. However, the sample size is of sufficient magnitude to test behavioral change, with reliable effect over concurrent and historical controls during a 2-yr study period. Despite limitations, this study demonstrates the clinical value of a novel audiovisual alert system that helps providers correct a missed treatment opportunity in real time.

In conclusion, we have demonstrated that AlertWatch is associated with a significant increase in intraoperative treatment and rechecking of hyperglycemia. This treatment behavior modification is independent of preoperative patient factors, provider-level factors, and case complexity. These findings support the hypothesis that lack of real-time awareness of abnormal BG values largely contributes to their undertreatment.

Support was provided solely from institutional and/or departmental sources.

The University of Michigan (Ann Arbor, Michigan) developed the AlertWatch and has equity interest in it. Dr. Ramachandran has received honoraria for scientific consulting with Galleon Pharmaceuticals, Horsham, Pennsylvania, and Merck Sharp and Dohme, Whitehouse Station, New Jersey, and has active research funding from Merck Sharp and Dohme. The other authors declare no competing interests.