Development and Evaluation of a Graphical Anesthesia Drug Display

An iterative design process was used to develop a drug display (fig. 1) for use by anesthesiologists in the operating room. The multidisciplinary team (bioengineering, architecture, anesthesiology, computer engineering, and cognitive psychology) defined the requirements of the display from an assessment of the clinician's working task (table 1) and performed several iterations of interactive design and usability evaluations ∥∥to create the prototype display (an animation of the display may be viewed at http://abl.med.utah.edu/∼noahs/dd1/dd1.html). Intravenous sedatives are shown on the top of the display, analgesics in the middle, and neuromuscular blocking agents on the bottom. On the left, narrow color-coded histogram bars (e.g. , blue for remifentanil) show the size of the drug bolus doses. The predicted effect-site concentrations display the drugs’ kinetics for a 40-min period: from 30 min in the past to 10 min in the future. The histogram bars and the concentration curves move from right to left with time. The objects on the far right, a pie chart and two bar graphs, show predicted levels of sedation, analgesia, and neuromuscular blockade, respectively. Each effect (shown on the right) is calculated from the drug's effect-site concentration (shown on the left). For instance, when the effect-site concentration of a drug reaches 1.0 × EC⁹⁵, the predicted effect will be 95%. As the effect-site concentration increases above EC⁹⁵, the drug effect approaches 100%, and the object on the right becomes completely filled.

When more than one drug in a class is administered, the relative contribution of each drug is shown, as seen in figure 2. For simplicity, we calculate the combined effect using the sum of the predicted effect-site concentrations for each drug (i.e. , the drug display does not incorporate drug–drug synergism or antagonism). If the total effect-site concentrations of all drugs are greater than 100%, then the contributions from each drug become a fraction of the total. Thus, given that n sedative drugs are administered, the relative effect on sedation for drug i is:

To use the display interactively, a drug delivery program was implemented. A graphic dialog window allowed the investigators to select a patient's weight, height, age, and gender and enter bolus doses of intravenous drugs. The drug delivery program notified the drug display of the patient information, drug type, and bolus information remotely over a network.

After obtaining approval from the institutional review board at the University of Utah Health Sciences Center, seven attending and eight resident (three Clinical Anesthesia year 2, five Clinical Anesthesia year 3), anesthesiologists were selected to participate in the study. The average age ± SD of the 15 participants was 36 ± 4.1 yr. The average postresidency experience of the attending anesthesiologists was 6.0 ± 3.8 yr.

Subjects were instructed to play the role of an attending anesthesiologist and command the operator of an anesthesia simulator (playing the role of the resident) to administer anesthesia, intubate, and care for a simulated patient. The anesthesiologist's control of drug delivery was evaluated in two simulated surgical conditions. In one of the conditions, participants used only traditional monitors provided by the simulator (electrocardiogram, pulse oximetry, noninvasive blood pressure, train-of-four, qualitative vital signs) to obtain feedback about the patient's status. In the other condition, subjects used the graphic drug display in addition to the traditional monitors. The drug display was assessed by comparing effect-site drug concentrations, physiologic vital signs, and questionnaire results for both display conditions.

The Anesoft (Issaquah, WA) personal computer–based anesthesia simulator was used to simulate a patient in the operating room. The investigator used the graphic interface to perform clinical procedures, administer drugs, and interact with the surgeon when requested to do so by the participants. The simulator interface displayed the monitored variables typically observed in the operating room. In addition, qualitative information (e.g. , mental status, eyelid reflex, pupil diameter and reactivity, skin color and temperature, and patient movement) was available under a pull-down menu to assess the patient.

When subjects arrived for the experiment, they completed a questionnaire to elicit their experience level, length of time that they worked before the study, caffeine consumption, sleep history, color vision, and whether they required vision correction. After this, subjects were trained to understand and use the drug display and the software simulator. After the pertinent portions of the simulator were explained, the subject was instructed to direct an operator to administer anesthesia and provide care for an exemplar simulator patient. The investigator answered any questions concerning the use of the monitors and the drug display. For all subjects, training time lasted less than 10 min.

Subjects were then tested sequentially in both experimental conditions, with order of patients and condition being balanced. Each subject was asked to administer intravenous anesthesia (bolus doses of propofol, remifentanil, and rocuronium), intubate, and provide care for two different simulated patients. In all cases, the subject was informed that the patient should be anesthetized and paralyzed for the procedure. When the subject made a request to administer a drug, the simulator operator gave the intravenous drug to the simulated patient. A second investigator simultaneously entered the same dose into the drug delivery program, and the drug information was presented on the drug display.

After each simulation, subjects completed the NASA-TLX workload questionnaire 11( Appendix 1). At the end of the session, they answered a short questionnaire about the usability of the drug display ( Appendix 2). The study session lasted approximately 1 h. Subjects were compensated $50 for their participation.

Subjects were trained to use the Anesoft simulator and the drug display. The simulated patient used in training was different than the patients in the experiment. The investigator showed the pertinent capabilities of the simulator: electrocardiogram, side-stream carbon dioxide monitor, gas analyzer, pulse oximeter, noninvasive blood pressure monitor (updated every 3 min), ventilator settings, gas flows, physical patient characteristics, surgeon dialog window, how drugs are administered, and the patient window. The subject was allowed to ask questions about the simulator and its functionality. Simulator training was completed when the subject understood the simulator and felt comfortable with directing the operator to administer anesthesia and care for the patient. Next, subjects were shown static screen shots of the drug display monitor depicting the effect-site concentrations and current effects of propofol, remifentanil, and rocuronium. The display was explained in detail.

The subjects were randomly allocated to one of four groups. Each group differed with respect to the order of patient presentation and whether the drug display monitor was used in the test (table 2). When the subject was allowed to use the drug display monitor in addition to the simulator's standard physiologic display, the drug display program was shown on the same computer screen directly above the Anesoft simulator display.

Two similar young and healthy patients were selected from the Anesoft simulator's patient library. Patient A (18-yr-old woman; height, 66 cm; weight, 52 kg) required anesthesia for a mass removal, and patient B (22-yr-old man; height, 180 cm; weight, 74 kg) was anesthetized for drainage of an abscess. The expected duration for each surgery was modified to be 20 min. The maintenance phase of anesthesia, defined as the time from the start of surgery to the time when the surgeon began to close the incision, lasted 15 min and 24 s.

The subjects were given a blank anesthetic record and were asked to complete it as they administered anesthesia during each simulated surgery. Before each case, the subjects were given a record of the patient's medical history, laboratory values, baseline vital signs, and surgery type and expected duration. The patient was presented as having arrived in the operating room without previous sedation, an intravenous line, or preoxygenation; however, electrocardiogram electrodes and the noninvasive blood pressure cuff were already in place. The subject was reminded that they could administer only bolus doses of propofol, remifentanil, and rocuronium. The subjects were instructed that the simulation would end once the surgeon had finished closing the patient's incision. However, the subject was asked to administer anesthesia as if they would be responsible for awakening and extubating the patient.

Three intravenous drugs were given during this study: propofol for sedation, remifentanil for analgesia, and rocuronium for neuromuscular blockade. Effect-site concentrations were calculated using the algorithms proposed by Shafer and Gregg. 8The pharmacokinetic constants for each drug are listed in table 3. The remifentanil models used by the drug display and the simulator had the same pharmacokinetic parameters. For propofol, the drug display's pharmacokinetic constants could not be directly compared with those used by the Anesoft software. However, the models were determined to be similar by administering the same dose of drug to the drug display and the Anesoft simulator and comparing the resulting pharmacokinetic plots.

For the purpose of the evaluation, the drug display models’ pharmacodynamic parameters were adjusted until they matched the Anesoft patient's response. Specifically, the drug display's EC⁹⁵parameter for remifentanil was adjusted to the drug dose required for the simulated patient's pupils to become a pinpoint. The EC⁹⁵parameter for propofol was modified so that the patient became apneic when the predicted effect-site concentration reached the EC⁹⁵level. ##

The plots of effect-site concentrations shown in figure 1were scaled such that when a single ED⁹⁵dose of propofol or remifentanil was given, the plot of predicted effect-site concentration would peak at EC⁹⁵. For example, 1.0 mg/kg propofol was found to be the dose that caused the Anesoft simulated patient to become apneic. The scale on the display was set such that this bolus dose of propofol caused the projected effect-site concentration of propofol to increase to the EC⁹⁵level. The responses to the drugs for both simulated patients were similar because the pharmacokinetic models adjust the effect-site concentrations according to patient weight, gender, and age. When a single ED⁹⁵dose of rocuronium 12(0.6 μg/kg) was given to the Anesoft patient, the train-of-four response disappeared for more than 30 min. Thus, in the study, the train-of-four response was obliterated for the duration of the test period by the intubating dose of rocuronium. Although participants were not informed that rocuronium was not used as a measure of performance, no one administered a second dose in the evaluation. If the volunteer asked why there was a mismatch between the effect-site concentration shown on the drug display and the expected patient response, we explained that this was caused by variability in patient sensitivity to rocuronium.

During the simulated surgeries, effect-site concentrations of all administered drugs were recorded at 2-s intervals. The values of the vital signs were recorded at 4-s intervals. Heart rate and blood pressure values were extracted and entered on a spreadsheet.

Data are presented as mean ± SD. A criterion value of P < 0.05 was used for all analyses. The precision of drug administration was measured as the SD and the root-mean-square error between the drug's effect-site concentration (normalized to the drug's EC⁹⁵) and the drug's EC⁹⁵during the maintenance phase of anesthesia. ***A two-by-two repeated-measurement analysis of variance was used to analyze differences in precision of administration of remifentanil and propofol [(with or without the display) × (patient A or patient B) × display order]. A t  test was used to examine differences in the number of propofol and remifentanil bolus doses administered during maintenance.

Heart rate and mean arterial blood pressure during and at the end of the maintenance phase were used to determine the patients’ response to pain. For this analysis, the interval in which heart rate or mean arterial blood pressure deviated by 10% or more above the baseline value was computed. The baseline values for the vital signs were determined by averaging vital sign data of the first 36 s of simulation data before intubation and drug administration. Vital sign differences between the two display conditions during and at the end of maintenance were analyzed using a t  test.

Finally, the mean and SDs of the scores were computed for the participants’ answers to the NASA-TLX workload survey and the first question of the evaluation questionnaire regarding display usefulness ( Appendix 2). A t  test was used to determine differences between the experimental conditions.

For both remifentanil and propofol, effect-site concentrations were maintained closer to the EC⁹⁵in presence of the drug display. Analysis of variance revealed a significant main effect of display for the root-mean-square error of propofol (P < 0.01) and remifentanil (P < 0.01) effect-site concentrations. The means and SDs for the effect-site concentrations and the root-mean-square error for both drugs are presented in table 4.

Use of the drug display changed clinical practice. Remifentanil and propofol were controlled differently during the maintenance phase of anesthesia when subjects were provided with the drug display (figs. 3A and B). When the drug display was available, significantly lower variance was observed in the effect-site concentrations of propofol and remifentanil. On average, the remifentanil effect-site concentrations were maintained at a higher level, while propofol concentrations were maintained at a lower level.

During maintenance, remifentanil doses were administered more frequently with the drug display (without display: 3.1 ± 1.3 doses; with display: 4.7 ± 1.8 doses;P < 0.01). There was no difference in the number of propofol administrations (without display: 3.1 ± 2.1 doses; with display: 2.7 ± 1.3 doses).

The differences in heart rate and blood pressure were not statistically significant during maintenance or at surgical closure. Without the drug display, heart rate was elevated (≥ 10% above baseline) for 72.3 ± 62.3 s, and with the display, it was elevated for 80.0 ± 85.3 s. Blood pressure was elevated for 5.6 ± 14.8 s without the display and for 43.5 ± 46.4 s with the drug display. At the end of maintenance, eight patients had an increased heart rate without the drug display, and four patients experienced tachycardia with the drug display. One patient had an elevated blood pressure during both conditions.

The results of the NASA-TLX survey for 11 of the 15 participants are shown in figure 4. The remaining four participants’ answers were not included in the analysis because of technical problems while collecting the data. The drug display was associated with reduced mental demand, frustration level, and effort (all P < 0.05), and the participants concluded that the drug display enhanced performance during the administration of anesthesia (P < 0.05).

In the utility and satisfaction questionnaire, the average participant's score for their perception of the graphic drug display's utility was 8.7 ± 1.1 of 10 (95% confidence interval, ± 0.6; range, 7–10). All participants felt that the drug display should be added to current operating-room monitoring capabilities. Thirteen of 15 participants felt the simulator provided an adequate representation for anesthesia during surgery, given the constraints of the study.

The results suggest that visualizing real-time pharmacokinetics and pharmacodynamics of drugs may change the administration of intravenous drugs, resulting in less variation in drug effect-site concentrations and a reduction in cognitive workload. Participants more accurately controlled the effect-site concentration of remifentanil and propofol when using the drug display. The number of remifentanil doses increased while the variability in the effect-site concentrations decreased. User satisfaction and perceived usability were overwhelmingly positive.

This study supports the idea that observing the effect-site concentrations of intravenous drugs helps clinicians to maintain drug concentrations within a “therapeutic window.” This is consistent with Stanski's assertion that observing the kinetics of the drug biophase, particularly the effect of accumulation, helps to optimize drug administration. 13Better titration of intravenous anesthetics could decrease the occurrence of intraoperative awareness and improve hemodynamic control.

Tools that aid in understanding the relation between the dose of a drug and the resulting drug's plasma and biophase concentrations are especially important when optimizing to a desired therapeutic effect. 13Several participants remarked during the formal evaluation that they would not usually administer remifentanil as a bolus dose but would prefer to use an infusion pump. Despite being unfamiliar with the biophase of bolus remifentanil delivery, the drug display allowed participants to observe the kinetics of remifentanil and deliver it more effectively. Thus, the display may have educational value.

Visualizing pharmacokinetics and pharmacodynamics may facilitate the use of traditional and target-controlled infusion pumps. Showing the predicted effect-site concentrations while using infusion pumps may help clinicians to select an optimal infusion rate and duration. The use of target-controlled infusion pumps in conjunction with a display of predicted effect-site concentrations would support the anesthesiologist's decision to target a certain drug concentration. Once an infusion pump has been stopped, predicted pharmacokinetics would help to understand the time required for recovery.

Currently, anesthesiologists rely on the anesthetic record to guide them. Unfortunately, it is frequently inaccurate or incomplete. 14Automated records are designed to be more accurate and readable, but past implementations of electronic anesthesia records have not fulfilled their promise. In fact, the clumsy user interface of electronic record keepers may make their accuracy as a record of intravenous drug administration even more suspect. 15,16Accurately tracking drug history with the drug display could better support the drug delivery task.

The population-based multicompartment pharmacokinetic models are imperfect, and there will be instances in which the model will not match the real clinical situation. Because the drug models are based on population data, they do not always match individual patient drug kinetics. When the mismatch between the model predictions and the actual patient state is significant, the drug display could mislead the anesthesiologist. The clinician must be keenly aware that the display is model driven. For example, model-driven automated infusion pumps that target drug plasma concentrations have been shown to have a median absolute performance error of 20–30%. 17–19If the anesthesiologist follows the predictions without vigilantly observing important physiologic signs of drug effect, the result could be inadequate anesthesia for a patient who is insensitive or an anesthetic overdose in a patient with high sensitivity.

Showing the drug effects for sedation, analgesia, and neuromuscular blockade according to a drug's single EC⁹⁵value is oversimplified. The current display shows effect-site concentrations and drug effects without considering different analgesia and anesthesia objectives. For example, alfentanil requirements necessary to prevent a hemodynamic response to surgical incision are significantly higher than that needed to prevent a response during wound closure. 20In different clinical situations, the anesthesiologist may choose a drug for its different effects and will need to consider several EC⁹⁵values for the same drug. The present prototype display did not address this complexity.

The prototype display did not incorporate drug–drug synergism. For instance, both pharmacokinetic and pharmacodynamic interactions have been observed between opioids and propofol. Propofol plasma concentration is increased in the presence of alfentanil, and increased sufentanil and alfentanil plasma concentrations have been observed as a result of propofol's tendency to inhibit opioid metabolism. 21Furthermore, there are pharmacodynamic interactions between opioids and other classes of sedatives, either synergism or antagonism, depending on depth of anesthesia, for opioids and barbiturates. 22,23When intravenous drugs and inhalation agents are used together, the combined effects on sedation, analgesia, and neuromuscular blockade become complex. In future work, it will be important to judiciously select and use drug–drug interaction models that allow clinicians to visualize these complex relations.

In addition to the limitations of the display, the study design had a number of limitations. The evaluation was conducted using a personal computer–based simulator rather than a full-body simulator or a clinical study in the operating room. The simulator provides only a fraction of the clinical tasks normally performed and thus may have reduced realism, affecting clinicians’ vigilance and workload. Monitored data were shown in a single window in an unfamiliar format. The personal computer simulator showed a limited set of patient responses, and the participant could only check a patient's physical response by asking the investigator to reveal the information from a pull-down menu. As a result, the participants seemed to largely ignore the patient's physical responses.

To simplify this evaluation, the participants were limited to giving bolus doses of three intravenous agents. Participants felt that administering remifentanil via  bolus administrations was an uncommon technique. The results may differ if participants used more conventional anesthetic regimens.

To encourage participation with minimal disruption of the participants’ clinical responsibilities, the study was designed to last approximately 1 h. This time limitation necessitated an unusually brief surgical duration, and the study was terminated before emergence from the simulated anesthetic. Consequently, analysis of patient recovery was limited. The advantages of the drug display may have been more apparent in longer surgeries.

We did not specifically define the criteria to be used for a successful anesthetic and assumed that participants would judge the adequacy of anesthesia by monitoring heart rate and blood pressure, pupil response, and movement. The results showed that the simulated patients had no difference in vital signs with or without the drug display, which was inconsistent with the observed differences in drug effect-site concentrations. Because the simulated patients were young and healthy, participants may have felt it unnecessary to tightly control blood pressure and heart rate in either display condition. Thus, it is inconclusive whether the drug display would aid in tighter hemodynamic control.

This study was designed to evaluate the display during conditions in which the pharmacokinetic and pharmacodynamic models determine patient response. The results provide no information on display performance when a patient's response varies from the model's predicted response. The drug display will need to be evaluated in a more realistic clinical setting.

The study findings suggest a number of enhancements to the drug display. The y-axis of the effect-site concentration plots could show the reference lines for intubation and wake up. Although this would add complexity to the display, it may better support the use of EC⁹⁵values in longer, more complex anesthetics. Showing only bolus doses of intravenous drugs is unrealistic for clinical practice, and future displays will incorporate information provided by intravenous drug infusion pumps and anesthetic gas analyzers. The drug display would be more practical with additional pharmacokinetic and pharmacodynamic models including inhaled volatile agents. In addition, the redesigned display should show drug synergism and the combined effects of multiple drugs on levels of sedation, analgesia, and neuromuscular blockade. 24–32Finally, future designs of the display would emphasize human factors design principles.

In conclusion, an interdisciplinary team designed, implemented, and evaluated an anesthesia drug display to support drug-delivery decision-making. The team used an interactive process to design a graphic display of drug pharmacokinetics and pharmacodynamics. An evaluation of the drug display prototype suggested tighter control of propofol and remifentanil effect-site concentrations, a perceived reduction in mental workload, and a positive endorsement of the concept. The evaluation indicates the potential value of displaying model-based, real-time pharmacokinetic and pharmacodynamic predictions during administration of anesthesia. Additional design enhancements and a more comprehensive evaluation are needed before clinical implementation.

The authors thank Steven Farnsworth, M.D., Talmage Egan, M.D. (Associate Professors of Anesthesiology), and Steven Kern, Ph.D. (Assistant Professor of Anesthesiology and Assistant Professor of Pharmaceutics), (all from the Department of Anesthesiology, The University of Utah, Salt Lake City, Utah) for their clinical assistance and advice.

FIGURE 

FIGURE 

• 1. How useful was the additional information provided by the drug display?

  Not useful————————————————————–Very useful

  1 2 3 4 5 6 7 8 9 10

• 2. Should a drug display like this prototype be added to the equipment you already use in the operating room?

  Yes No

  3. In spite of the controls implemented by the study (choice of drugs, etc. ), did the simulator provide an adequate representation of a typical anesthetic during a surgery?

  Yes No

• 4. What was unclear about the drug display?

• 5. How would you improve on the drug display or the study?

• 6. Would you be willing to participate in a future study?

  Yes No