This study was conducted to compare the performance accuracy of the independent variables Bispectral Index (BIS), A-Line ARX index (AAI), and predicted propofol effect-site concentration (CePROP) to measure the dependent variables of loss of responses to different stimulation defined as loss of response to verbal command (LORverbal), eyelash reflex (LORlash), and noxious stimulus (LORnoxious) during stepwise increased levels of propofol infusion with and without remifentanil.
Forty-five patients were randomly allocated to one of three groups (0, 2, and 4 ng/ml remifentanil) to receive graded CePROP and predicted effect compartment controlled remifentanil (CeREMI). At every step, the ability to respond to verbal command using the Observer's Assessment of Alertness/Sedation Scale (OAA/S), eyelash reflex, and electrical tetanic noxious stimulus were compared against BIS, AAI, and CePROP. Prediction probability and sensitivity/specificity were calculated.
Increasing CeREMI increased BIS and AAI values at LORverbal and LORlash and decreased CePROP. Similar findings were found for LORnoxious. The overall prediction probability to measure the hypnotic component of anesthesia remained accurate in the three groups for BIS, AAI, and CePROP. Combined information from CePROP, CeREMI, and BIS or AAI increased the overall prediction probability for predicting the OAA/S scale and LORlash. Less accuracy to LORnoxious was found in all independent variables.
Although BIS, AAI, and CePROP were influenced by remifentanil during propofol administration, their ability to detect OAA/S and LORlash remained accurate. Improved performance is obtained when BIS and AAI are measured in conjunction with drug targeted effect-site concentrations. Remifentanil decreases the ability of these independent variables to detect LORnoxious.
BOTH electroencephalography- and midlatency auditory evoked potential (MLAEP)-derived variables have been proposed as measures of the hypnotic state during anesthesia. 1For the electroencephalography, the Bispectral Index (BIS) incorporated in the A-2000 BIS® monitor (Aspect Medical Systems, Inc., Newton, MA) has been proven to have a high sensitivity and specificity to measure anesthetic drug effect, compared with other processed electroencephalographic variables. 2–4Previously, MLAEP has been used by various investigators to study anesthetic depth. 5Recently, Jensen et al. 6,7developed a new method for extracting the MLAEP from the electroencephalographic signal by using an autoregressive model with an exogenous input (ARX) adaptive model. This method allows extraction of the MLAEP signal within 15–25 sweeps of 110 milliseconds’ duration each, resulting in only a 6- to 15-s response delay time. A new monitoring variable, called the A-Line ARX index (AAI), is then calculated from this fast extracted MLAEP wave. This new technology is incorporated in a recently commercialized system called A-Line® (Danmeter A/S, Odense, Denmark). Recently, Struys et al. 8compared the accuracy of both BIS and AAI for measuring loss of responses to different stimulation defined as loss of verbal command (LORverbal), loss of eyelash reflex (LORlash), and loss of response to noxious stimulus (LORnoxious) during steady state propofol administration. We found that BIS, AAI, and predicted effect-site concentration of propofol (CePROP) revealed a similar level of infor-mation on LORverbaland LORlashbut did not predict LORnoxious. Also, AAI has recently been shown to reliably assess the level of consciousness during propofol, 8,9sevoflurane, 10and midazolam 11anesthesia. The BIS correlated better to propofol plasma concentrations than AAI, but AAI correlated better than BIS to the clinical signs during recovery from propofol anesthesia. 11
In previous work done with the AAI, propofol was administered solely. Although a clear pharmacodynamic interaction between remifentanil and propofol has been described, 12controversy still exists on the influence of opiates on the accuracy of both BIS and MLAEP to measure loss of response to different stimulation. 13–16During remifentanil monoinfusion, AAI measured changes in patients’ level of arousal better than BIS. 17However, in this study, a different electrode position was used, where the AEP was recorded between a frontal (+) and an occipital electrode (−). It is possible that a change in montage such as this could lead to significantly different results compared with those from the current study.
A fundamental question is whether the electroencephalograph- or AEP-derived variables can be used to optimize drug delivery. A first step to answer this question is to examine the performance and accuracy of these monitors under various conditions and to compare them with the accuracy and usefulness of on-line calculated drug effect-site concentration. A secondary question is how opiates affect these performance parameters. This study was conducted to assess the performance accuracy of the AAI to reflect the hypnotic component of anesthesia and to measure loss of responses to different stimulation defined as LORverbal, LORlash, and LORnoxiousduring stepwise increased levels of propofol infusion with and without remifentanil. The performance of AAI was compared with that of BIS and CePROP.
Materials and Methods
After Institutional Ethics Committee (Ghent University Hospital, Gent, Belgium) approval, informed consent was obtained from 45 female patients with American Society of Anesthesiologists class I, aged 18–60 yr, who were scheduled to undergo ambulatory gynecologic surgery. Exclusion criteria included weight less than 70% or more than 130% of ideal body weight, neurologic disorder, and recent use of psychoactive medication, including alcohol. They were randomly allocated to one of the three groups. In all groups, patients received a “staircase” computer-controlled infusion of propofol targeting the effect compartment. Initially, an effect-site concentration of 1.5 μg/ml was targeted in the group without remifentanil, and 1 μg/ml was targeted in the two other groups; this was increased every 4 min by 0.5 μg/ml until loss of response to all relevant clinical measures of anesthetic depth was observed. In the 0 ng/ml remifentanil group, no remifentanil was given. In the 2 ng/ml and 4 ng/ml remifentanil groups, an effect compartment controlled infusion of remifentanil was started 4 min before the start of propofol. Calculated remifentanil effect-site concentrations (CeREMI) of 2 and 4 ng/ml were targeted, respectively.
Propofol and remifentanil were administered via a computer-assisted continuous infusion device to a target effect-site concentration (RUGLOOP **) using a three-compartment model enlarged with an effect-site compartment. For propofol, the pharmacokinetic-dynamic model previously published by Schnider et al. 18,19was used. For remifentanil, the pharmacokinetic-dynamic model previously published by Minto et al. 20,21was used. CePROP was computed to yield a time-to-peak effect 22of 1.6 min after bolus injection, as also published by Schnider et al. 18,19and clinically confirmed by Struys et al. 23For remifentanil, a t1/2ke0of 1.020619 min was applied as published by Minto et al. 20,21Propofol and remifentanil infusion were administered using a Fresenius Modular DPS Infusion Pump connected to a Fresenius Base A (Fresenius Vial Infusion Systems, Brézins, France). RUGLOOP steers the pump at infusion rates between 0 and 1,200 ml/h via an RS-232 interface. By using this infusion technique, we were able to obtain a steady state condition for both propofol and remifentanil at every target level after 4 min of infusion. Hereby, steady state is defined as the equilibration between the calculated plasma and effect-site concentration of the drug. Remifentanil and propofol were infused via a large left forearm vein. Every patient received approximately 200 ml crystalloid fluid during the study period. No fluid load was given before induction. No patient received preanesthetic medication. No other drugs were given. All patients maintained spontaneous ventilation via a facemask delivering 6 l/min O2.
Heart rate and noninvasive blood pressure, oxygen saturation measured by pulse oximetry (Spo2), and capnography were recorded at 1-min intervals using an AS3 monitor (Datex, Helsinki, Finland). BIS (version 3.4) was derived from the frontal electroencephalogram (At-Fpzt) and calculated by the A-2000 BIS® monitor using a BIS-Sensor® (Aspect Medical Systems, Inc.). The smoothening time of the BIS® monitor was set at 15 s. The AAI from the MLAEP was calculated using the A-Line® monitor. The MLAEPs were elicited with a bilateral click stimulus of 70 dB in intensity and 2 ms in duration. Three electrodes (A-Line AEP electrodes; Danmeter A/S) were positioned at the mid forehead (+), left forehead (reference), and left mastoid (−). The extraction of the MLAEP using a short moving time average technique together with an ARX model and the calculations of the AAI are described elsewhere. 8
Ten seconds before each increase in CePROP (after 4 min of infusion at the specific target effect-site concentration), the independent variables of response (BIS, AAI, and CePROP) were recorded. Immediately after that, patient responsiveness to different stimuli was tested using the dependent variables eyelash reflex, Observer's Assessment of Alertness/Sedation Scale (OAA/S 24;table 1), and response to electrical tetanus (at 100 Hz and 50 mA for 2 s; applied to the volar forearm level) assessments, in that order. Verbal stimulation proceeded noxious stimulus. Patients were considered responsive to verbal stimulus if their OAA/S value was 3, 4, or 5 and were considered unresponsive to verbal command at OAA/S values of 0, 1, and 2. Transition between a responsive and an unresponsive state was defined as loss of response to verbal command (LORverbal). Loss of response of eyelash reflex was defined as LORlash. Patients were considered responsive to noxious stimulus if they responded to the tetanic electrical stimulation, regardless of whether they responded to trapezius squeeze during the OAA/S assessment. Loss of response to tetanic stimulus was defined as LORnoxious.
Both BIS and AAI indices were also logged automatically. RUGLOOP digitally recorded the BIS index every 10 s, and the A-Line monitor recorded AAI index values nominally every 6 s. The time marks of both systems were synchronized with the manual timing for stimulus and manually recorded events to within ±1 s.
The significance level was set at 5% unless otherwise reported. The ability of the independent variables (BIS, AAI, and CePROP) to detect the level of OAA/S, LORlash, and LORnoxiouswas evaluated using prediction probability (PK), which compares the performance of independent variables with different units of measure, as developed by Smith et al. 25,26Consider a variable such as BIS or AAI and a “definitive standard” measure of anesthetic depth such as the multilevel OAA/S score or the two-level responsiveness (yes/no) to eyelash reflex or noxious stimulus. Then, a PKof 1 for the BIS or AAI variable would mean that BIS or AAI always increases (decreases) as the anesthesia gets lighter (deeper) according to the definitive standard depth measure. Such an independent variable can perfectly measure anesthetic depth. Alternatively, a PKvalue of 0.5 would mean that the independent variable is useless for measuring anesthetic depth. For the OAA/S score, a PKwas computed for all OAA/S levels combined. Similarly, PKvalues for LORlashand LORnoxiouswere determined. The jackknife method was used to compute the SE of the estimate, based on the assumption that all assessments were independent. 25,26A Student t test with Bonferroni correction was used to evaluate whether the PKfor one variable was different from another one. Significance level was set at 0.0167. Prediction probability was calculated using a custom spreadsheet macro, PKMACRO, developed by Smith et al. 25,26The PKvalue was calculated for each independent variable in each group. A PKanalysis for each independent variable with the three groups pooled was performed.
To study whether the combined information from BIS and AAI together with the drug effect-site concentrations offers more accurate information than the independent variables alone, two new composite variables have been designed based on the combined information from BIS + CePROP + CeREMI and AAI + CePROP + CeREMI, and a PKanalysis was performed on both composite variables. The OAA/S score was used as clinical comparator. The PKon the combined information was calculated using ordinal logistic regression, as described in the Appendix.
The power on the PKvalues was calculated using a t statistic defined as the difference considered of clinical importance divided by the SE of the difference between two independent variables. Assuming a PKdifference of 0.05 as being of significant with an SE of 0.02, 15 patients should be included to find significant differences with P < 0.025 (Bonferroni correction for two t tests). Previous assumption were based on previous results of the AAI and other AEP-independent variables. 27
By applying Probit analyses, the effective concentration or index at which 50% (ED50) and 95% (ED95) of the patients reached LORverbal, LORlash, and LORnoxiouswere calculated for all independent variables. For all independent variables, ED50and ED95values were compared between groups using one-way analysis of variance statistics. If significant, an unpaired two-sided Student t test with Bonferroni correction was used (P < 0.0167).
We calculated cutoff (threshold) values for the ability of the BIS, AAI, and CePROP to detect LORverbal, LORlash, and LORnoxiousin each group. For these calculations, we used “positive” to denote a test result that suggested responsiveness and “negative” to denote a test result that suggested nonresponsiveness. We assumed that increases in the BIS and AAI and a decrease in CePROP corresponded to an increased likelihood of responsiveness. We computed sensitivity as the proportion of responsive patients with positive test results (value higher than cutoff value for BIS and AAI and lower than cutoff value for CePROP). Similarly, we computed specificity as the proportion of nonresponsive patients with negative test results (value lower than cutoff value for BIS and AAI and higher than cutoff value for CePROP). We computed the cutoff values for each independent variable and specificity at a level of 100% sensitivity and at which the sum of sensitivity and specificity were highest. The same sensitivity/specificity analyses were performed for composite variables.
The demographics (mean ± SD) of the 45 female patients in the three groups are shown in table 2. No significant demographic differences were found between groups.
Figures 1A and Bshow the behavior of BIS and AAI versus CePROP and CeREMI. The correlation coefficient between BIS and the drug effect-site concentrations (r = 0.92) was significantly higher than for AAI (r = 0.82). In all groups, a stepwise increase in CePROP resulted in a monotonic decrease in the OAA/S score. BIS and AAI decreased in all groups with decreasing OAA/S scores, as shown in figure 2.
Figures 3A–Cshow the behavior of BIS, AAI, and CePROP at LORlash. For all independent variables, a significant decrease of the variable values between the responsive and unresponsive state was observed in all groups. The BIS, AAI, and CePROP values at LORnoxiousare plotted in figures 3D–F. In all groups, all independent variables changed significantly.
By applying Probit analyses, the effective BIS, AAI, or CePROP at which 50% (ED50) and 95% (ED95) of the patients produced a LORverbal, LORlash, and LORnoxiouswere calculated and are shown in table 3. For all three groups, the probability curves for the observations at LORverbal, LORlash, and LORnoxiousare shown in figures 4A–Cfor BIS, in figures 4D–Ffor AAI, and in figures 5A–Cfor CePROP.
The ability of the BIS, AAI, and CePROP to predict the level of the OAA/S, LORlash, and LORnoxiousas presented by the PKvalues is shown in table 4. Overall, similar performances were found for BIS, AAI, and CePROP. A PKanalysis for each independent variable with the three groups pooled revealed similar ability to detect both the OAA/S levels and LORlash. In contrast, the overall ability for detection of LORnoxiouswas lower when pooling the groups together.
By calculating the t statistic, we found that this study including 15 patients for each group had the power to determine significant differences between independent variables to predict OAA/S larger than 0.052, which is in accordance with our initial assumption that only differences larger than 0.05 would be considered significantly different.
The PKvalues to detect the OAA/S level using the combinations of input variables were better compared to the PKvalues of pooled data for BIS, AAI, and CePROP alone. For the composite index using the combination BIS + CePROP + CeREMI, the PKvalue was 0.91 (SE = 0.01), and for the composite index using the combination AAI + CePROP + CeREMI, it was 0.93 (SE = 0.01) to detect the OAA/S level.
We also performed more in-depth sensitivity/specificity analysis. Within the three groups, table 5shows the cutoff values for each independent variable at which the sum of the sensitivity and specificity was the highest, representing the independent variable value where the overall “errors” are minimized. 28 Table 6shows the different cutoff values and their specificity at level of 100% sensitivity. When using the combined information of BIS or AAI together with CePROP and CeREMI, the sensitivity/specificity profiles were more accurate. For both combinations, the best sensitivity-versus -specificity combinations were found at 100% sensitivity for 82% specificity.
Table 7shows at which average OAA/S score at LORlashand LORnoxiousoccurred in each of the three groups. The OAA/S is an ordinal scale, but it was considered continuous to be able calculate an average of OAA/S score with a decimal value.
This study was conducted to compare the performance accuracy of the independent variables BIS, AAI, and CePROP to measure anesthetic depth and how opiates quantitatively influence this information. Therefore, we observed the influence of remifentanil on the accuracy of the AAI, a new index calculated from the MLAEP, to reflect the hypnotic component of anesthesia (measured by the different levels of the OAA/S) and to measure loss of responses to different stimulation defined as LORverbal, LORlash, and LORnoxiousduring stepwise increased levels of propofol infusion with and without remifentanil. The performance of AAI was compared with that of BIS and CePROP.
In previous work, our group concluded that, during propofol anesthesia (without opiates) under steady state conditions, BIS, AAI, and CePROP were accurate independent variables to measure the hypnotic component of anesthesia and loss of responsiveness to different stimuli. Hemodynamic variables did not perform accurately enough to measure the hypnotic-anesthetic status of the patient. BIS correlated best with CePROP, followed by the AAI. Hemodynamics did not correlate well. 8To study the hypnotic component of anesthesia, two clinical measures (also called dependent variables) were used. We selected the OAA/S score because it provides a good correlation with a clinical reflection of the hypnotic component of anesthesia and has been tested prospectively. 29Also, loss of eyelash reflex was used because it is a simple binary variable and commonly applied by anesthesiologists in clinical practice to detect loss of consciousness. 14
In all groups, a stepwise increase in CePROP resulted in a monotonic decrease in BIS and AAI as shown in figure 1. Similar to our previous work using propofol alone, 8the correlation between BIS and the combined drug effect-site concentrations was higher than for the AAI. In all groups, a monotonic decrease in both BIS and AAI was observed at decreasing OAA/S levels as shown in figure 2. Similar findings were found for LORlash, as seen in figures 3A–C.
LORverbaland LORlashwere reached at higher independent variable values in a dose-dependent manner when adding remifentanil. This can be concluded from the Probit analyses shown in figure 4and the extracted ED50and ED95as seen in table 3. Although some differences between groups for the ED50and ED95did not reach the level of significance, a clear trend could be observed. Our results are in contrast with the findings from Iselin-Chaves et al. , 30who reported no significant influence of opiates on the BIS and MLAEP to detect the probability of loss of consciousness and lack of recall by using the OAA/S score. However, they used lower concentrations of opiates (50 and 100 ng/ml alfentanil; potency ratio alfentanil:remifentanil around 40:1). Recently, Mi et al. 14also found that when applying fentanyl pretreatment during propofol administration, the hypnotic endpoints were achieved at higher BIS values. The addition of remifentanil potentiates the effect of propofol. LORverbaland LORlashwere detected at lower CePROP in the 2 and 4 ng/ml remifentanil groups compared to the 0 ng/ml remifentanil group, as shown in figure 5. Also, EC50and EC95propofol decreased significantly or tended to decrease in a remifentanil dose-dependent matter, as also described previously by others. 31
A possible bias might have been caused by the influence of opiates on the clinical dependent variables. Both OAA/S (from level 2 to 0) and eyelash reflex use tactile stimuli, which might have been lost earlier when adding remifentanil. To demonstrate this, the averaged OAA/S scores at LORlashand LORnoxiouswere calculated and are shown in table 7. LORlashresults showed some increase. A clear increase in the OAA/S value is observed when remifentanil is added. In the 2 and 4 ng/ml remi-fentanil groups, several patients were still responsive at LORnoxious. This resulted in larger transitions when the transition between the response and the unresponsive state, for example, from OAA/S 3 to OAA/S 0 without passing through OAA/S levels 2 or 1, causing an increase in averages. This finding obviously confirms that remifentanil blocks the response to pain, without causing LORverbaland LORlash, but proves also that it might be difficult to quantify the dependence of the electronic independent variables such as BIS and AAI to opiates because the definitive standard to which they were compared changed its behavior through the groups. This fact highlights that care should be taken when validating loss of response to different stimuli by using OAA/S score or eyelash reflex when opioids are administered. However, because no better validated clinical scoring techniques exist at this moment, we have to accept this possible bias.
Skepticism still exists in the literature regarding the accuracy of cerebrally derived parameters to measure anesthetic depth when adding opiates. Several articles suggested a weaker correlation between BIS and CePROP in the presence of opiates. 14,32,33Although they state that this might reveal the importance of an analgesic component on the efficacy of depth-of-anesthesia electronic monitors, these studies did not apply specific statistical techniques such as prediction probability 25or specificity/sensitivity calculations. 1Also, one should clearly differentiate the phenomenon of the influence of opiates on the changes in cutoff values for LORverbal, LORlash, and LORnoxiousand the overall change in accuracy of these anesthetic depth independent variables when adding opiates.
The prediction probability, PK, provides a good alternative to investigate the overall relative performance of the different independent variables to measure the hypnotic component of anesthesia and loss of responsiveness to different stimuli. 25For BIS, Lysakowski et al. 34found no decrease in PKwhen adding clinical dosages of fentanyl, alfentanil, remifentanil, or sufentanil versus placebo during propofol administration. Iselin-Chaves et al . 16studied the influence of different dosages of alfentanil on the accuracy of both BIS and nonprocessed MLAEP during propofol administration and found no differences in PKbetween groups. In their study, BIS performed better than MLAEP. In our study, as indicated in table 4, the performance results indicate that both BIS and AAI are reliable independent variables for assessing the level of OAA/S and LORlashand did not decrease by the addition of remifentanil. Before being able to conclude that the ability to measure the hypnotic component remained intact, a PKanalysis with the three groups pooled was needed. As seen in table 4, the PKremained similar to the group-based PKfor both OAA/S and LORlash, indicating an overall equal accuracy of the independent variables with and without the addition of opiates.
At the level of significance used in our study, BIS and AAI were found to be comparable in performance to the estimated steady state propofol concentration, CePROP. When drug effect-site concentrations are known, one might argue that it only make sense to measure BIS or AAI if the combined information offers more accuracy in measuring depth of anesthesia than a single measure alone. Vice versa , it might be asked whether drug effect-site concentrations offer additional information for the clinician when BIS or AAI is used. However, it must be stated that electroencephalographic and MLAEP monitors can be attached to the patient at any random time, whereas calculation of drug effect-site concentration requires knowledge of the complete administration history. As none of the independent variables in the present study gave a PKlarger than 0.9, when pooling the data from the three groups into one, it could be interesting to explore whether a new independent variable defined as a composite of the electronic independent variables and the anesthetics concentrations would produce a larger PKwhen predicting the OAA/S. Therefore, we calculated the PKvalue for the combined information from both BIS + CePROP + CeREMI and AAI + CePROP + CeREMI using ordinal logistic regression to develop a predicted OAA/S score (see Appendix). For both combinations, the overall prediction probability for predicting the levels of OAA/S increased (or tended to increase) compared to the pooled PKdata.
Because the PKconcept was developed to generalize nonparametric receiver operating characteristic curves area to polytomous ordinal patient state, 26we thought it was interesting to observe some specific sensitivity/specificity characteristics for BIS, AAI, and CePROP. Statistically speaking, the most frequently applied combined sensitivity/specificity lies at the elbow of the receiver operating characteristic curve, where the sum of both sensitivity and specificity is the highest. 28However, as defined by J. Drummond 35recently, a depth of anesthesia-independent variable should have, at a minimum, a 100% sensitivity (no false negatives) if what the clinicians seek is a specific numeric threshold (cutoff value) that can be interpreted to mean “not aware.” Therefore, we observed the influence of remifentanil on the cutoff value for LORverbaland LORlashand the corresponding sensitivity/specificity profiles for BIS, AAI, and CePROP at the two important points of the receiver operating characteristic curves: (1) highest sum of sensitivity and specificity and (2) level of 100% sensitivity. For both BIS and AAI, the cutoff values at LORverbaland LORlashrevealing the highest sum of sensitivity/specificity clearly increased in a dose-dependent manner during remifentanil addition. However, the changes in the values of the highest sum were minimal and similar between the different anesthetic depth-independent variables, as observed in table 5. This means that the addition of remifentanil did not decrease the overall sensitivity/specificity profiles of the tested independent variables when observing the cutoff value at the level of maximum combined sensitivity and specificity. As shown in table 6, the cutoff values detecting LORverbaland LORlashat the 100% sensitivity level were influenced by the addition of remifentanil. For BIS, a trend toward higher cutoff values at the 100% sensitivity level was observed at LORverbaland LORlash. For AAI, the cutoff value only increased in the 4 ng/ml remifentanil group when observing LORverbaland slightly increased between the 0 and 2 ng/ml remifentanil groups, between the 0 and 4 ng/ml remifentanil groups, and when observing LORlash. The CePROP cutoff values decreased at this 100% sensitivity level when adding remifentanil. For all independent variables, the influence of remifentanil on the specificity at a level of 100% sensitivity was calculated. For the AAI, a clear decrease in specificity was observed in the 2 ng/ml remifentanil group when observing LORverbal, and a decrease between the 2 and 4 ng/ml remifentanil groups was seen when observing LORlash. This might be caused by the severity of the used statistical criterion, where one outlier might cause a dramatic decrease in specificity at the 100% sensitivity level. No decrease in specificity was observed between groups for BIS when observing LORverbaland LORlash. Inconsistencies are found in the data for CePROP when comparing the specificity results obtained by LORverbalwith these obtained by using LORlash. Better sensitivity/specificity profiles were obtained when the information from drug concentrations and BIS or AAI were used together.
This study also tested the influence of opiates on the performance of the independent variables to predict LORnoxious. The supramaximal tetanic stimulus used in this study was previously used by others as a substitute for conventional forms of stimulation in humans. 36In previous work, when only using propofol without opiates, we observed that measures from the cerebral cortex such as BIS and AAI were poor predictors for LORnoxious. 8The overall accuracy for LORnoxioustended to be lower for all independent variables compared with their hypnotic prediction accuracy. The PKvalues were lower, however, because of the larger SE, not statistically significant at the level of significance used in our study. Sensitivity/specificity profiles for all independent variables in the three groups were calculated at the two classic points. As seen in table 5, the values of the highest sum of sensitivity and specificity to LORnoxiouswere lower compared with the values for the detection of LORverbaland LORlash. Also, the cutoff values at the point of highest sensitivity/specificity were clearly influenced by the addition of remifentanil in a dose-dependent manner. At the level of 100% sensitivity to detect LORnoxious, a very low specificity and a clear influence of remifentanil on the cutoff values was observed (table 6).
A strong interaction between propofol and remifentanil is observed at the ED50and ED95levels when studying the analgesic component of this interaction at LORnoxious, as plotted in figure 5and table 3. This interaction is more pronounced than the hypnotic interaction between remifentanil and propofol, resulting in the fact that in this “staircase study,” patients in the 2 and 4 ng/ml remifentanil groups might reach LORnoxiousbefore reaching LORverbaland LORlash, as shown in table 7. Similar findings were observed for fentanyl and propofol by Smith et al. 37and by Katoh et al. 38for fentanyl and sevoflurane.
In conclusion, we found that although BIS, AAI, and CePROP were increased by remifentanil during propofol administration, their ability to detect OAA/S and LORlashremained accurate. Improved performance is obtained when BIS and AAI are measured in conjunction with drug effect-site concentrations. Remifentanil decreases the ability of these independent variables to detect LORnoxious.
The authors thank Paul Manberg, Ph.D. (Vice-President, Aspect Medical Systems, Newton, Illinois), Bob Butterfield (Research Fellow, Alaris Medical Systems, San Diego, California), Bernardo Rodriguez and Pablo Martinez (Research Fellows, Danmeter A/S, Odense, Denmark), and Geert Byttebier and Mieke De Bosschere (Statisticians, General Biometric Services and Consulting, Gent, Belgium) for their help during the preparation of this manuscript.
Appendix: Definition of an OAA/S Predictor Based on Combinations of AAI/BIS, Propofol, and Remifentanil Effect Concentrations
Two composite indexes were defined to study whether the combined information BIS + CePROP + CeREMI or AAI + CePROP + CeREMI offers more accuracy in loss of response to hypnotic stimuli than a single measure alone (BIS, AAI, CeREMI, and CePROP). The composite index was defined as an OAA/S predictor based on combined information from the data set BIS + CePROP + CeREMI or AAI + CePROP + CeREMI. The method of choice for designing a composite index was the ordinal logistic regression. The ordinal logistic regression has the advantage, as compared with multiple linear regression, that the output variable is ordinal. An ordinal variable is a categorical variable that has three or more levels of natural ordering, such as awake, drowsy, asleep, and deep asleep. This is suitable for the kind of data from this study because the OAA/S scale is ordinal and not linear. Two models were proposed: model A, where the input variables are BIS, CePROP, and CeREMI; and model B, where the input variables are AAI, CePROP, and CeREMI. The composite index is the output variable that in both models is a prediction of the OAA/S as shown in figures 6 and 7. The composite index is therefore ordinal in the 0–5 range as the OAA/S.
An ordinal logistic regression model was calculated by using SPSS software (SPSS Inc., Chicago, IL), which also provides a more detailed description of the method. The model is composed of constants, θi, factors, βn, and a link function, here the “logit.” The constants are associated with the ith event, which in the current study refers to the level of OAA/S.
The factors are obtained from each covariable (input), which here are BIS, AAI, CePROP, and CeREMI at every jth sample in time. Hence, the two models are defined in equations 1 and 2
The accumulated probability of being at or lower than the ith level of OAA/S at the jth sample in time, γij, is now calculated as
Example: The following state has been recorded: remifentanil = 0 ng/ml, propofol = 4 μg/ml, AAI = 25. When using these data as input to model B, the corresponding probabilities for OAA/S 0, 1, 2, 3, 4, and 5 are 0.7944, 0.8867, 0.9620, 0.9935, 0.9998, and 1. The model output is always the OAA/S level, where the accumulated probability exceeds 0.67. In this example, the predicted OAA/S is 0 because the first probability (0.7944) is larger than 0.67.
Tables 8 and 9show the parameters of the ordinal logistic regression for models A and B, with the corresponding levels of significance for each constant and factor as well as the P value for the whole model. A P value less than 0.05 means that the parameter is significant in the model. Both models were statistically significant. The OAA/S values predicted by the two models were compared with the measured OAA/S and evaluated using the PKanalysis, shown in table 10.