We greatly appreciate the interest of Drs. Myles, Scheinin, Långsjö, and Helwani in the Michigan Awareness Control Study (MACS).1 We are delighted that the study provoked a wide range of commentary from the semantics of clinical research terminology to the fascinating developments in the neuroscience of consciousness and anesthesia.
Dr. Myles states categorically that the methodology of MACS is not consistent with comparative effectiveness research (CER), referring to the Institute of Medicine’s definition for support. The Institute of Medicine defines CER as:
…the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.2
The report goes on to state that the “key elements of this definition are the direct comparison of effective interventions, the study of patients in typical day-to-day clinical care, and the aim of tailoring decisions to the needs of individual patients.” MACS resulted in the generation of evidence by a direct comparison of the benefits of alarms based on the alternative methods of minimum alveolar concentration and the Bispectral Index (BIS), to monitor and prevent the clinical condition of intraoperative awareness with explicit recall. Both end-tidal anesthetic concentration monitoring (B-Unaware trial)3 and BIS (B-Aware trial)4 had been previously demonstrated to be effective interventions that can prevent awareness and improve delivery of care in the intraoperative setting. The interventions were triggered by physiologic or pharmacologic data derived from individual patients across multiple care settings. After the investigators implemented the alerting algorithm through the perioperative information system at the beginning of the study, hundreds of nurse anesthetists, residents, and attending anesthesiologists engaged in their typical day-to-day clinical care, without any requirement on their part to set or optimize conditions for study success. We acknowledge that aspects of MACS and other awareness trials are subject to interpretation with respect to the designation of CER. However, Dr. Myles presents no compelling arguments for why MACS does not meet the essential definition of CER proposed by the Institute of Medicine.
Although we do not agree with his assertion, Dr. Myles furthermore states that the methodology of MACS is not generalizable outside of the “very specific setting” of the trial. It is therefore surprising that Dr. Myles incorporated the results of MACS into a meta-analysis, a statistical technique that relies on generalization. In a textbook on statistical methods,5 Drs. Myles and Gin propose that a meta-analysis should only include trials that have “similar patient groups, using a similar intervention, and measure similar endpoints.” Recognizing these constraints, we have attempted a synthesis of evidence regarding prevention of intraoperative awareness in a Clinical Commentary published recently in this journal.6 In agreement with Dr. Myles’s caveats about inclusion criteria for meta-analyses, we did not believe that the meta-analytic approach was appropriate to summarize the findings of the major awareness prevention trials. Specifically, we were concerned about disparate patient groups (high-risk patients vs. unselected surgical patients), different anesthetic techniques (volatile-based vs. total intravenous anesthesia), dissimilar interventions (routine clinical practice vs an alternative protocol in the control groups), and inconsistently defined endpoints (credible awareness report vs dreaming as criteria for “possible awareness”). Instead, we offered a nuanced discussion of the literature, providing an evidence-based care algorithm that could be used by providers around the world with limited resources (administering primarily volatile anesthetics with end-tidal concentration monitoring) and providers in more resource-rich care settings (in which there is availability of processed electroencephalographic monitors, disposable electrodes, electronic information systems, or target-controlled infusion pumps for total intravenous anesthesia). Furthermore, and in contrast to Dr. Myles, our approach allowed us to exclude studies that were clearly not designed to contribute to CER decision-making regarding intraoperative awareness (specifically that of Puri and Murthy7 ) and included trials of central relevance, such as the Chinese study by Zhang et al.8 examining BIS for the prevention of awareness in a cohort of patients receiving only total intravenous anesthesia. Finally, our review facilitated consideration of possible awareness events, which have potential psychological sequelae and which could not be studied with meta-analysis because of inconsistencies in definitions across the trials.
Dr. Myles concludes with the sentiment that our patients expect that findings from large clinical trials be incorporated into clinical practice quickly. We would argue that our patients also expect us to be judicious with respect to the adoption or elimination of interventions based on the results of a single study. The medical literature is rife with investigations that have led to the premature incorporation of practices that ultimately proved to be ineffective, harmful, or unnecessarily costly.
Drs. Scheinin and Långsjö raise interesting points regarding the performance of the BIS monitor and the cognitive neuroscience of consciousness. With respect to the upper threshold for alarms based on BIS, we did indeed use the value of 60, which is recommended based on initial studies focused on the endpoint of responsiveness to command. However, it is important to note that the median BIS value was approximately 40 across all patients in MACS and the companion BIS or Anesthetic Gas to Reduce Explicit Recall trial. Thus, it did not appear that practitioners were maintaining values at just below the upper limit. Drs. Scheinin and Långsjö offer interesting insight as to why values across these diverse studies converged on 40. With respect to the underlying neuroscience, we congratulate the authors for their Journal of Neuroscience article on the neurobiology of anesthetic emergence.9 In general, we believe strongly that the next generation of brain monitors for anesthesia should be based on the results of such studies. In particular, the limited neocortical involvement associated with the return of consciousness was a striking finding of the investigation. However, it is not yet clear how these data inform the performance of electroencephalographic monitors, which detect activity of the cortex. The determination of returning consciousness by Långsjö et al.9 was subject responsiveness to the command to open their eyes. This command may require minimal cortical involvement because eye opening can occur as a result of arousal in response to a variety of verbal or auditory stimuli rather than awareness, per se. The pioneering study of Långsjö et al.9 will require follow-up investigation to assess whether unambiguous commands or complex sensory experiences are associated with detectable cortical changes. Such studies will hopefully be informative in designing new monitors based on the neuroscience of consciousness.
Dr. Helwani inquires about the percentage of patients receiving total intravenous anesthesia in this study. Only 1.2% of patients in the cohort received total intravenous anesthesia; because no end-tidal concentration was available, the alerting algorithm was based solely on documented concentrations of propofol. (We refer Dr. Helwani to our study protocol10 and a preliminary alerting study11 for further information on the algorithm.) In patients receiving total intravenous anesthesia, one experienced definite awareness and one experienced possible awareness; both of these patients were in the anesthetic concentration arm of the trial. Dr. Helwani’s comment regarding the inter-rater agreement is an important one. The explanation we propose is that intraoperative awareness, by definition, is a subjective event and the distinction between definite or possible awareness cannot be guided by objective indices. Thus, interpretation is likely prone to variability among different raters, although the determination of “no awareness” was associated with better consistency. With respect to the post hoc “no intervention” group, Dr. Helwani is correct that decreased vigilance may have occurred. Indeed, we suggest that there may be a “dose–response” relationship between alert burden and the incidence of awareness events. We agree that a routine care group could be informative in future trials. However, with the evidence that has accumulated suggesting the effectiveness of anesthetic concentration–based or processed electroencephalogram–based protocols in preventing awareness, we are reticent about recommending “routine care” with no alerts as an ethical arm in a clinical trial. Finally, ongoing studies with the MACS,1 B-Unaware,3 and BAG-RECALL12 cohorts are focused on identifying risk factors for intraoperative awareness with explicit recall. It is our hope that these investigations will shed new light on this persistent and potentially catastrophic complication of anesthetic care.