Trust, but Verify: Examining the Role of Observational Data in Perioperative Decision-making

THE first description of knee arthroplasty dates back to late 1800s when artificial materials were used to repair the joint. In 1954, Shiers¹  performed a knee replacement using a metallic hinged joint, and the first description of the “modern” total knee arthroplasty (TKA) was published in 1971. Since this time, the histories of techniques for TKA and total hip arthroplasty (THA) have been typified by progressive refinements to operative approaches and prosthetic implants; yet overall, the fundamental principles of joint replacement have remained largely the same. In contrast, almost 60 years after the beginning of total joint arthroplasty, practicing anesthesiologists remain divided over basic questions regarding the best primary anesthetic technique for joint replacement procedures. In this issue of Anesthesiology, Memtsoudis et al.²  present compelling evidence from a retrospective study that neuraxial anesthetic techniques, including epidural or spinal blockade, may result in superior perioperative outcomes relative to general anesthesia alone after THA and TKA.

Memtsoudis et al.²  examined a previously validated, proprietary dataset including almost 400,000 patients undergoing THA and TKA between 2006 and 2010 at 400 hospitals in the United States, comparing rates of adverse outcomes among patients receiving general, neuraxial, or combined (general–neuraxial) anesthesia. In contrast to earlier findings in smaller cohorts that observed general anesthesia to be used in a minority of total joint arthroplasty procedures,³  74.8% of patients in the study by Memtsoudis et al.²  received general anesthesia, 14.2% received combined general–neuraxial anesthesia, and 11.1% received a neuraxial technique alone. Using multivariate regression to control for a number of potential confounders, the authors examined a range of outcomes of relevance to clinical decision-making and health policy. They found no difference in severity-adjusted mortality according to anesthesia type among THA patients, but noted statistically significant increases in 30-day mortality among TKA patients receiving general anesthesia relative to those receiving neuraxial or combined approaches. For patients undergoing THA and TKA alike, general anesthesia was associated with a greater adjusted odds of pulmonary complications, nonischemic cardiac complications, pneumonia, infectious complications, and acute renal failure.

What implications, if any, should these findings have for clinical practice? The study by Memtsoudis et al.²  is observational rather than experimental in nature, drawing on a retrospective analysis of a large, secondary dataset. As such, treatment assignment—to neuraxial, general, or combined anesthesia—is nonrandom, creating the real possibility that the authors’ findings may reflect the confounding effects of differences in patient severity rather than effects attributable to anesthesia type per se. The variables included in the approach by Memtsoudis et al.²  to risk adjustment address a range of potential confounders of the association between anesthesia type and outcome; however, these methods do not rule out the possibility that their results still may reflect biases created by unmeasured confounders not included in their regression models.

Such considerations are vital to understanding what insights such analyses can offer for clinical practice and what they cannot. Alternate approaches to observational data analysis, such as instrumental variable methods that use natural experiments to mimic randomized trial designs, go beyond multivariate regression techniques and hold potential to yield causal estimates of effect. Although such methods are slowly gaining acceptance in the biomedical literature,⁴  advocates of evidence-based medicine have long pointed to randomized trial designs as the “gold standard” approach to making inferences about the effects of medical treatments. Adhering to Paul Holland’s adage of “no causation without manipulation,” such arguments highlight the unique strength of randomized trials to potentially account for unmeasured confounders through the random assignment of individuals to one or another treatment.⁵ 

However, despite their important limitations, studies such as those of Memtsoudis et al.²  and others⁶  hold potential to yield useful insights for perioperative care. Many of the major perioperative events that Memtsoudis examines, such as renal failure, myocardial infarction, stroke, and 30-day mortality, are rare in the context of elective joint replacement, each occurring in less than 1 of 100 patients. As a result, the large sample sizes required to study the association of these events with anesthesia type may be difficult to achieve, either in individual randomized trials or in meta-analyses that pool data from multiple studies. Indeed, in the context of total joint replacement, meta-analyses of available trial data remain underpowered to detect such rare complications.^7,8 

The traditional focus of clinical trials on “efficacy”—how well a given treatment works under ideal conditions—rather than “effectiveness”—how well a treatment works when applied in “real world” settings—highlights other opportunities for observational studies to add to clinical decision-making. Notably, clinical researchers have recently focused new attention on the development of “pragmatic” trials that approximate the diversity of patients and working conditions seen in actual practice.⁹  Nonetheless, findings such as those of Memtsoudis et al.²  continue to offer important insights about the comparative effectiveness of differing anesthesia techniques for preventing adverse perioperative outcomes to complement the efficacy perspective of most current clinical trials.

Such considerations highlight a need for anesthesiologists and other care providers to take a reflective approach to determining the role that observational studies can play in guiding clinical practice. In an important critique of hierarchical “pyramid of evidence” approaches to grading and evaluating medical research studies, Tonelli¹⁰  has emphasized the extent to which viewpoints that characterize observational studies as offering a lower “strength” of evidence than randomized trials potentially foreclose important opportunities to draw insights from biomedical research. Indeed, to Tonelli, differing study designs offer distinct kinds of evidence that potentially complement one another and demand to be integrated and synthesized by the individual practitioner.

David Sackett, a founder of the evidence-based medicine movement, has commented that “the practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.”¹¹  Sackett’s emphasis on how clinicians can “follow the trail” of external evidence and their own expertise toward the best decisions highlights just how much may be gained by making pluralism, rather than parochialism, the guiding principle in using research evidence to inform clinical care. Anesthesiologists and other practitioners should approach the expanding body of observational research on perioperative care, of which the study by Memtsoudis et al.²  is a noteworthy example, with both optimism and caution for their unique insights and their inherent limitations. Although the work of Memtsoudis et al.²  is unlikely to be the final word on the longstanding question of the optimal primary anesthetic technique for total joint replacement, it will likely move practitioners one step nearer to understanding how the outcomes of THA and TKA can be improved. As clinicians and researchers endeavor together to come progressively closer to that goal, the work of Memtsoudis et al.²  stresses how they can benefit from an approach that aligns with a political truism—“trust, but verify”—in understanding what observational studies can teach us and what they cannot.