Fallacy…. Really?

The word “fallacy” stands out in the title of the recent editorial by Avidan and Evers.¹  It is a word rarely encountered in the biomedical lexicon, because it implies a nontruth, or in this case, that the null hypothesis has been proven. Since the null hypothesis can only be disproven, the choice of the noun, “fallacy,” appears unduly well settled to us, particularly when used to characterize data presented in the authors’ evidentiary pyramid. It is instructive to recall that level I evidence is only achieved from a systematic review of level II evidence (randomized controlled trials [RCTs]). The systematic reviews referred to by Avidan and Evers are of level III and IV evidence. Thus, their “highest quality of evidence” is actually far from level I evidence. Moreover, many of the studies on the “not supporting an effect” edge of the pyramid report, on closer examination, reported clinically significant effect sizes of 20 to 50% in favor of persistent cognitive decline after surgery but were underpowered.^2–4  In our view, studies that cannot rule out clinically important effects cannot be used to bolster either side of the argument. Moreover, the positive study by Liu et al.⁵  was a prospective randomized trial, and Williams-Russo et al.’s⁶  randomized trial addressed a completely different question (regional vs. general anesthesia). Of note, the investigation being advocated as the nail-in-the-coffin was itself statistically positive, although the effect size was considered by its authors to be negligible.⁷  Which edge of the pyramid does this go on? Would the effect size have been larger if those lost to follow-up (1.3 times more likely to have had surgery) were included? Within the discordant twin pairs wherein previous surgery was associated with persistent cognitive decline (about half), was there an unrevealed risk factor leading to a larger effect size? Recent work by Sprung et al.⁸  is similar in that surgery was associated with persistent cognitive decline only when the additional risk factor of age was included. A study just released from Oregon Health and Science University⁹  went a step further. In a longitudinal prospective cohort, surgery was associated with persistent cognitive decline in the entire group, an effect that became stronger when focusing on subgroups, women, and ApoEε4 carriers. It has been argued that these “vulnerability” factors are simply comorbid surrogates for an accelerated downward cognitive trajectory as compared to others, hence the association with postoperative cognitive decline. However, there is sound clinical evidence for a superimposed inflammatory event accelerating such a trajectory,¹⁰  so the possibility cannot be discounted with a one-model-fits-all notion. Subgroup analyses are essential.

We argue that evidence for or against “persistent cognitive decline” after surgery is insufficient to lay the matter to rest. What steps should be taken to provide the evidence? Because retrospective, case–control, or cohort studies suffer from numerous well-recognized problems that limit interpretation and RCTs might never be large or long enough unless enriched for groups at risk, prospective, observational cohort experimental designs are preferred. Surgical and anesthetic independent variables must be clearly defined, explicitly justified, and well documented. Single “baseline,” presurgical psychometric evaluations are confounded by transient factors (e.g., emotional impacts of diagnosis, fear of procedure, sleep deprivation, new drugs), so are not an ideal proxy for serial evaluations of a patient’s cognitive trajectory. Because emergence delirium (hours), postoperative delirium (days), and postoperative cognitive decline (3 months, 12 months, and thereafter) may correlate with one another, seeking evidence of each phenotype in all patients is an efficient and well-justified use of scarce research assets. Minimally invasive biomarkers of cognition stand at the frontier of postsurgical cognition research, including magnetic resonance imaging, positron emission tomography neuroimaging and proteomic, metabolomic, exosomic, genomic, and epigenomic profiling. Accordingly, an article in the same issue of Anesthesiology showed how cerebrospinal fluid biomarkers were predictive of postoperative cognitive decline.¹¹  Biomarkers validated in the dementia and delirium fields may enhance presurgical risk assessments, promote patient matching, and provide objective indices to test the effects of modifiable risk factors and therapies in future RCTs. Just as postoperative cognitive trajectory and functional status should be part of every prospective trial, so should blood, saliva, and cerebrospinal fluid collections.

Beyond these badly needed studies, adoption of publication standards for postoperative cognitive investigations is overdue. Consensual standards for nomenclature, experimental designs, sample size estimates, psychometric test panels, study intervals and durations, reporting standards for surgical and anesthetic variables, inclusion and exclusion criteria, and background variables may be provided as publication guidelines by a consortium of journal editors and reviewers, in compliance, for example, with the Newcastle–Ottawa Scale for assessing the quality of nonrandomized studies. Particular scrutiny of external data selected for normative cognitive comparisons is urged, and differences between a sample cohort and the normative standards cohort must be identified, controlled, and reported. Periodic neurologic examinations to rule out other causes of cognitive impairment should be an expected threshold for publication, particularly when “persistent” dysfunction is examined. An encouraged option is to append prospective cohort studies of postsurgical dementia to any of the numerous ongoing longitudinal studies of cognitive aging. Outliers in prospective observational and RCTs provide an opportunity for more thorough explorations of patient-specific risk factors and comparisons. Finally, directors and sponsors of ongoing observational and RCT investigations of dementia may be more willing to incorporate surgical and anesthetic variables into their databases if advocated by a representative body of surgery and anesthesiology organizations than by isolated investigators.

Since scientists are rarely keen to study a “fallacy,” the use of this term has a chilling effect on further inquiry in the area. Although there are doubtless others who want to put this question behind us, it is clear to most that the issue of persistent cognitive decline after surgery is still unresolved. For all the reasons above, we consider it premature to discourage investigation into the question of why our patients keep telling us, “I’ve never been the same since my surgery,” and have herein proposed key elements that will facilitate an answer.

The authors declare no competing interests.