We thank Drs. Eckenhoff, Evered, and Hogan for engaging in an important debate on the issues we raised in our editorial, “The Fallacy of Persistent Postoperative Cognitive Decline.”1 Their letter challenges several aspects of our editorial including (1) our use of the word “fallacy”; (2) the content of our analysis; and (3) the logic of our argument. We welcome the opportunity to sequentially respond to each of these points.
First, we stand by our use of the word fallacy. To be clear, we are not asserting that the existence of persistent postoperative cognitive decline (POCD) has been definitively refuted, and is thus fallacious. Rather, our editorial suggests that persistent POCD is likely a post hoc, ergo propter hoc (after this, therefore because of this) misattribution fallacy. The fallacy is to assume causation purely on the basis of a temporal relationship. A relevant example of this type of fallacy is the assertion that measles vaccine causes autism. There is currently an alarming increase in the prevalence of autism spectrum disorders. Largely uncontrolled observational research has implicated measles vaccination, and tellingly, there are compelling anecdotes of toddlers who are cognitively normal before their vaccine and who shortly after become neurodevelopmentally impaired.2 Yet, based on the preponderance of evidence,3 most scientists are convinced that it is incorrect to attribute autism to measles vaccination. There is similarly an alarming increase in the prevalence of cognitive decline among older adults. Highly publicized uncontrolled observational studies have implicated surgery and anesthesia,4,5 and tellingly, there are compelling anecdotes of older adults who are cognitively normal before surgery, and thereafter rapidly become demented. Yet, based on the preponderance of evidence referred to in our editorial,1 we suggest that it is likely a fallacy to attribute persistent cognitive decline or incident dementia to uncomplicated surgery with general anesthesia.
Eckenhoff, Evered, and Hogan also challenge the content of our editorial, charging that we have misinterpreted nonsignificant results as evidence of a negligible effect.6 In citing selected examples, they point out that some of the studies on persistent POCD have found clinically significant, although statistically nonsignificant, results. They suggest that these studies have been underpowered (or too small), explaining why their results have not been statistically significant. There are two problems with their contention. The first is that meta-analyses including these very studies do not cumulatively find a statistically or clinically significant association between surgery/anesthesia and persistent POCD.7 The second is the misconception that studies that do not find statistically significant results are necessarily underpowered.8 The calculation of power after a study is completed is considered inappropriate, and confidence intervals are more informative.8 An assumption is often made that a larger study would reveal both a statistically and a clinically significant result. However, small studies often find large effect sizes, which are not replicated in larger, more rigorous trials.9 Indeed, it is frequently the case that when larger studies are conducted, strikingly large effects that were found in small trials vanish into clinical insignificance.9 In a similar vein, it is also an error to conflate statistical and clinical significance. Eckenhoff, Evered, and Hogan ask on which side of the pyramid the study by Dokkedal et al.10 should be placed, because it found some statistically significant, but clinically irrelevant, results. The answer is that a result that is less than the minimum clinically important difference should be viewed as a negative result. It is unsurprising that a large study with multiple statistical tests finds some statistically significant, albeit clinically negligible, results.
Finally, Eckenhoff, Evered, and Hogan challenge our logic, asserting that the evidential pyramid is not robust and that we have not proved the case against persistent POCD. Of course, one can never prove the nonexistence of anything! The burden of proof rests on providing evidence supporting the existence of persistent POCD. In our editorial, we evaluated this evidence and found it to be weak. Analogy to the following two controversial hypotheses illustrates our logic (1) that peptic ulcer disease is caused by a bacterium and (2) that vaccination causes autism. Like persistent POCD, neither of these hypotheses can be conclusively disproved, but with appropriate experimental designs, they could be strongly corroborated (if they were true). Indeed, the first hypothesis was boldly verified by Marshall, who infected himself with Helicobacter pylori, and has led to better health for countless people.11 The second is turning out to be a stubbornly persistent misattribution fallacy, which is leading to deadly measles outbreaks.3
Since Bedford12 proposed the persistent POCD hypothesis based on an uncontrolled case series in 1955, there have been numerous attempts to verify it. However, not even randomized controlled trials,13–15 comparing cardiac surgery patients (those considered to be at highest risk for persistent POCD) with patients undergoing percutaneous coronary intervention, have found evidence for persistent POCD. In fact, some of these studies found cognitive improvement after cardiac surgery.16 We are concerned that, despite the lack of corroboratory evidence, the misattribution fallacy endures in the popular press and in the medical community, and fear of persistent POCD dissuades many older adults from undergoing life-enhancing, elective surgery.
In conclusion, society has limited resources for research, and it is important that common public health problems are prioritized. Even if persistent POCD does occur, Eckenhoff, Evered, and Hogan concede that it is likely to be too rare for detection in randomized controlled trials, and instead they advocate large (expensive) prospective, observational cohort experimental designs. A 3,988-patient prospective cohort trial, published after our editorial, reinforces the evidence finding no indication of persistent cognitive decline or incident dementia attributable to surgery.17 In fact, its only significant finding was that patients with exposure to surgery and general anesthesia had a decreased risk of dementia.17
The authors declare no competing interests.