To the Editor:

I enjoyed the recent article by Terekhov et al.,1  “Preoperative Surgical Risk Predictions Are Not Meaningfully Improved by Including the Surgical Apgar Score” (SAS). I value the contributions of these authors to this field of investigation, including their pioneering work with the SAS.2  The authors made two methodological choices that may have contributed to the study concluding no improvement in prediction, so I humbly offer two suggestions to permit a more definitive test of their hypothesis.

First, would the authors consider performing their analyses using an alternative sampling interval for vital signs? The authors constructed the vital signs components of the SAS, lowest heart rate, and lowest mean arterial pressure, using “instantaneous” measures, or the true lowest heart rate and lowest mean arterial pressure in the record. These “instantaneous” values for the SAS are the least useful option for predicting outcomes when compared with alternatives such as moving median values over 5- and 10-min windows.3  In essence, the choice of instantaneous values biases the assessment to no benefit of the SAS.

Second, would the authors consider adding a calculation of risk reclassification to better test the clinical utility of the SAS? The authors reported the c-statistic and Brier score to evaluate the utility of the SAS. Although statistically robust, neither of these measures provides clinical insight. Moreover, the c-statistic is known to change minimally even when important improvements are made with risk prediction.4  For this reason, the use of a reclassification measure may be applied to provide a more clinically meaningful assessment of change in risk prediction.5  Reclassification approaches can be problematic, but the concept of categorizing patients into high- and low-risk groups is clinically intuitive and actionable, because we treat high-risk patients differently such as with admission to the intensive care unit.

The potential for real-time risk revision is not known, and with these suggestions, the authors may be able to more robustly test its potential.

Competing Interests

Although the author is a current awardee of the Anesthesia Patient Safety Foundation and Anesthesia Quality Institute, this letter was written on separate time. “No Funding Received” is the most accurate description of the funding details. The author receives no funding from industry or honoraria and has no conflicts of interest to disclose.

References

1.
Terekhov
MA
,
Ehrenfeld
JM
,
Wanderer
JP
:
Preoperative surgical risk predictions are not meaningfully improved by including the Surgical Apgar Score: An analysis of the risk quantification index and present-on-admission risk models.
Anesthesiology
2015
;
123
:
1059
66
2.
Regenbogen
SE
,
Ehrenfeld
JM
,
Lipsitz
SR
,
Greenberg
CC
,
Hutter
MM
,
Gawande
AA
:
Utility of the surgical Apgar score: Validation in 4119 patients.
Arch Surg
2009
;
144
:
30
6; discussion 37
3.
Hyder
JA
,
Kor
DJ
,
Cima
RR
,
Subramanian
A
:
How to improve the performance of intraoperative risk models: An example with vital signs using the surgical Apgar score.
Anesth Analg
2013
;
117
:
1338
46
4.
Cook
NR
:
Use and misuse of the receiver operating characteristic curve in risk prediction.
Circulation
2007
;
115
:
928
35
5.
Leening
MJ
,
Vedder
MM
,
Witteman
JC
,
Pencina
MJ
,
Steyerberg
EW
:
Net reclassification improvement: Computation, interpretation, and controversies: A literature review and clinician’s guide.
Ann Intern Med
2014
;
160
:
122
31