In Reply

We thank Kappen et al. for their interest in our article describing the development of the Preoperative Score to Predict Postoperative Mortality (POSPOM).¹  The inconsistencies are consequences of an unfortunate error in the text (in the validation cohort, a POSPOM score equal to 30 was associated to a predicted in-hospital mortality of 7.40%, and not 5.65%). All tables and figures have been verified and corrected. An erratum has been issued and appears in this issue.² 

In our validation cohort of 2,789,932 patients, only 22,136 (0.79%) had a predicted risk of in-hospital mortality greater than 10%. As a consequence, we intentionally presented a truncated calibration plot focusing on the probability range that includes more than 99% of patients. The reporting of calibration for prediction models remains difficult, as underlined by the comment by Kappen et al. Graphical analysis of the calibration plot remains the preferred approach.^3,4  However, most patients were at very low risk of in-hospital mortality. This resulted in a heavily left-skewed distribution, making the histogram of predicted probabilities uninformative. Because the calibration plot included more than 99% of patients, we believe that it is correct to state that in the validation cohort, POSPOM “has good calibration with only a small underestimation of in-hospital mortality in the validation cohort for predicted probabilities ranging from 1 to 10%.” However, we agree with the authors that a closer inspection of those patients with high probabilities of in-hospital mortality is required.

As noted by Kappen et al., POSPOM overestimates the risk of in-hospital mortality in patients with a risk greater than 10%. The observed in-hospital mortality in these patients was 13.3%, and the average predicted risk in these patients was 17.1%, as presented in the figure provided by Kappen et al.

From a clinical point of view, we believe that any preoperative risk of in-hospital mortality greater than 10% (i.e., 20 times the average risk in our population) reflects very high-risk procedures. The role of a general preoperative assessment tool (e.g., POSPOM) is not to distinguish between patients with a postoperative mortality risk of 23 and 62%, especially as these cases are uncommon (409 patients presented a POSPOM greater than or equal to 40 in the validation cohort, namely, 0.01% of the population). Rather, it aims to identify clinical situations that would require further preoperative investigations to determine an appropriate care strategy.

Beyond the discussion about the appropriateness of the calibration of POSPOM in our validation cohort,³  we believe that an evaluation of the performance of POSPOM in a completely different cohort (i.e., true external validation) is the necessary next step before implementation of this prediction model.