Background

The Hypotension Prediction Index (the index) software is a machine learning algorithm that detects physiologic changes that may lead to hypotension. The original validation used a case control (backward) analysis that has been suggested to be biased. This study therefore conducted a cohort (forward) analysis and compared this to the original validation technique.

Methods

A retrospective analysis of data from previously reported studies was conducted. All data were analyzed identically with two different methodologies, and receiver operating characteristic curves were constructed. Both backward and forward analyses were performed to examine differences in area under the receiver operating characteristic curves for the Hypotension Prediction Index and other hemodynamic variables to predict a mean arterial pressure less than 65 mmHg for at least 1 min 5, 10, and 15 min in advance.

Results

The analysis included 2,022 patients, yielding 4,152,124 measurements taken at 20-s intervals. The area under the curve for the index predicting hypotension analyzed by backward and forward methodologies respectively was 0.957 (95% CI, 0.947 to 0.964) versus 0.923 (95% CI, 0.912 to 0.933) 5 min in advance, 0.933 (95% CI, 0.924 to 0.942) versus 0.923 (95% CI, 0.911 to 0.933) 10 min in advance, and 0.929 (95% CI, 0.918 to 0.938) versus 0.926 (95% CI, 0.914 to 0.937) 15 min in advance. No variable other than mean arterial pressure had an area under the curve greater than 0.7. The areas under the curve using forward analysis for mean arterial pressure predicting hypotension 5, 10, and 15 min in advance were 0.932 (95% CI, 0.920 to 0.940), 0.929 (95% CI, 0.918 to 0.938), and 0.932 (95% CI, 0.921 to 0.940), respectively. The R2 for the variation in the index due to MAP was 0.77.

Conclusions

Using an updated methodology, the study found that the utility of the Hypotension Prediction Index to predict future hypotensive events is high, with an area under the receiver operating characteristics curve similar to that of the original validation method.

Editor’s Perspective
What We Already Know about This Topic
  • The hypotension prediction index is an alarm system approved by the Food and Drug Administration used to predict a mean arterial pressure less than 65 mmHg for at least 1 min in the operating room and critical care environments.

  • It is based on a proprietary algorithm derived using machine learning from components of the arterial waveform processed using pulse contour analysis methods.

  • A previous simulation study in Anesthesiology suggests that the predictive ability of the index might have been influenced by a selection bias due to use of a “backward” (case control method) in which a gray zone was used.

  • It has been suggested that the use of a “forward” (cohort) methodology may be a more clinically appropriate validation method.

What This Article Tells Us That Is New
  • Using a deidentified cohort pooled from nine previous studies involving operative and critical care populations monitored either invasively or noninvasively, this study analyzed the area under the receiver operator curve using either a backward approach with a gray zone or a forward approach without a gray zone, as well as relation of the index to concurrent mean arterial pressure in predicting hypotension 5, 10, and 15 min in advance. Other hemodynamic variables were also assessed as secondary analyses along with clinically based predictive indices.

  • The area under the receiver operating characteristics curve values for the index were very high and similar for either the backward or forward approaches, and concurrent MAP was the only variable with an area under the receiver operating characteristics curve greater than 0.7 with similar values to the index. The results were similar between clinical cohorts and mode of monitoring.

  • Although the areas under the receiver operating characteristics curve were very high, the positive predictive value for either the index or the concurrent mean arterial pressure were high in the backward analysis but low in the forward analysis, suggesting that future clinical studies need to carefully consider this issue in their design.

The Acumen Hypotension Prediction Index (“the index”) software is a logistic regression machine learning–based algorithm that detects early hemodynamic instability that may lead to hypotensive events defined as a mean arterial pressure (MAP) of less than 65 mmHg for at least 1 min. The algorithm is a probabilistic model that analyzes multiple arterial pressure waveform features and their inter-relationships to detect physiologic alterations in central compensatory mechanisms that precede episodes of hypotension. The index has been shown to be an accurate predictor of future hypotensive events, although studies linking hypotension reduction to improved postoperative outcomes currently remain lacking.1–5 

The methods used for the index validation5  have been questioned by Enevoldsen and Vistisen,6  who also suggested that the concurrent value MAP may predict hypotension as well as the index. Based on simulated data, they postulated that the performance may be overestimated due to a selection bias in the definitions of hypotension and normotension. A hypotensive event allowed for the full range of preceding MAP values to be used in the model training data set. The same did not apply for normotension, which was defined as the midpoint of a continuous 30-min episode during which MAP was consistently greater than 75 mmHg. Enevoldsen and Vistisen6  concluded that a data sample with a MAP of less than 75 mmHg will therefore always correspond to a future hypotensive event. The accompanying editorial7  advocated for a validation including data for the full range of values for predictor variables and suggested that the analysis should mimic the flow of data seen in clinical practice, i.e. a cohort design using a forward-looking approach rather than the backward case control approach previously used.

Studies that previously used a forward methodology to validate the index4,8,9  with similar results to the original validation, however, were not explicit in how data were selected. We therefore provide a direct comparison of the original backward validation of the index software with the suggested forward methodology and address the issue of whether concurrent MAP values can predict future hypotension.

We conducted a retrospective analysis of prospectively gathered anonymized data. This study adhered to the Enhancing the Quality and Transparency of Health Research guidelines.10  The data were analyzed from subjects from nine previously reported studies.2–5,8,9,11–13  Institutional review board approval was not required because we analyzed pre-existing anonymized data. Analyses were based on invasive arterial waveforms via radial arterial cannulation, and noninvasive arterial waveforms were derived from a finger cuff (ClearSight, Edwards Lifesciences, USA). Detailed inclusion and exclusion criteria are presented in the individual reports, and a summary is provided in supplemental material table S1 (https://links.lww.com/ALN/D526). Briefly, all studies enrolled adults greater than 18 yr of age. Two studies collected noninvasive measurements in subjects undergoing noncardiac surgery.2,4  Five studies analyzed invasive blood pressure measurements in patients undergoing moderate to major surgery; one included elective total hip arthroplasties,12  two included elective major noncardiac surgery only,8,13  one included elective cardiac surgery requiring cardiopulmonary bypass only,3  and one included a mix of elective major noncardiac surgery and off-pump cardiac surgery.1  One study included patients in the intensive care unit (ICU) with a diagnosis of COVID,9  and one study included a mixed group containing predominately ICU patients but also mixed and surgery that included cardiac, neurosurgical, general, vascular, and thoracic surgeries.5 

From the collected data, variables were computed, including both the index and MAP. All collected variables are shown in supplementary data file 1 (https://links.lww.com/ALN/D527) and supplementary data file 2 (https://links.lww.com/ALN/D528). Our analysis was based on 20-s averages of beat-to-beat values. Poor arterial waveform signals (e.g., line flushing, significantly damped waveforms, and other waveform artifacts) were detected by the arterial beat detection algorithm and excluded from the analysis. Across all studies, 1.1% of data was categorized as unreliable for invasive A-line data, and 2.1% of data was categorized for noninvasive A-line data, equally distributed.

Statistical Methods

Descriptive statistics are presented as mean (SD) for normally distributed continuous data or median (25th to 75th percentiles) for non-normally distributed data. Normality of data was assessed by using the Shapiro–Wilk statistic. The data set was analyzed in its entirety to describe the total number of hypotensive events (MAP less than 65 mmHg for at least 1 min measured either invasively or noninvasively), absolute duration of hypotension, area under the threshold of 65 mmHg, and time-weighted average of area under the threshold. Hypotension Prediction Index and MAP data were plotted to assess their relationship, and the coefficient of determination was calculated to determine the proportion of the variation in the dependent variable that is predictable from the independent variable. In addition, a reclassification table was created to compare the performance of Hypotension Prediction Index and MAP for the prediction of a future MAP of less than 65mmHg (full details in supplementary material, https://links.lww.com/ALN/D526).

Receiver operating characteristic curve analysis was used to evaluate the performance of variables in predicting hypotension 5, 10, and 15 min before the hypotensive event. The receiver operating characteristic curve analysis was performed with both a backward analysis with dual boundary conditions (gray zone) and a forward analysis using all data points. Analysis was performed separately for invasive arterial waveform and noninvasive waveform data. We also performed receiver operating characteristic curve analysis on MAP, stroke volume (SV), and heart rate (HR) at various thresholds to assess the effect of receiver operating characteristic curve analysis on time series data to predict future values.

No data were available to determine whether potential interventions occurred to treat or prevent hypotension, which could include surgical incision, change in position, or vasopressor treatment, among others. Treatment interventions were assumed when MAP increased more than 5 mmHg within 20 s (most probably caused by vasopressor or inotropic injections) or when MAP increased more than 8 mmHg within 2 min (change in vasopressor or inotropic infusion rate or fluid bolus) when the MAP was less than 75 mmHg. As sensitivity analyses for the effects of excluding presumed treatment periods, we also conducted a forward analysis with a threshold of 10 mmHg MAP change to define an intervention, as well as an analysis with no interventions censored. We used bootstrapping to account for repeated measurements from each subject for the calculation of CI.14 

Analysis Method 1: Backward Analysis

Receiver operating characteristic curve analysis was performed to evaluate the performance of variables to predict a future hypotensive event including the index, MAP, and ΔMAP at different blood pressure thresholds. ΔMAP analysis was performed in the MAP ranges of 75 to 85 and 85 to 95 mmHg. ΔMAP was the difference in two MAP measurements 3 min apart. Our methodology has been described previously.5  In summary:

  • A hypotensive event was identified as a section of at least 1-min duration, with an MAP of less than 65 mmHg for all three 20-s averages. A positive data point was chosen as the data point at 5, 10, or 15 min before the hypotensive event. All positive data points were included in the analysis regardless of their MAP values.

  • A nonhypotensive event was identified as a 30-min continuous section of data points at least 20 min apart from any hypotensive event and with an MAP of greater than 75 mmHg for all data points in that section. A negative data point was chosen as the center data point of the nonhypotensive event to reduce intraclass correlation. Only one data point within a 30-min window was selected because selecting all the data points within the window would have introduced statistical bias because neighboring 20-s data points are highly correlated.

  • This approach incorporates a “gray zone” between MAPs of 65 and 75 mmHg, and values within this range were excluded. This is a zone of a quantitative test in which there is an area of values where the discriminatory performance is “insufficient,” in the sense that a value in the gray zone does not allow the target disease to be scored as either present or absent.15  A flow diagram illustrates the backward methodology in figure 1.

Fig. 1.

Flow diagram to illustrate the methodology and assignment of true and false positive and negative data points for the backward analysis. MAP, mean arterial pressure.

Fig. 1.

Flow diagram to illustrate the methodology and assignment of true and false positive and negative data points for the backward analysis. MAP, mean arterial pressure.

Close modal

Analysis Method 2: Forward Analysis

We used receiver operating characteristic curve analyses to evaluate the performance of the same variables as in the backward analyses. As previously, a hypotensive event was defined by an MAP of less than 65 mmHg persisting at least 1 min, with all other pressures being considered nonhypotensive events. In this analysis, every data point except those that were during hypotensive events or that had an unreliable arterial line trace were examined. Every prediction index data point was compared to a threshold (for example 85), and a window of 5, 10, or 15 min immediately after the data point was searched to identify any hypotensive events. No gray zones were used in this analysis.

As in previous work assessing the performance of real-time and continuous risk scores,4,9,16–19  the analysis sequence was as follows: A true positive prediction occurred when the index was above the designated threshold and there was a hypotensive event in the search window. A false positive prediction occurred when the index was above the designated threshold and there was no hypotensive event in the search window. A true negative prediction occurred when the index was below the designated threshold and there was no hypotensive event in the search window. A false negative prediction occurred when there was a hypotensive event that the index failed to predict at least 2 min in advance. The above process was repeated so that the designated threshold covers all possible values of the index. Similar analyses were also performed for other variables. A flow diagram illustrates the forward methodology in figure 2, similar to published clinical analyses of other real-time and continuous risk scores.4,9,16–19 

Fig. 2.

Flow diagram to illustrate the methodology and assignment of true and false positive and negative data points for the forward analysis. MAP, mean arterial pressure.

Fig. 2.

Flow diagram to illustrate the methodology and assignment of true and false positive and negative data points for the forward analysis. MAP, mean arterial pressure.

Close modal

Index versus MAP Analysis

The index versus MAP analysis was performed to assess the relationship between the index and MAP. All paired data points of the index and MAP from all patients were analyzed together; then, for each integer value of MAP, we calculated the mean, SD, minimum, and maximum of the corresponding index values. Similarly, we calculated the mean, SD, minimum, and maximum of corresponding MAP values for each integer value of the index.

All statistics were performed with MATLAB (version R2021a; The MathWorks Inc., USA). The data are available upon reasonable request and institutional review board approval. The analysis script can be found at https://github.com/InstabilityPrediction/Analysis2024.

A total of 2,022 patients (913 women and 1,109 men) with a mean age of 61 (14) years were included in the analysis. Based on 20-s recording epochs, there were a total of 4,152,124 data points. Among the enrolled patients, 339 (17%) were from an ICU, and 1,683 (83%) were intraoperative surgical patients. The data were obtained from an arterial catheter in 1,210 patients (60%) and from a ClearSight noninvasive cuff in 812 patients (40%).

The median monitoring time per patient was 230 min (interquartile ratio, 144 to 420 min), and 1,683 of 2,022 patients (83%) had at least one hypotensive event, which was defined as a MAP of less than 65 mmHg for 1 min or longer. In total, 24,654 hypotensive events were detected, and the median number of events per patient was 4 (1 to 10) with a median duration of 2 (1 to 5) min per event. The median cumulative duration of hypotension per patient was 13.3 (2.7 to 44.0) min, which was 5.5% (1.0 to 16.4%) of total monitoring time. The median area under the threshold of 65 mmHg was 79.9 mmHg/min (15,271 mmHg/min), and the median time-weighted average area under the threshold was 0.30 mmHg (0.05 to 0.96 mmHg).

A plot of all paired index and MAP data measured at the same time points is shown in figure 3. The R2 for the variation in the index due to MAP was 0.77. Mean, SD, minimum, and maximum values of the index for different MAP thresholds and vice versa are shown in supplemental material table S2 (https://links.lww.com/ALN/D526).

Fig. 3.

Index versus mean arterial pressure (MAP) plot for all paired data points. Each circle is a paired data point, the solid white line represents the mean of the index values for each distinct value of MAP, and whiskers represent 2 SDs. The red line indicates the region where the index has values greater than 85.

Fig. 3.

Index versus mean arterial pressure (MAP) plot for all paired data points. Each circle is a paired data point, the solid white line represents the mean of the index values for each distinct value of MAP, and whiskers represent 2 SDs. The red line indicates the region where the index has values greater than 85.

Close modal

Receiver Operating Curve Analysis: Comparison of a Backward versus Forward Analysis for the Index to Predict Hypotension

Receiver operating characteristic curves for the index predicting hypotension 5, 10, and 15 min in advance by backward and forward analysis are shown in figure 4 separated for invasive and noninvasive arterial waveform analysis. For invasive arterial waveform data, the areas under the receiver operating characteristics curve (AUC) for the index predicting hypotension by backward and forward methodology, respectively, were 0.957 (95% CI, 0.947 to 0.964; sensitivity, 86%; specificity, 95%) versus 0.923 (95% CI, 0.912 to 0.933; sensitivity, 87%; specificity, 84%) 5 min before the event, 0.933 (95% CI, 0.924 to 0.942; sensitivity, 81%; specificity, 93%) versus 0.923 (95% CI, 0.911 to 0.933; sensitivity, 88%; specificity, 83%) 10 min before the event, and 0.929 (95% CI, 0.918 to 0.938; sensitivity, 81%; specificity, 92%) versus 0.926 (95% CI, 0.914 to 0.937; sensitivity, 87%; specificity, 84%) 15 min before the event.

Fig. 4.

Receiver operating characteristics curves for the index predicting hypotension 5, 10, and 15 min apart by backward (dual-boundary gray zone) and forward analysis (single-boundary condition, no gray zone) separated for invasive and noninvasive arterial waveform analysis. AUC, area under the receiver operating characteristics curve.

Fig. 4.

Receiver operating characteristics curves for the index predicting hypotension 5, 10, and 15 min apart by backward (dual-boundary gray zone) and forward analysis (single-boundary condition, no gray zone) separated for invasive and noninvasive arterial waveform analysis. AUC, area under the receiver operating characteristics curve.

Close modal

For noninvasive arterial waveform data, the AUC for the index predicting hypotension by backward and forward methodology respectively was 0.950 (95% CI, 0.939 to 0.961; sensitivity, 85%; specificity, 94%) versus 0.917 (95% CI, 0.908 to 0.925; sensitivity, 88%; specificity, 84%) 5 min before the event; 0.930 (95% CI, 0.914 to 0.944; sensitivity, 81%; specificity, 93%) versus 0.918 (95% CI, 0.906 to 0.928; sensitivity, 87%; specificity, 86%) 10 min before the event, and 0.912 (95% CI, 0.893 to 0.929; sensitivity, 79%; specificity, 91%) versus 0.923 (95% CI, 0.910 to 0.933; sensitivity, 88%; specificity, 86%) 15 min before hypotension. The full details of all analyses for all hemodynamic variables including area under the curve, area under the precision recall curve, sensitivity, specificity, positive predictive value, negative predictive value, and optimal cutoff values are shown in supplemental data file 1 (backward analysis, https://links.lww.com/ALN/D527) and supplemental data file 2 (forward analysis, https://links.lww.com/ALN/D528).

Receiver Operating Characteristic Curve Analysis Using Hemodynamic Variables to Predict Future Values in Time Series Data

The AUCs using forward analysis for MAP predicting a MAP of less than 65 mmHg for at least 1 min 5, 10, and 15 min in the future were 0.932 (95% CI, 0.920 to 0.940; sensitivity, 86%; specificity, 86%), 0.929 (95% CI, 0.918 to 0.938; sensitivity, 87%; specificity, 85%), and 0.932 (95% CI, 0.921 to 0.940; sensitivity, 88%; specificity, 84%), respectively. Figure 5 shows receiver operating characteristic curves for SV, HR, and MAP predicting a decrease in future values below various thresholds 10 min into the future for invasive arterial waveform data only. All variables have high AUCs in predicting their future values regardless of the threshold chosen. MAP predicted its future value 10 min in advance of being less than 75 mmHg with an AUC of 0.908 (95% CI, 0.898 to 0.917), less than 70 mmHg with an AUC of 0.921 (95% CI, 0.911 to 0.929), and less than 65 mmHg with an AUC 0.929 (95% CI, 0.911 to 0.929). SV predicted its future value being less than 60 ml with an AUC of 0.924 (95% CI, 0.917 to 0.932) and less than 50 ml with an AUC of 0.929 (95% CI, 0.920 to 0.936). HR predicted its future value 10 min in advance, with an AUC of less than 60 beats/min of 0.973 (95% CI, 0.967 to 0.977) and an AUC for less than 50 beats/min of 0.980 (95% CI, 0.968 to 0.987).

Fig. 5.

Receiver operating characteristics curves for stroke volume (SV), heart rate (HR), and mean arterial pressure (MAP) predicting a decrease below various thresholds 10 min into the future using cohort analysis. AUC, area under the receiver operating characteristics curve.

Fig. 5.

Receiver operating characteristics curves for stroke volume (SV), heart rate (HR), and mean arterial pressure (MAP) predicting a decrease below various thresholds 10 min into the future using cohort analysis. AUC, area under the receiver operating characteristics curve.

Close modal

The area under the curve, sensitivity, specificity, positive predictive value, negative predictive value, and optimal cutoff for SV, HR, and MAP predicting their future values 10 min into the future at different thresholds are shown in supplemental data file 3 (https://links.lww.com/ALN/D529). A net reclassification improvement is shown in supplementary material table S3 (https://links.lww.com/ALN/D526) with a 45% net improvement in false positives compared to false negatives with the index compared to MAP assigning a 10:1 ratio to emphasize the importance of a nonevent and a 7.6% improvement assuming equal importance.20 

Using a forward methodology in the analysis of the index to predict hypotension defined as a MAP of less than 65 mmHg for at least 1 min, there was a significant reduction in the AUC compared to our previously reported backward methodology at 5 min only; however, the AUC remained greater than 0.9.1,5  Thus, the utility of the index to predict future hypotensive events remained excellent.

The original validation used a backward (case control methodology) in which hypotensive events were identified and then looked backward in time to see whether a variable predicted the event. This method means that, unlike the forward approach, interventions that avoid hypotension do not need to be identified. Using a forward approach, due to the lack of contemporaneous information on clinical treatment, potential interventions must be accounted for by predefined rapid changes in MAP, which is a potential weakness. Future studies aimed at validating predictive algorithms will require highly annotated data on interventions to avoid this confounder. However, it is a fair criticism that using a backward analysis does not represent how clinicians use data and that a forward methodology better mimics the flow of data in clinical practice.

As part of the backward methodology, a gray zone or zone of uncertainty was used in which hypotensive events were defined as a MAP of less than 65 mmHg and nonhypotensive events as greater than 75 mmHg. These definitions were selected because they span the most common range of reported harm thresholds for acute kidney injury and myocardial injury after noncardiac surgery; i.e. they have a higher prevalence below the lower threshold and a lower prevalence above the higher threshold.

The use of an uncertainty zone is common in clinical care because there is often ambiguity about disease states, especially at early stages. This type of ambiguity exists in many clinical areas with most measured clinical parameters because physiology is not a binary process, but rather an ensemble of complex and fuzzy processes.21  For example, such areas exist in labeling of definitive sepsis and definitive absence of sepsis,22  between clearly normal and clearly abnormal levels of blood pressure, and in the labeling of brain natriuretic peptide data for diagnosis of heart failure.15  Using a zone of uncertainty was proposed to evaluate biomarkers.23  This concept is common in machine learning, and as machine learning applications expand more in the medical field, using the concept of the zone of uncertainty in their development and validation will become critical for the clinical adoption of such technologies. In a sensitivity analysis, exclusion of the uncertainty zone did not significantly alter the predictive ability of the Hypotension Prediction Index.

Enevoldsen and Vistisen,6  in a simulated data set, suggested that a selection bias was present because of the definition of a nonhypotensive event and that this bias would cause an overestimation of the predictive ability of the index. In the original validation study, hypotension was defined by a MAP of less than 65 mmHg for 1 min using the full range of preceding MAP values. A nonhypotensive event was identified at the midpoint of a 30-min continuous section of data points at least 20 min away from any hypotensive event and with a MAP greater than 75 mmHg for all data points in that section. Enevoldsen and Vistisen6  also concluded that the index was therefore taught that if the MAP is less than 75 mmHg, then the only possible future event is hypotension. This is not how the index was trained, and hence, it does not perform as proposed. At a MAP of 75 mmHg, the index can take any value from 7 (low probability of hypotension) to 98 (high probability of hypotension); therefore, it does not assume that hypotension is the only possible outcome. Using all data points (no selection bias) resulted in only a small reduction in the AUC; however, there is the possibility that the predictive ability is overestimated or underestimated by using points that have high intraclass correlation; hence, the original methodology avoided this, also allowing for a balanced receiver operating characteristic curve analysis. Area under the precision recall curve can be used for analysis of unbalanced classes; however, the methodology uses the positive predictive value. The positive predictive value is not a direct metric to demonstrate the performance of algorithms because it is primarily affected by the prevalence of the hypotension. If prevalence is low, then the positive predictive value is low even if the algorithm has extraordinarily high performance of sensitivity and specificity.

Various investigators have asserted that the index is simply a mirror of MAP24,25  and that MAP itself can be used to predict and treat hypotension. First, although there is an average nonlinear relationship between MAP and the index, there is a difference when looking at the individual samples. The two variables convey different information, and the relationship between the index and MAP not only changes within individuals at different time points but also varies between different individuals.26  Neither MAP, systolic pressure, nor diastolic pressure are direct features in the index model. All 23 features of the index model are mathematically combinatorial features and represent combinatorial interaction effects and not static effects. There is no single dominant feature that influences the index, as the median influence values are similar.

Using current MAP to avoid hypotension is clearly an obvious solution and represents current practice. However, MAP has been available for hemodynamic management for a considerable period, and yet hypotension remains common.27  Although it is appealing to think the associations of intraoperative hypotension and poor outcomes has changed practice, recent studies have shown that the prevalence of hypotension remans high across all settings, surgery types, and patient groups.28,29  It is, however, a reasonable question to ask whether current MAP predicts future hypotension, as well as a complex proprietary algorithm.

A simple linear extrapolation of MAP predicts future hypotension better than the change in MAP in patients having high-risk noncardiac surgery,30  although the index was not compared in that analysis. Using the absolute value of MAP to predict future hypotension, rather than linear extrapolation or delta MAP, gives a high AUC as in this analysis. However, this reflects a statistical relationship that arises when using any variable as both the independent and dependent variable, as is shown in this analysis (fig. 4), in which SV, HR, and MAP all predict future values at any threshold in their range with a high AUC. Mathematical coupling of data in statistical analyses has been previously described.31  Due to succession of numbers in time series, any variable has high receiver operating characteristic curve when compared to a fixed threshold representing a number in the range of the times series variable. When using the current MAP to predict a MAP of less than 65 mmHg in the future, the receiver operating characteristic curve analysis simply illustrates the likelihood that a MAP value of 65 mmHg will be preceded by other MAP values (e.g., a MAP of 67 or 70 mmHg), which is intuitive because to reach a hypotensive event from a higher MAP, the blood pressure must pass through these values.

If it is assumed that MAP can predict itself, then there are additional clinical considerations. The optimal cutoff to predict a MAP of less than 65 mmHg would be 72 mmHg; therefore, this value should be always treated to avoid future hypotension, as has been suggested.6,25  This is an extrapolation of a statistical analysis that does not translate into reasonable clinical practice. Clinicians do not treat an absolute value of MAP without examining the previous trajectories. If a threshold of 72 mmHg is taken as an absolute treatment threshold, this would lead to overtreatment. In our data set, taking every MAP of 72 mmHg in the 1,210 patients monitored with an arterial catheter, only 18% led to a MAP of less than 65 mmHg within the next 15 min. Therefore 82% of the episodes would have been treated inappropriately because hypotension would not have occurred. This is of potential concern because increasing doses of vasopressors have been associated with worse outcomes.32,33  The index should be used in combination with MAP to distinguish patients likely to subsequently develop hypotension from those who are unlikely to, because it is a probabilistic model that provides that information.

The use of the index in clinical practice reduces hypotension compared to using a target MAP12,13,34  in some although not all trials.8  Education of clinicians on the importance of maintaining a target MAP threshold did not result in a reduction in hypotension, despite them believing they had sufficient knowledge and skills. The use of the index reduced hypotension and had perceived clinical utility.35 

There are several limitations to this analysis. First, this is a retrospective analysis, and data on interventions were not available. Interventions such as certain drug administration, change of ventilation or blood loss will predict hypotensive events, and the analysis does not account for these interventions in the operating room or the ICU. Future studies should include these data and need to evaluate the effect of using the index to minimize hypotension focusing on clinical outcomes such as acute kidney injury, myocardial injury after noncardiac surgery, or a composite outcome rather than numerical measurements of hypotension depth and duration. We thus had to make assumptions about interventions based on MAP patterns. However, the results were similar over a wide range of assumptions, including no adjustment for any interventions. Second, some clinicians had access to the index data, which presumably resulted in some potential future hypotensive episodes being treated before they occurred. However, treatment would generate false-positive predictions, making the index perform worse. Fully addressing this issue will require a data set, presumably obtained in the context of a trial, in which all interventions are well documented.

As predictive tools become more prevalent in clinical practice, consensus will be needed regarding the best methodology for validation of complex algorithms that considers the impact of both statistical bias and clinical utility. In the meantime, our analysis shows that the index software accurately predicts hypotensive events regardless of the validation methodology used.

Research Support

Support was provided solely from institutional and/or departmental sources.

Competing Interests

Dr. Davies, Dr. Fleming, Dr. Veelo, Dr. Maheshwari, Dr. Sessler, Dr. Sander, and Dr. Cannesson have received consultancy fees, speaker fees and funded research from Edwards Lifesciences (Irvine, California). Dr. Mythen was paid consultancy and speaker fees from Edwards Lifesciences. As of August 1, 2023, Dr. Mythen is a paid employee of Edwards Lifesciences. Dr. Mythen recently resigned as a founding Director of the Perioperative Quality Initiative and is also a board member of Evidence Based Perioperative Medicine CiC and co-editor in chief of TopMedTalk. Dr. Hatib, Dr. Jian, Dr. Scheeren, and Dr. Settels are paid employees of Edwards Lifesciences. Dr. Scheeren has received consultancy fees, speaker fees, and funded research from Masimo Inc. (Irvine, California). Dr. Vlaar has received consulting fees from Edwards Lifesciences (paid to institution) and Philips. Dr. Cannesson is the founder of Sironis (Newport Beach, California) and Perceptive Medical (Newport Beach, California) and owns patents and receives royalties for closed loop hemodynamic management technologies that have been licensed to Edwards Lifesciences. Dr. van der Ster declares no competing interests.

Supplemental Digital Content

Supplemental material: Additional analyses and tables, https://links.lww.com/ALN/D526

Supplemental table 1: Patient characteristics

Supplemental table 2: Corresponding Hypotension Prediction Index and MAP values at different thresholds.

Supplemental table 3: Reclassification table

Supplemental data file 1: Backward methodology full analysis, https://links.lww.com/ALN/D527

Supplemental data file 2: Forward methodology full analysis, https://links.lww.com/ALN/D528

Supplemental data file 3: Analysis of HR, SV, and MAP to predict future values, https://links.lww.com/ALN/D529

1.
Davies
SJ
,
Vistisen
ST
,
Jian
Z
,
Hatib
F
,
Scheeren
TWL
:
Ability of an arterial waveform analysis-derived Hypotension Prediction Index to predict future hypotensive events in surgical patients.
Anesth Analg
2020
;
130
:
352
9
2.
Maheshwari
K
,
Buddi
S
,
Jian
Z
et al
.:
Performance of the Hypotension Prediction Index with non-invasive arterial pressure waveforms in non-cardiac surgical patients.
J Clin Monit Comput
2021
;
35
:
71
8
3.
Shin
B
,
Maler
SA
,
Reddy
K
,
Fleming
NW
:
Use of the Hypotension Prediction Index during cardiac surgery.
J Cardiothorac Vasc Anesth
2021
;
35
:
1769
75
4.
Wijnberge
M
,
van der Ster
BJP
,
Geerts
BF
et al
.:
Clinical performance of a machine-learning algorithm to predict intra-operative hypotension with noninvasive arterial pressure waveforms: A cohort study.
Eur J Anaesthesiol
2021
;
38
:
609
15
5.
Hatib
F
,
Jian
Z
,
Buddi
S
et al
.:
Machine-learning algorithm to predict hypotension based on high-fidelity arterial pressure waveform analysis.
Anesthesiology
2018
;
129
:
663
74
6.
Enevoldsen
J
,
Vistisen
ST
:
Performance of the hypotension prediction index may be overestimated due to selection bias.
Anesthesiology
2022
;
137
:
283
9
7.
Wijeysundera
DN
,
McIsaac
DI
,
London
MJ
:
The promise and challenges of predictive analytics in perioperative care.
Anesthesiology
2022
;
137
:
275
9
8.
Maheshwari
K
,
Shimada
T
,
Yang
D
et al
.:
Hypotension Prediction Index for prevention of hypotension during moderate- to high-risk noncardiac surgery.
Anesthesiology
2020
;
133
:
1214
22
9.
van der Ven
WH
,
Terwindt
LE
,
Risvanoglu
N
et al
.:
Performance of a machine-learning algorithm to predict hypotension in mechanically ventilated patients with COVID-19 admitted to the intensive care unit: A cohort study.
J Clin Monit Comput
2022
;
36
:
1397
405
10.
von Elm
E
,
Altman
DG
,
Egger
M
,
Pocock
SJ
,
Gøtzsche
PC
,
Vandenbroucke
JP
;
STROBE Initiative
:
Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies.
BMJ
2007
;
335
:
806
8
11.
Davies
S
,
Jian
Z
,
Hatib
F
,
Gomes
A
,
Mythen
M
:
Indicators of haemodynamic instability and left ventricular function in a porcine model of esmolol induced negative inotropy.
J Clin Monit Comput
2022
;
37
:
651
9
12.
Schneck
E
,
Schulte
D
,
Habig
L
et al
.:
Hypotension Prediction Index based protocolized haemodynamic management reduces the incidence and duration of intraoperative hypotension in primary total hip arthroplasty: A single centre feasibility randomised blinded prospective interventional trial.
J Clin Monit Comput
2019
;
34
:
1149
58
13.
Wijnberge
M
,
Geerts
BF
,
Hol
L
et al
.:
Effect of a machine learning–derived early warning system for intraoperative hypotension vs. standard care on depth and duration of intraoperative hypotension during elective noncardiac surgery.
JAMA
2020
;
323
:
1052
9
14.
Liu
H
,
Li
G
,
Cumberland
WG
,
Wu
T
:
Testing statistical significance of the area under a receiving operating characteristics curve for repeated measures design with bootstrapping.
J Data Sci
2022
;
3
:
257
78
15.
Coste
J
,
Jourdain
P
,
Pouchot
J
:
A gray zone assigned to inconclusive results of quantitative diagnostic tests: Application to the use of brain natriuretic peptide for diagnosis of heart failure in acute dyspneic patients.
Clin Chem
2006
;
52
:
2229
35
16.
Bedoya
AD
,
Futoma
J
,
Clement
ME
et al
.:
Machine learning for early detection of sepsis: An internal and temporal validation study.
JAMIA Open
2020
;
3
:
252
60
17.
Hyland
SL
,
Faltys
M
,
Huser
M
et al
.:
Early prediction of circulatory failure in the intensive care unit using machine learning.
Nat Med
2020
;
26
:
364
73
18.
Tomasev
N
,
Glorot
X
,
Rae
JW
et al
.:
A clinically applicable approach to continuous prediction of future acute kidney injury.
Nature
2019
;
572
:
116
9
19.
Wiens
J
,
Guttag
J
,
Horvitz
E
:
Patient risk stratification with time-varying parameters: A multitask learning approach.
J Mach Learn Res
2016
;
17
:
2797
819
20.
Pencina
MJ
,
D’Agostino
RB
,
Vasan
RS
:
Statistical methods for assessment of added usefulness of new biomarkers.
Clin Chem Lab Med
2010
;
48
:
1703
11
21.
Pinsky
MR
:
Complexity modeling: Identify instability early.
Crit Care Med
2010
;
38
:
S649
55
22.
Vincent
JL
:
The clinical challenge of sepsis identification and monitoring.
PLoS Med
2016
;
13
:
e1002022
23.
Ray
P
,
Le Manach
Y
,
Riou
B
,
Houle
TT
:
Statistical evaluation of a biomarker.
Anesthesiology
2010
;
112
:
1023
40
24.
Michard
F
,
Futier
E
:
Predicting intraoperative hypotension: From hope to hype and back to reality.
Br J Anaesth
2023
;
131
:
199
201
25.
Michard
F
,
Biais
M
,
Futier
E
,
Romagnoli
S
:
Mirror, mirror on the wall, who is going to become hypotensive?
Eur J Anaesthesiol
2023
;
40
:
72
4
26.
Ripollés-Melchor
J
,
Ruiz-Escobar
A
,
Fernández-Valdes-Bango
P
et al
.:
Hypotension prediction index: From reactive to predictive hemodynamic management, the key to maintaining hemodynamic stability.
Front Anesthesiol
2023
;
2
:
1138175
27.
Panjasawatwong
K
,
Sessler
DI
,
Stapelfeldt
WH
et al
.:
A randomized trial of a supplemental alarm for critically low systolic blood pressure.
Anesth Analg
2015
;
121
:
1500
7
28.
Saasouh
W
,
Christensen
AL
,
Chappell
D
et al
.:
Intraoperative hypotension in ambulatory surgery centers.
J Clin Anesth
2023
;
90
:
111181
29.
Saasouh
W
,
Christensen
AL
,
Xing
F
et al
.:
Incidence of intraoperative hypotension during non-cardiac surgery in community anesthesia practice: A retrospective observational analysis.
Perioper Med
2023
;
12
:
1
10
30.
Jacquet-Lagrèze
M
,
Larue
A
,
Guilherme
E
et al
.:
Prediction of intraoperative hypotension from the linear extrapolation of mean arterial pressure.
Eur J Anaesthesiol
2022
;
39
:
574
81
31.
Archie
JP
:
Mathematic coupling of data.
Ann Surg
1981
;
193
:
296
303
32.
Ariyarathna
D
,
Bhonsle
A
,
Nim
J
et al
.:
Intraoperative vasopressor use and early postoperative acute kidney injury in elderly patients undergoing elective noncardiac surgery.
Ren Fail
2022
;
44
:
648
59
33.
Chiu
C
,
Fong
N
,
Lazzareschi
D
et al
.:
Fluids, vasopressors, and acute kidney injury after major abdominal surgery between 2015 and 2019: A multicentre retrospective analysis.
Br J Anaesth
2022
;
129
:
317
26
34.
Solares
GJ
,
Garcia
D
,
Monge Garcia
MI
et al
.:
Real-world outcomes of the hypotension prediction index in the management of intraoperative hypotension during non-cardiac surgery: A retrospective clinical study.
J Clin Monit Comput
2023
;
37
:
211
20
35.
de Keijzer
IN
,
Vos
JJ
,
Yates
D
et al
.:
Impact of clinicians’ behavior, an educational intervention with mandated blood pressure and the hypotension prediction index software on intraoperative hypotension: A mixed methods study.
J Clin Monit Comput
2023
;
38
:
325
35
This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.