A Review of Current and Emerging Approaches to Address Failure-to-Rescue

Previous attempts to address unexpected deterioration in hospital inpatients fall into four main groups. First, there have been many retrospective studies of deterioration cases, which often attempt to identify common warning signs. Second, there are some systems that attempt to classify patient risk levels prospectively based on demographic or physiologic indicators. Third, some studies have described or evaluated the use of various technologic monitoring modalities to obtain continuous physiologic state information on hospital inpatients. Fourth, studies evaluated the impact of RRS and its executive arm, Medical Emergency Teams, on patient outcomes.

There has long been interest in using cases of deterioration and death to identify potential risk factors. A 1991 study by Bedell et al.  20examined all cardiopulmonary arrests in which resuscitations were attempted during the year 1981 at Boston's Beth Israel Hospital, a 504-bed university teaching hospital. The stated purpose of the study was to examine specifically those arrests judged to have “resulted from a therapy or procedure or from a clearly identified error of omission.” However, the study presents a thorough review of the demographic and clinical characteristics of all arrests. The mean age of patients who arrested was 70 years, and 43% were female. The most prevalent clinical conditions in patients' medical histories were coronary artery disease (76%) and congestive heart failure (76%). There was a mean of 10 days between hospital admission and arrest, and the arrests were spread relatively evenly over the ward (41%), ICU (37%), and emergency department (18%). Most arrests occurred in patients on the medical service (88%), as opposed to just 12% on the surgical service. Twenty-six percent were classified as respiratory arrests, with the rest having a primarily cardiac mechanism of arrest. Overall, 50% survived the arrest, 33% of those survived to discharge, and 68% of those discharged survived more than 1 yr after arrest.

More recent studies have both pursued more detail on the characteristics of deteriorating patients and widened the scope of the deteriorations analyzed. Bobay et al.  in 2008 examined patient factors in FTR cases at five Midwestern hospitals.21They used a National Quality Forum definition of FTR as death after one of five preventable postsurgical complications: sepsis, gastrointestinal bleeding, deep-venous thrombosis/pulmonary embolism, cardiac arrest/shock, or pneumonia.

A chart review of cases from five metropolitan Midwestern hospitals yielded 376 potential cases of FTR, although only five were ultimately judged to be nurse-sensitive FTR (the remainder had nursing care that “met established guidelines”). Reviewers used patient charts to collect 16 variables, including vital signs and laboratory results from up to 3 days before ICU admission. Heart rate, respiratory rate, temperature, blood sodium concentration, and urine output were shown to be statistically significant indicators of deterioration for this patient group, whereas the Glasgow Coma Scale and oxygen saturation were among the nonsignificant indicators. Age was also an important indicator of deterioration, with most cases occurring in patients older than 70 years, and the study authors suggest that closer monitoring is warranted for this population. The study also emphasizes the difficulty of isolating true FTR cases in large-scale retrospective chart reviews because most of the cases identified as probable by a computer search of complication codes proved to be non-FTR (either operating room deaths or deaths classified as nonpreventable by nurses who followed established guidelines) on closer review.

Buist et al.  in 2004 also investigated the predictive value of several physiologic indicators14using cases of cardiac arrest rather than FTR. Data were collected over 33 weeks in 1999 through daily visits to each ward by the investigators, who reviewed medical records and charts for any adverse events and asked staff members about any undocumented occurrences. Investigators looked specifically for 10 abnormal observations, including Spo²<90%, low or high blood pressure, low or high pulse rate, low or high breathing rate, and loss of or decreased level of consciousness. The 6,303 patients studied (including everyone on the monitored wards, regardless of resuscitation status) represented 38,115 patient days, with a mean stay length of 6 days. Abnormal observations were found in 8.9% of patients, with an overall rate of 4.2 events per 100 bed-days. Low Spo²represented approximately half of all abnormal observations, followed by hypotension at 17.3%, and tachycardia, hypertension, bradycardia, and high respiration rate (each of these at approximately 6%). Most of the abnormal observations were reported to have spontaneously resolved (66.7%), with 21.6% resolved through treatment on the ward.

In the 564 patients who exhibited abnormal physiologic signs, 146 (26%) later died. Univariate linear logistic regression was used to associate abnormal observations and mortality. Independent risk factors were also identified using elimination algorithms, and the odds of mortality associated with each independent factor were then estimated by a multiple linear logistic regression model. The strongest predictor of mortality in univariate analysis was found to be low respiratory rate (less than 6 breaths/min), which was associated with a 13.7-fold increase in the risk of mortality by hospital discharge. Changes in consciousness and high respiratory rate (more than 30 breaths/min) were also found to increase risk of mortality in the univariate model. In the multivariate analysis, changes of consciousness, hypotension, high or low respiratory rate, and low Spo²were all found to be significant independent predictors. High numbers of events also correlated with increased mortality, with a probability of death of 16.2% for patients with only one abnormal observation, compared with 88.2% for four or more. Overall, the study authors observed that changes in neurologic and respiratory status were associated with high risk of mortality, while accounting for less than 5% of all observations.

Although these studies have a diversity of patient populations, aims, and outcomes, they do share some important characteristics. Each relied on chart data and intermittent vital sign data, whether collected prospectively or retrospectively. Each found evidence that at least some unexpected deteriorations can be predicted, but none identified a single overriding factor that would predict all deteriorations. Furthermore, none described attempts to predict deterioration based on their findings. The complexity of data collection and analysis is representative of this class of studies, as are the conclusions presented. More examples are available in the literature.8,22–26 

Risk scoring systems represent an attempt to operationalize the findings of studies such as those presented previously. Various algorithms exist for synthesizing elements of patient medical history, demographic information, and clinical readings into a single score designed to indicate risk of deterioration, and several studies have examined their effectiveness. Whether created with the FTR problem in mind or not, these scoring systems all address the general question of how to evaluate patient severity, an important component of the RRS paradigm.

An early attempt at validation of one scoring system was made in 2001 by Subbe et al .27The study investigated the relationship between a Modified Early Warning Score (MEWS) and increased risk of death or critical care admission. A total of 709 medical emergency admission patients were included over a period of 1 month (although technical restrictions limited data collection for any given patient to 5 days) at a district general hospital; patients who were admitted directly to coronary care or critical care were excluded. Each patient was scored twice daily based on thresholds for nursing measurements of blood pressure, heart rate, respiratory rate, temperature, and the Alert, Voice, Pain, Unresponsiveness measure. For patients with a maximum score over a predetermined threshold of five, the study found increased odds ratios [OR] for death at 6 months (OR, 5.4), ICU admission (OR, 10.9), and high dependency unit admission (OR, 3.3). Critical care transfers were decided by physicians without using the MEWS score. The authors suggest that such a scoring system could be valuable in identifying patients from the general admissions pool who warrant increased clinical attention.

Morrice and Simpson in 2007 compared MEWS with three other algorithms, in a population of 67 adult inpatients.28The Intensive Care Society (ICS) classification divides patients into several levels based on their perceived risk of deterioration as identified by descriptive categories, e.g. , “needs can be met through normal ward care” as opposed to “a patient requiring staff with special expertise.” Although developed for use in ICU settings, ICS has been adapted for general ward use. The MEWS scores patients on physiologic measurements, with measurements outside of normal ranges attracting higher scores and indicating a greater overall risk of deterioration. The Therapeutic Intervention Scoring System (TISS-28) is designed to map the medical interventions required for a particular patient to an estimate of the nursing time taken to provide the care and has been used as a general acuity scoring mechanism for ward patients.

Morrice and Simpson first scored all adult inpatients using the ICS classification, then compared the classification groups using demographic and physiologic variables in addition to the three additional scoring systems listed previously. ANOVA techniques were used to test the hypothesis that the three ICS classification groups did in fact represent distinct patient populations as indicated by the alternative measures. The study concluded that physiologic measurements alone were relatively weak indicators of future deterioration and that although level two ICS patients (requiring a high level of external support) represented a distinct population, it was more difficult to identify significant differences between ICS levels 0 (normal ward care) and 1 (at serious risk of deterioration). In fact, the authors state that there is no single tool currently able to identify at-risk patients.

Kho et al. , also in 2007, explored an alternative approach to risk estimation using automated scoring of electronic medical records of 1,878 patients at a 725-bed academic medical center in Chicago.29This system was based on MEWS but subtracted the mental status element and added more readily available age and body mass index data. Vital signs data were collected from electronic charting performed by ward nurses. Sensitivity and specificity were calculated against the outcome of RRT activation. The study found that the modified MEWS had a receiver operating curve with area similar to the original MEWS, suggesting that an early warning system based only on electronic medical records data could perform as effectively as a more labor-intensive manual scoring technique at indicating RRT activation. The authors suggest that adding such an automated system could provide an additional level of safety in identifying some patients at risk of deterioration.

Finally, a systematic review of 36 papers describing 25 distinct scoring systems was performed by Gao et al.  in 2007.30The systems studied included several variations of Medical Emergency Teams calling criteria (primarily from Australia) and MEWS (primarily from England), together with multiple individual systems from Canada, the United States, and the United Kingdom. The review found that no system met the requirements of a level 1 clinical decision rule (a rule widely validated to change clinical behavior and improve outcomes). The review authors note the wide variety of published systems and a corresponding lack of repeated, statistically sound validation of any particular system. A review of datasets provided by study authors also revealed that many systems have low sensitivity and positive predictive value, and this may be because of high triggering thresholds; sensitivity could potentially be improved by triggering clinical response at lower scoring levels, although this would lead to increased workload.

Despite the preponderance of risk scoring techniques, there is no system that has been shown to perform reliably the difficult task of separating patients who will deteriorate from those of a similar acuity level who will not. All systems discussed here are also limited by their reliance on the medical history, which is relatively static, and on the intermittent vital signs checks performed by nursing staff. Similar results and limitations can be seen in the literature.31–33 

Continuous monitoring systems represent a more proactive approach to identifying patient deterioration, based on the premise that physiologic changes can indicate, and perhaps predict, deterioration episodes. Multiple technologies are in use for both the measuring of physiologic data and its analysis.

Electrocardiograph monitoring has long been used for cardiac patients and has been extended to other patient classes at risk of developing cardiac dysrhythmia. Larson and Brady in 2008 performed a review of the use of inpatient electrocardiogram monitoring to characterize its effect and attempt to identify common criteria for the use of electrocardiogram in the inpatient population.34The study identified numerous shortcomings in the use of electrocardiogram monitoring. Several studies have shown limited utility for electrocardiogram monitoring of patients with low-risk electrocardiographic patterns,35,36and that an electrocardiogram is not necessarily highly reliable in identifying even patients currently undergoing cardiac arrest.37In spite of this finding, physicians were shown to consistently overestimate the usefulness of cardiac monitoring when compared with the actual rate of care decisions made based on electrocardiogram readings. The authors found that unnecessary cardiac monitoring contributed to overcrowding of emergency departments through a shortage of ICU and electrocardiogram-equipped beds, an overburdening of cardiac telemetry wards leading to higher rates of missed cardiac events, and a serious financial burden on both hospitals and patients. The review concluded that correct identification of low-risk patients not in need of cardiac monitoring would offer substantial benefits.

Another approach to continuous monitoring is the multiparameter model advocated by Tarassenko and Oxford BioSignals Ltd. (Oxford, United Kingdom). BioSign brings together five physiologic parameters (heart rate, respiration rate, blood pressure, Spo², and skin temperature) into a single indicator of patient status.§The scoring system is based on a model of physiologic normality derived from a large training dataset of high-risk adult patient data. Deviations in the measured physiologic values from this learned normal model cause the patient status indicator to rise, and sufficiently large deviations trigger automated alerts.

In a study in which 168 patients in Oxford, England were monitored using BioSign, there were 105 patients with BioSign alerts, with a mean alert rate of one every 7.8 h. Clinicians reviewing the vital signs traces after the fact for the 5 min determined that some were caused by motion artifact. Later work by researchers not associated with the device or manufacturer, however, showed no effect on outcome measures between monitored and nonmonitored patient groups in a randomized control trial.38 

As these studies show, the promise of electronic physiologic monitoring for continuous detection and prediction of deterioration has not been fully realized.

Numerous studies have shown that warning signs of physiologic instability exist before cardiac arrests.8,10,14,16,20,24,39,40Most patients have a period of deterioration that precedes cardiopulmonary events by 6–8 h.7,22Hospitals have not been able to respond to these events early and adequately enough.7,14,22,26,39,40 

The notion of intervention during the time of deterioration, to actually prevent the adverse event, led to the idea of RRS and its executive arm, the RRT. These teams bypass the vertical approach of the medical consultation system (nurse, resident, consultation) to respond in a more timely manner by using horizontal activation systems (the patient's nurse activates the RRT). RRT members are typically drawn from the critical care providers at the institution, and specific activation criteria can vary considerably between hospital systems, although most current systems allow nurses to activate the response at their discretion even for patients who have not met a specific criterion.

The study of the effect of RRTs on outcomes is challenging. Because of the difficulties studying the topic by randomized controlled trials in the same hospital, studies have focused on either hospital randomization (cluster randomization)41–43or single-center before-and-after studies.44–46A recent meta-analysis16found reduced relative risks in hospitals with RRTs for cardiac arrests and hospital mortality but concluded that a general recommendation for RRTs was premature based on the heterogeneity and wide confidence intervals of the effect. Similarly, a Cochrane review performed in 200947identified only two studies42,43that were sufficiently rigorous for inclusion and judged the effects of RRTs to be inconclusive. The success of RRTs depends primarily on early detection of deterioration, and this intuitively makes sense—the RRT can only be as good as the monitoring that activates it. Indeed, studies have shown a circadian activation pattern48–50that correlates with nurse-to-patient ratios. Not surprisingly, Calzavacca et al .17found delayed medical emergency team (MET) activation to be the most important factor effecting outcome. In a recent review of research related to the adoption of medical emergency teams, Tee et al .51stated: “As the MET call criteria depend heavily on physiologic alteration of signs, poor monitoring equipment, methods and recognition by staff may be a major stumbling block in improving outcomes and RRS performance.”

Early detection of deterioration on a broad level requires comprehensive patient surveillance. Patient surveillance in the in-patient general care setting is a new concept in medicine that needs to be differentiated from condition monitoring. Historically, condition monitoring has been applied when physicians have identified patients to be at a risk for problems and then have dedicated resources to these patients (e.g. , cardiotelemetry). The resources may have consisted of closer observation or special monitoring. In contrast, the monitoring used in the operating room by anesthesiologists for low-risk patients is called surveillance monitoring. All patients are monitored per the American Society of Anesthesiologists standards (e.g. , oxygen saturation, blood pressure, electrocardiogram) without exception because the operating room is in itself a factor associated with unexpected events. This monitoring may be combined with that beyond routine or surveillance monitoring that is based on a patient's individual risk profile, known as condition monitoring. Examples of condition monitoring are the use of arterial lines for patients with risk for hemodynamic instability in the ICU or operating room as well as the use of pulmonary artery catheters in patients with risk for low cardiac output failure.

Condition monitoring has been used outside of the ICU and operating room environment in care areas that are usually unmonitored, e.g. , patients with sleep apnea who are using postoperative opioids. Condition monitoring of selected patients in general care settings is similar to monitoring patients in the operating room or the ICU in that alarms have a higher probability of being actionable alarms (alarms that trigger an intervention) than the monitoring of patients not perceived to be at risk using the same static threshold alarms (Andreas H. Taenzer, M.S., M.D., unpublished data, 2011). Data are based on incidence of self-correcting decreases of pulse oximetry oxygen saturation readings less than 93% with patient surveillance in the general care settings versus  actions required by readings of less than 93% in the operating room.

In surveillance monitoring outside the operating room, a large number of patients (e.g. , all patients on one ward, in a section, or an entire hospital) are monitored, making nuisance alarms a predominant problem. In the general ward setting, the nurse-to-patient ratio is lower, and physicians are typically less readily available, making the immediate attention to alarms more difficult. In these environments, the likelihood of alarms being activated is much less than in condition monitoring or surveillance monitoring in high-risk settings such as the operating room. DeVita et al.  stated “there was concern that current technology is clinically inadequate due to a potential for high false positive or negative rates.”18 

Nuisance alarms consist of false-positive and nonactive alarms. The concept of nuisance alarms has been used in the human factor, ergonomic, and medical literature.52–55Frequent nuisance alarms lead to desensitization of personnel and discounting of alarms.52Thus, the implementation of surveillance monitoring requires a different approach to alarm variables than condition monitoring on alarm parameters in terms of static alarm settings as well as notification delay.

Although early warning scoring systems have been able to take data at low temporal resolution and predict deterioration with some success, they are demonstrably unable to catch fast-developing deteriorations that are not preindicated by some element of the patient's medical history or demographic characteristics. Furthermore, vital sign checks are by their nature interruptive; providers checking vital signs will in most cases rouse the patient, intentionally or not. As patients who were asleep or resting become more active to assist in the data collection and to interact with the provider, many of their vital signs change: respiration rate will naturally increase, subsequently increasing Spo², and heart rate and blood pressure may also rise. This means that assessment of vital signs is not necessarily an accurate measure of patient physiology over time; providers are affecting the readings simply by obtaining them.

Continuous physiologic monitoring systems avoid the limitations of low data rates and misrepresentative readings associated with provider vital sign checks. However, most studies of continuous physiologic monitoring to date have been focused on making the case for feasibility and the value of monitoring systems in real-time detection of adverse events. Those studies that have attempted to predict deterioration in advance of adverse occurrences have not demonstrated significant clinical impact.34,38,56 

Surveillance monitoring of a large patient population must address the problem of human response to frequent alarms. Typically, alarms that are 90% accurate are usually acted upon whereas alarms that are 10% accurate are ignored.57Studies conducted in ICU settings showed that alarms are less than 10% accurate.58–60If surveillance monitoring is to be successful, the problems that cause alarms to be ignored in condition monitoring of ICU patients need to be addressed and overcome. Although training and policy adjustments are important to create an atmosphere where additional monitoring can succeed, appropriate management of alarm rates by adjusting thresholds and notification delays is the single most important factor in effective performance of the human element of surveillance monitoring systems.

Alarm thresholds may be chosen based on the underlying normal distribution of typical physiology of patients in one unit to find the optimal balance between sensitivity and specificity. More recently, an abstract by Taenzer et al.  described that the distribution of heart rates and oxygen saturation varies very little among surgical patients,∥implying that the same static alarm settings may be used for different patient groups. As has been the case with early warning scoring systems, it is likely that thresholds will be developed on an institutional basis to cope with differing patient populations and resource allocation strategies. Surveillance monitoring systems that include the ability to archive physiologic readings can be used to develop these customized thresholds with a sound statistical basis, addressing one of the main objections to widespread validation of existing early warning systems.

Notification delay is the other important concept in alarm frequency management. Appropriate delay eliminates many transient and motion artifact-generated false alarms by stipulating that an alarm condition must persist for a set amount of time before that alarm is annunciated. In a recent publication, Taenzer et al.  instituted a 15-s audio alarm delay at the bedside and an additional 15-s delay for pager annunciation, leading to a 30-s delay before a nurse would be notified by pager of violation of alarm thresholds.61Notification delay has been used as a concept to control nuisance alarms for smoke and fire alarm detectors as well as for network intrusion detection systems. It is increasingly used in ICU settings to help to cope with the problem of alarm fatigue.52In a setting such as inpatient wards where surveillance monitoring is completely new, an alarm delayed by a matter of seconds still represents a major improvement over hourly vital signs checks, and such a delay can have a large effect on nuisance alarm rates.

Filter mechanisms are another method to control alarms. Median filtering has been widely studied, using a variety of different techniques with a wide array of results. Mäkivirta et al.  used preprocessing median filtering in the ICU setting. The filter increased the fraction of true alarms from 12–49%, whereas the false alarm rate remained at 4.5 per hr per patient.62 

Together, alarm thresholds and notification delay represent points of control that an institution can use to bring an appropriate balance of sensitivity and specificity to surveillance monitoring systems. Furthermore, these controls can be adjusted frequently on a unit- or institution-wide basis without the need for staff retraining imposed by changes to manual scoring systems. This fine-grained control is critical to maintaining staff acceptance of the surveillance system while keeping sensitivity at high enough levels to affect patient safety.

Patient surveillance using biotechnology adds to currently implemented safeguards (direct interaction with patients to assess mental status and sample vital signs) and by no means should be considered to replace other monitoring means at this point. PS should not lure institutions into a false sense of security. Relying entirely on biotechnology and reassigning nurses to different tasks or lowering nurse-to-patient ratios are actions not supported by any data and are considered to be dangerous by the authors at this time.

Premature implementation of systems without fine-grained control of alarms will lead to alarm fatigue by first responders (registered nurses) and lack of acceptance.52In scenarios where the nursing staff is not convinced of the benefits of surveillance, patient acceptance of continuous monitoring usually declines (as experienced by the Dartmouth Patient Surveillance Group). As is commonly the case with introduction of new systems, PS needs to be implemented with support and education for those using and exposed to the system.#

Large gaps still exist with current technology, and we can only improve on the current FTR problem, not solve it. Institutions need to be aware that patients who are being transported through hospitals, waiting for diagnostic tests in hallways or waiting areas, continue to be unmonitored until all surveillance is completely wireless with access everywhere, with complete patient acceptance and fail-free.

Continuous patient monitoring with alarm settings designed for a collective group of patients will benefit a group of patients. This is important to emphasize because it is a departure from what is most common practice in medicine: to tailor and individualize care based on a patient's specific needs. Therefore, PS cannot be assumed to be the sole solution for the detection of physiologic deteriorations of all patients in the general ward setting. It provides a broad underlying safety net for those currently unmonitored, and individual monitoring for patients may be added.

Cost and benefit of surveillance must be carefully considered. Institutions or patient care areas that have no adverse events will likely derive no benefit from PS. Cost effectiveness of surveillance will depend on many factors too complex to discuss here in detail. With cost control in health care of utmost importance, deploying systems with considerable cost needs to be carefully weighed against their benefit. In addition, at times safeguard systems develop autonomy of their own and are not just a subject of cost-benefit considerations, but become ethically mandatory (code teams).

Finally, one has to be aware that continuous monitoring may be prompting unnecessary responses, disturbing patients, distracting and interrupting nurses in their work, and leading to harmful interventions as discussed in other settings of monitoring and screening.63–65Many quality improvement interventions are changes to complex systems and are subject to unintended consequences as a side effect of change as pointed out by Reason.66 

Recently, the effect of implementation of a PS system based on pulse oximetry with wireless nursing notification (Patient SafetyNet™; Masimo, Irvine, CA) on transfers to the ICU and rescue events was described.61Detection of deterioration in this model occurs at an earlier time point than in other described models and goes beyond the optimization of RRTs because the goal is to prompt intervention even before RRT activation becomes necessary. Indeed, a decrease of MET activations from 3.4-1.2 per 1,000 patient discharges (and ICU transfers from 5.2-2.9 per 1,000 patient days) (fig. 2) was shown.61In an accompanying editorial,67Abenstein and Narr remarked that “The importance of the Dartmouth study is introduction of a rudimentary electronic decision support system to a traditionally unmonitored ward and its associated results ... These systems will be able to filter those patients who require intervention from those who do not.”67 

After a decade of research and work on RRS we believe we will enter a decade of research on PS. Much work has been and continues to be done on the efferent limb of RRS; this needs to be complemented by work on the afferent limb. In this review we have argued that continuous patient monitoring should be the next step for early intervention in the FTR field.

Research in PS needs to take into account that this is a multidisciplinary effort that will only work with support from all stakeholders in the process, including patients, nurses, physicians, and administration. Similar to the evaluation of the effect of the response component of RRTs, the success of continuous patient monitoring will be influenced by local hospital culture. We are just beginning to understand why similar process interventions work in some institutions but not others. Therefore, research efforts need to expand beyond the monitoring modalities to the issues of successful deployment: What training should be offered and received by clinicians (nurses and physicians), biotechnologists, and administrators? What structures and processes hinder or facilitate success? Although system configurations will likely continue to be customized to the needs of individual institutions, can a set of best practices for system deployment be described and validated?

Regarding continuous monitoring systems, much work needs to be done in establishing which parameters and what combination of them are the optimal physiologic variables to monitor; if variables should be combined to a summarized score or if they should be considered individually in triggering alarms; if monitoring should be done with static alarm limits, trends, or a combination; if PS should be used universally or only for population groups at risk; if patient monitoring is cost-effective; and if monitoring with automated technology is better or worse than monitoring with additional staff.

As technology or the application of existing technology to new areas progresses, respiratory rate monitoring, transcutaneous carbon dioxide, and hemoglobin are among the variables likely to be added to surveillance based on current hardware development trends. Ongoing research will doubtlessly identify additional physiologic measures. In general, as parameters are added, we need to be aware of the effect of doing so. Adding variables with a specificity of 90% will result in an overall system specificity of 81% for two, 73% for three, and 65% for four variables. Doing the same for a specificity of 95% will yield 90%, 85%, and 81% overall system specificity, respectively. To find an optimized specificity and sensitivity for each parameter as well as for the system as a whole while controlling for nuisance alarms will require much work and likely would be best done in simulation on an available, representative dataset before implementation and testing in clinical settings. To control nuisance alarms, specificity needs to be high, most likely at the cost of sensitivity. As more parameters are added, this balance will become increasingly difficult. The nature of monitoring data also makes data mining and machine learning techniques very attractive. This class of analysis methods can extract time- and frequency-domain relationships between physiologic data and deterioration that are not necessarily evident in statistical approaches nor to standard clinical assessment. A wide range of machine-learning techniques has been applied to clinical data in the past, including fuzzy logic,68neural networks,69,70patient-specific trained classifiers,71wavelets,72syntactic analysis,73Bayesian networks,74and Markovian models.75None of these studies had data on non-ICU general ward patients who underwent continuous monitoring. But the same principles apply and these techniques potentially could introduce smart alarm control.

Monitoring the effect of quality improvement projects in the area of patient safety is challenging. Measurable outcome variables depend on many factors, such as patients, providers, and organizations and their interplay, which remain poorly understood.76,77Unanticipated consequences such as new problems,78,79failure, or uncertainty to reach the desired goal have been common when complex systems have altered with the goal of improving them.80,81 

The outcomes of most interest are rare (mortality, codes, rescue events, and ICU transfers); the criteria for some outcomes are ill-defined, changing, or depend on other factors (when to transfer a patient to the ICU, bed availability); and the denominator, the patient population, is ever changing and so is the associated risk profile. Because of these challenges, the evaluation of new techniques or strategies for patient surveillance is difficult. The measurement of effect requires enormous power and sample sizes because of the rarity of the relevant outcome variables. Similar to other areas of patient safety and quality improvement, it will be easier to study effect with process control charts or before-and-after studies, rather than using methods that establish a very high level of evidence such as cluster randomized controlled trials.

The effect of PS needs to be measured on multiple levels: satisfaction of patients; patient outcomes (as difficult as that may be, as discussed); satisfaction of users; and financial ramifications. Some of theses outcomes can follow examples of related interventions, and others may require new approaches. Patient and family satisfaction likely can be measured with traditional measures such as Likert or Visual Analog Scales. Patient outcomes may be analyzed with similar means as RRT interventions with some caveats. Although rescue calls in traditional evaluations of the efferent limb of RRTs often increase as a sign of a functional and mature system, in PS a reduced number of calls may signify an intervention that occurs so early in a potential deterioration process that a rescue event is not necessary. Transfers to the ICU are considered not very reliable indicators of the success of RRT because the decision to transfer is affected by too many other variables (e.g. , available bed space, changing indications for transfers, varying practices among institutions). In PS, a change in ICU transfers may further indicate a change in culture: physicians may feel comfortable keeping patients on a regular ward because they are now continuously monitored.

The satisfaction of those who use the system is also an important component of assessing PS. Parameters such as nursing satisfaction as well as nursing staff retention rates are easy to track and are meaningful, although isolating the effects of any individual factor on staffing satisfaction is challenging. The addition of PS-specific probes to satisfaction surveys is one potential technique.

Finally, in times of financial constraints in health care, we are obligated to analyze the cost-effectiveness of new systems when we plan to introduce them into medicine (even though this has been rarely done in the past and established systems such as a code team are rarely questioned).

In summary, tremendous opportunities for improvement of patient safety and research exist in the field of PS—a truly new and exciting area in medicine.

The authors thank George T. Blike, M.D., Professor of Anesthesiology, Dartmouth Medical School and Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire.