Anesthesiologist Direction and Patient Outcomes

All Pennsylvania Medicare claims records for patients 65 yr or older were analyzed for general and orthopedic surgical admissions between 1991 and 1994. The study involved 194,430 “directed” and 23,010 “undirected” patients in 245 hospitals. Outcomes studied included death rate within 30 days of admission, in-hospital complication rate, and the failure-to-rescue rate (defined as the rate of death after complications). We obtained the Medicare Standard Analytic Files for all general surgical and orthopedic DRGs (diagnosis-related groups) in Pennsylvania between 1991 and 1994 (Medicare Part A data). For each patient we created a longitudinal record by appending all medical and surgical inpatient and outpatient claims and physicians’ claims (Medicare Part B data) during that time interval. Data also included the American Hospital Association Annual Surveys for 1991–1993, and the Pennsylvania Health Care Cost Containment Council Data Base for years 1991–1994.

We developed predictive models for a random sample of 50% of Medicare patients who underwent general surgical or orthopedic procedures in Pennsylvania between 1991–1994 and tested our results on the other 50%. Final results are reported regarding the full sample of 217,440 individual patients. The DRGs included in this study are listed in table 1. The first hospital admission for any one of these DRGs triggered the identification of a study hospital admission.

During the years discussed in this study, the Healthcare Financing Administration (HCFA) required that anesthesia care be either medically directed or supervised by a physician (supervision is defined as a level of physician participation that is less than that defined by medical direction). According to HCFA, the supervisor or director must have been a licensed physician, but not necessarily an anesthesiologist. 4To bill for medical direction, as defined by HCFA, 5physicians must have met all the criteria listed in table 2. Otherwise, the level of involvement was defined as “supervision” and physicians received markedly reduced payment.

Cases billed to Medicare as “personally performed” or directed by an anesthesiologist were defined in this study as directed. Otherwise, cases were defined as undirected.

Personally performed cases also included those in which an anesthesiology resident was directed by an attending anesthesiologist. (Anesthesiologist cases in which residents were directed were billed as personally performed for the first 3 yr of the study interval, and changes in the HCFA guidelines caused direction of resident cases to be billed as “directing 2–4 cases” in the final year of the study.)

There were 23,010 patients defined as undirected in this study, of which 14,137 patients (61% of the undirected group) were not billed for anesthesia and 8,873 (39%) were billed for anesthesia. The “no-bill” cases were defined as undirected because there was no evidence of anesthesiologist direction, despite a strong financial incentive for an anesthesiologist to bill Medicare if a billable service had been performed. The cases in which an anesthesiology bill was not submitted showed billing data that indicated that a surgical procedure on our study list was performed. These cases either were supervised by a physician or a staff nurse anesthetist employed directly by the hospital or they represented undirected anesthesiology resident cases. Of these 14,137 no-bill cases, only 1,287 at most were anesthesia resident cases (or 5.6% of all undirected cases), assuming all no-bill cases at institutions with anesthesia residency programs reflected resident cases. The remaining undirected cases consisted of 8,873 patients (39% of the undirected group) for which procedures were supervised but not directed by an anesthesiologist or directed by a nonanesthesiologist physician. None of these cases included residents. Billing codes included “unknown physician specialty” (code 99) or “unknown provider” (code 88) associated with a nurse anesthetist specialty code 43 or nonanesthesiologist physician direction of the nurse anesthetist, including many other specialty designations, such as pathology (code 22) or general medicine (code 11). Of the 217,440 patients, 20,066 (9.9%) patients underwent anesthesia procedures on more than 1 day during their hospital stay. We labeled a patient undirected if on any day of the hospital stay, all anesthesia procedures performed that day were not directed by an anesthesiologist.

In HCFA billing records the specialty code for anesthesiologist is denoted by an “05” designation. Anesthesiologist designation did not imply board certification. We used information from the American Board of Medical Specialties (ABMS) to verify Medicare data. In one instance, Medicare data indicated that the directing physician was a nonanesthesiologist, yet that same physician was noted to be board certified in anesthesiology according to the American Board of Medical Specialties files. We therefore recoded that person as an anesthesiologist for our purposes.

Death within 30 days of admission was determined from the HCFA Vital Status file. Complications (table 3) were identified using a set of 41 events defined by International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) and CPT (Physician’s Current Procedural Terminology, 4th edition) codes available from HCFA databases for the hospital stay of interest, previous hospital stays, and outpatient visits within 3 months before the index hospital stay. CPT codes billed before the hospital stay were used to determine long-standing conditions that would aid in distinguishing complications from comorbidities. Failure-to-rescue rate (FR) was defined as the 30-day death rate in those in whom either a complication developed or who died without a recorded complication. It can be expressed mathematically as follows: FR = D/(C + D|no C) or the number of patients who died (D) divided by the number of patients with complications (C) plus the number of patients who died without complications noted in the claims data (D|no C). 6,7 

Estimates of excess deaths/1,000 patients were derived using a direct standardization approach using the full data set for both the directed and the undirected cases. 8Using the final fully adjusted model, the probability of death was estimated twice for each of the 217,440 patients in the study, once assuming each case was undirected and once assuming the case was directed. The resultant difference between the sum of the estimated death rates, divided by the sample size, and multiplied by 1,000, provides the number of excess deaths/1,000 patients when cases are not directed. The same method was used to estimate the excess number of failure-to-rescue cases in the undirected group, except the denominator of cases includes only those with complications. The advantage of this standardization approach is that all patients are used for both estimates, hence reducing bias.

We developed three logistic-regression models to adjust for severity of illness and case mix, one for each outcome in the 50% random or “development” sample. Candidate variables were selected if significant at the 0.05 level after univariate analysis for any of the three outcomes. DRG variables were grouped into DRG–principal procedure categories to produce more homogeneous risk groupings based on Haberman residuals 7,9,10and then included in each model. Each model included 42 DRG–principal procedure variables and 27 patient characteristics. A total of 37 interaction terms were included in the models, having been significant at the Bonferroni adjusted 0.05 level. We validated the derived models for the remaining 50% or “validation” sample. Coefficients were not statistically different between models derived in development and validation sets. Pearson correlation coefficients between predicted outcomes in the development set and the validation set were always greater than 0.93. Final models were constructed using both the development and the validation data sets.

To account for hospital characteristics that may have influenced our results, we adjusted the results using a list of 11 hospital characteristics that we, and others, reported previously. 7,11,12Further, we constructed an indicator variable for each hospital and report results adjusted for each individual hospital in the logistic-regression modeling. We also performed adjustments for each hospital using Mantel–Haenszel tests 13in a number of ways. We estimated the odds ratio (OR) associated with outcome and no direction by controlling for each hospital and stratified, in some analyses, using the risk of death or the propensity score 14–18to predict lack of direction. When stratifying using the risk of death, we refitted the mortality model, deriving new coefficients, using a separate data set of 1995–1996 Pennsylvania Medicare patients. This allowed for unbiased odds ratios derived from the Mantel–Haenszel tests when applied to the main study set comprising 1991–1994 data.

Table 4describes patient case mix and table 5displays patient characteristics that were present in at least 1% of the study population among the anesthesia directed and nondirected groups. Two odds ratios are presented in table 5. The first is the unadjusted odds ratio; the second is the Mantel–Haenszel 13odds ratio after adjusting for DRG category and each of the 245 hospitals in the study. Undirected patients were more likely to be male; to have a history of arrhythmia, congestive heart failure, and non–insulin-dependent diabetes; and to be admitted through the emergency department. Undirected patients were less likely to have cancer.

There were some associations between covariates and direction status that were unexpected. Some of these could be explained when we studied factors that were predictive of direction 14and factors predictive of procedures. For example, the unadjusted odds ratios in table 5suggest undirected cases had greater odds of occurrence in patients with insulin-dependent diabetes. However, undirected patients also had greater odds of undergoing wound debridement and skin grafts as a principal procedure, as compared with directed patients (OR = 10.14; 95% confidence interval [CI]= 8.31, 12.36). The higher rate of diabetes in the undirected group may, in part, have been caused by an increased propensity of the caregiver to perform skin graft procedures, and therefore it would not be surprising that there was an association between undirected cases and diabetes. Bickel et al.  19have shown the importance of such adjustments when making inferences concerning selection bias in graduate school admissions policies. Hence, after adjustment, it would appear as though there was far less imbalance in the covariates between directed and undirected cases than was initially appreciated. However, given the remaining differences between groups, careful severity corrections for all outcomes were performed before results could be accurately interpreted.

The distribution of hospital characteristics according to the presence of anesthesiologist direction is displayed in table 6. Generally, the hospitals in which undirected cases occurred tended to be smaller, to have less specialized technology and facilities, and were less likely to be involved with the teaching of medical students and residents.

Unadjusted death, complication and failure-to-rescue rates were greater when cases were undirected (table 7). Table 8displays the influence of anesthesia direction on outcome after results were adjusted for 64 patient characteristics and interaction terms, including demographic information, history variables, whether the patient was transferred from another short-term-care hospital, whether the patient was admitted from the emergency room, and 42 DRG—procedure categories used for this study. As in the unadjusted model, mortality and failure-to-rescue rates were greater when an anesthesiologist did not perform or direct care. The adjusted odds ratios for death and failure-to-rescue were significantly increased: (OR for death = 1.09, P < 0.021; OR for failure-to-rescue = 1.12, P < 0.003) corresponding to 2.8 excess deaths/1,000 patients and 8.4 excess deaths/1,000 patients with complications. Adding patient race to this model did not change these results.

A second analysis was performed adding admission MedisGroups (MediQual Inc., Westborough, MA) severity score (a physiologic based score) obtained from the Pennsylvania Health Care Cost Containment Council. 6,20–23During 1991–1994, MedisGroups scores were recorded for only 72.9% of our study patients. The ORs for the anesthesia direction covariate were as follows: (OR for death = 1.09, P < 0.016; OR for failure-to-rescue = 1.12, P < 0.002; OR for complication = 0.97, P < 0.052). These results provided further evidence that the models derived solely from the Medicare data were adequately adjusted.

We also explored whether the increased odds of death and failure-to-rescue in the undirected group were caused by admissions through the emergency department. When the non–emergency department cases were analyzed separately, the odds ratios for death and failure-to-rescue remained greater for those patients who did not receive anesthesiologist direction (adjusted OR for death = 1.17, P < 0.007 and adjusted OR for failure-to-rescue = 1.18, P < 0.005).

The lower portion of table 8displays the results of anesthesia direction when 11 hospital variables were included in the three outcomes models. Undirected cases were associated with greater death and failure-to-rescue rates: (OR for death = 1.08, P < 0.040; OR for failure-to-rescue = 1.10, P < 0.013), corresponding to 2.5 excess deaths/1,000 patients and 6.9 excess deaths/1,000 patients with complications, whereas the adjusted OR for the complication rate was insignificant (OR for complication 1.00, P < 0.796).When the MedisGroups severity score was added to the analysis, death and failure-to-rescue ORs were stable and the associated P  values became slightly more significant. When a variable reflecting the number of anesthesia procedures per hospital stay was added to the model, we again found the odds ratio estimates to be unchanged.

In a further analysis, we calculated the adjusted odds ratios for each outcome using the Mantel–Haenszel odds ratio, adjusting for all DRG categories and for each of the 245 hospitals in the study, and obtained very similar results. The adjusted odds ratio for death was 1.14 (P < 0.001), the odds ratio for failure-to-rescue was 1.11 (P < 0.008), and the odds ratio for complication was 1.06 (P < 0.001). We next constructed a model adjusting for the same patient characteristics as in table 8plus a hospital identifier variable for each hospital (grouping hospitals with fewer than 10 deaths into one indicator variable to allow for more stable coefficients). The results were almost identical to those in table 8. The adjusted odds ratio for death was 1.09 (P < 0.033), OR for failure-to-rescue was 1.10 (P < 0.016), and the OR for complication was 1.02 (P < 0.333).

We conducted an additional set of analyses concerning the influence of the hospital provider on outcome in this study. Using the full model for patient characteristics, as defined in table 8, we refitted the model coefficients for a separate set of 102,781 Pennsylvania Medicare patients from 1995 and 1996, using the same procedures as in the 1991–1994 study data set. We then calculated the initial risk of death before surgery for each patient in our 1991–1994 study data set and, as suggested by Cochran, 24we divided these risk scores at the quintiles of this distribution, yielding five risk groups of equal sample size. For each of the 245 hospitals in the data set, we then formed 245 × 5 = 1,225 cells using these five risk groups. This gave us a 2 × 2 × 5 × 245 contingency table, recording death by direction status by mortality risk strata by hospital. The associated Mantel–Haenszel odds ratio computed from the 2 × 2 × 5 × 245 cell contingency table was 1.16 (1.077, 1.246). This ratio was almost exactly the same as the Mantel–Haenszel test results with an odds ratio of 1.14, controlling for the individual hospital and DRG (see previous section in Results), whereas the logit model using hospital indicators also found a very similar odds ratio (1.09). Hence, we obtained almost identical results when the ORs were derived from regression models or derived by performing a Mantel–Haenszel analysis, controlling for risk of death, and forcing all comparisons to be stratified within the same hospital, thereby controlling for the “hospital effect.”

To control for selection bias associated with direction or lack of direction, we performed an additional set of analyses using the propensity score to predict direction. Similar to the stratification of mortality risk previously discussed, we divided the propensity score at the quintiles of its distribution, yielding five risk groups of equal sample size. For each of the 245 hospitals in the data set, we then formed a 2 (death status) × 2 (direction status) × 5 (propensity score risk strata) × 245 hospital contingency table. The associated Mantel–Haenszel odds ratio computed from the 2 × 2 × 5 × 245 cell contingency table was 1.11 (1.03, 1.19). Again, the odds ratio for death associated with direction status was almost identical to that determined by our previous methods using logit regression or methods without the propensity score.

Finally, we performed an adjustment stratifying by mortality risk, propensity score, and hospital using a 2 × 2 × 5 × 5 × 245 cell contingency table. Mortality risk was again estimated for the separate 1995–1996 patient population to avoid bias. This analysis yielded, again, similar results to the logit model reported in table 8, with an OR of 1.07, (0.99, 1.15). The slightly less significant P  value of 0.09 may reflect the fact that we were controlling for 5 times more strata than in the previous two analyses.

Table 9displays the results of the “fully adjusted patient model,” with the addition of all 11 hospital characteristics and the direction indicator for the three outcomes. For each hospital variable, and the anesthesiologist direction indicator, we present two results. The “marginal” result is computed by adjusting the OR for direction by all patient covariates and a single hospital variable or direction indicator. The “partial” analysis displays the results of a fully adjusted model using all patient covariates, all hospital covariates, plus the direction indicator (this “partial” model is also shown in table 8). The marginal analysis showed that hospitals with more sophisticated facilities, higher nurse staffing ratios, and more educational programs were consistently associated with reduced death and failure-to-rescue rates, whereas complication rates were greater in these hospitals. We reported this same pattern in other studies. 7,11,22Simultaneously adjusting for all the hospital variables and the anesthesiologist direction variable, we found that three factors continued to show independent effects on death and failure-to-rescue: hospital size, nurse-to-bed ratio, and direction by an anesthesiologist.

Furthermore, we asked whether the odds ratios associated with direction and outcome would have changed had we used only patients who were billed, rather than all records. The resulting logistic-regression derived odds ratios were unchanged. Finally, we asked whether adding variables denoting the size of the metropolitan area would account for the observed differences in outcome. Adjusting for the 11 hospital variables and for five levels of population size from rural to metropolitan areas greater than 1 million, we found very little difference in results (OR for death = 1.07, P < 0.057; OR for failure-to-rescue = 1.09, P < 0.021; OR for complication = 1.00, P < 0.853).

After adjustments for severity of illness and other confounding variables, we found higher mortality and failure-to-rescue rates for patients who underwent operations without medical direction by an anesthesiologist. Adjusted complication rates were not associated with medical direction. This finding is not inconsistent with the finding of higher mortality rates in the absence of medical direction. Our previous work showed that complication rates, as reflected in administrative claims data, are indicators of severity of illness, 7,11,22but adjusted complication rates are not well-correlated with adjusted death rates. 11,22,23In Medicare surgical patients, complication rates are poor indicators of quality of care 6,7and are not accurately coded to discern specific intraoperative events. The complication rate in this study reflects the number of patients who had complications, not the number of complications per patient. The complication list was developed to be inclusive and sensitive to most undesirable occurrences during the hospital stay, but was not specific for perioperative complications. Specific perioperative complications may not appear in the Medicare claims data, in which the limited number of fields and variation in recording patterns may prevent the complication rate from reflecting differences in quality. Hence, it is not surprising that adjusted complication rates were not different among providers, whereas 30-day mortality rate—a measure better defined and recorded—was different.

Because of these limitations in all studies involving the Medicare database, the failure-to-rescue rate was developed and validated, 6,7and complications were used as an adjustment tool for severity of illness, rather than as an isolated outcome measure. Failure-to-rescue assesses how complications are managed by studying the rate of death only in those patients in whom complications develop or in those who die without recorded complications. Failure-to-rescue may provide better insight regarding quality of care than either mortality or complication rates used alone 6,7because it can more easily account for differences in severity. For the current study, failure-to-rescue rates showed an even greater association with provider characteristics than did death rates. This suggests that advanced medical training may allow for better management of complications, thereby decreasing the severity of such complications, and leading to fewer subsequent deaths.

Adequate severity adjustment is always necessary for studies of the type reported herein. Given the apparent difference in the prevalence of specific comorbidities between the directed and undirected groups, adequate adjustment was especially important. As seen in table 5, much of the difference between groups could be explained by the different distribution of procedures found in the directed and undirected groups. Hence, looking at unadjusted prevalence rates of comorbidities can be deceiving in data sets such as this. A classic example of this same problem was provided by Bickel et al.  19in their 1975 article of graduate admission bias using data from The University of California at Berkeley. Although unadjusted admission acceptance rates would suggest females had been discriminated against because of the observed overall lower admission rates, after adjustment for the departments to which the female students applied, it was shown that there was no significant bias. This was because the female applicants more often applied to departments with lower rates of acceptance (for both males and females), whereas male applicants more often applied to departments with higher rates of acceptance (for both males and females). Hence, the overall, unadjusted numbers suggested an imbalance in admission rates (a bias against females), whereas such an imbalance was not seen at the individual department level.

It was reassuring that, in our study, after adjustment for DRG and hospital, the difference in the prevalence of covariates between the directed and undirected groups became much smaller. In part, this was caused by a tendency for undirected patients to be involved with slightly more minor procedures in patients with a greater number of comorbidities. Although adjustments in table 5helped to explain these differences in comorbidity rates among groups, more complete model-based adjustments were made when reporting final results.

There is strong supporting evidence that the model-based adjustments used in our study were adequate. Of interest, unadjusted rates of death, number of complications, and failure-to-rescue rates were all increased in the nondirected group. After using models that contained identical patient covariates for each of the three outcomes, we observed that the adjusted odds of development of complications decreased to 1, whereas ORs of death and failure-to-rescue remained greater than 1. Further, the unadjusted OR associated with no direction and failure-to-rescue (table 7) was almost identical to that in the fully adjusted model (table 8). This finding is consistent with a number of studies showing that a strength of the failure-to-rescue concept is that the failure-to-rescue rate appears to be less sensitive to omissions of severity of illness data than is the death or complication rate. 7,22Finally, when a physiologic severity adjustment measure, MedisGroups Score, was added to the models, results were virtually unchanged. If the association between anesthesiologist direction and outcome was an artifact of failure of the model to adequately control for critical aspects of patient severity, we would have expected the addition of the physiologic-based patient severity score to alter the results. Together, these findings provide consistent supporting evidence that the model was adjusted adequately for severity of illness among groups.

Without further adjustment, these results might still reflect differences in overall hospital quality, rather than differences in the type of anesthesiologist involvement. Therefore, the results were simultaneously adjusted for patient and hospital characteristics, yet the effect of anesthesiologist direction remained significant. When we adjusted for the individual hospital using Mantel–Haenszel adjustments and logistic-regression models, our results were unchanged. Further, adjustments for selection bias using the propensity score again revealed that our results were very stable. It appeared that the increased risk of death associated with lack of direction was not caused by selection bias at the hospital. Thus, these data support the concept that there is a benefit associated with medical direction by an anesthesiologist that is independent of the hospital effect and not a result of selection bias.

Our results were consistent with other large studies of anesthesia outcomes. 25,26Some studies suggest that the best outcomes may occur when anesthesia is provided by an anesthesia care team directed by an anesthesiologist. 27We also found that the single most important hospital variable associated with lower death and failure-to-rescue rates was a higher registered-nurse-to-bed ratio, 7and the importance of nurse staffing has been noted in several other studies. 7,28–30 

Our results also point to a common misconception when assessing anesthesia safety. Since the early (1954) study of Beecher and Todd 31reported an anesthesia-related mortality rate of 1 death/1,560 patients, anesthesia-related mortality has been the gold standard of gauging anesthesia safety. By 1982, the anesthesia-related mortality had decreased to 1 death/6,789 patients in the United Kingdom, 32and, by 1989, the anesthesia mortality rate had decreased to 1 death/185,056 patients 33; whereas Eichhorn, 34in 1989, reported anesthesia-related mortality of 1 death/151,400 patients among more than 750,000 healthy (American Society of Anesthesiologists physical status I or II) 35patients in the United States. These studies supported the concept that the incidence of death directly related to anesthetic events had decreased, but the concept of anesthesia-related mortality was narrowly defined. Modern perioperative intensive care (including that provided by anesthesiologists) often prevents immediate postoperative mortality, yet prolonged morbidity and delayed mortality may result even when the precipitating event occurred preoperatively or intraoperatively. Further, there is increasing evidence that anesthetic practice influences subsequent patient outcomes in ways that were not recognized previously. Even relatively simple measures, such asmaintaining normothermia or supplying supplemental oxygen in the perioperative period, can decrease the incidence of subsequent morbid events, including perioperative cardiac morbidity (ischemia, infarction, cardiac arrest), 36and postoperative wound infection. 37,38Our study underscores the importance of anesthetic practice in overall surgical outcome, potentially influencing mortality at the rate of 2.5 deaths/1,000 patients or 1 death/400 patients, more than 300 times greater than reported by Eichhorn 34and others, 32,33who used a far more narrow definition of “anesthesia related” that did not consider these wider associations.

This was a retrospective analysis based on administrative claims data and is limited by the associated errors inherent in using such data. The accuracy of our definitions for anesthesiologist direction (or no direction) is only as reliable as the bills (or lack of bills) submitted by caregivers. We also cannot rule out the possibility that unobserved factors leading to undirected cases were associated with poor hospital support for the undirected anesthetist and patient. Local, temporal, even psychologic factors may play a part in patient outcome, and such factors may not be noted in the available data set. For example, if anesthesiologists had a tendency not to submit bills for patients who died within 30 days of admission, our results could be skewed in favor of directed cases. Although our clinical experience suggests that this scenario is quite unlikely, we cannot rule out this possibility. We also cannot rule out the possibility that undirected cases occur more often in emergency situations that developed outside of the emergency department. For example, it may be that patients who required multiple anesthesia procedures were more ill and were cared for by an undirected anesthetist because of an emergency reoperation that did not allow time for the anesthesiologist to participate in care. Although we could find no evidence of this, because our study results were unchanged when a variable denoting multiple anesthesia procedures was added to the model, more extensive study involving individual chart review may be helpful for exploring these questions.

Future work will also be needed to determine whether the mortality differences in this report were caused by differences in the quality of direction among providers, the presence or absence of direction itself, or a combination of these effects. To address these limitations, we hope to pursue in-depth, large-scale medical chart review of surgical cases in the next phase of this research. We anticipate that review of medical charts will provide more detailed information that will assist in determining the etiology of differences in outcomes among provider type.

In summary, review of Medicare claims data in Pennsylvania suggests that medical direction by an anesthesiologist was associated with lower mortality and failure-to-rescue rates. In light of the large numbers of Medicare patients undergoing operations each day, future research must carefully identify the etiologic factors associated with these findings to define optimal provider models and improve outcomes.