To the Editor:— Studies that use propensity analysis, like the study of Vincent et al. ,1should not be perceived to be inferior to the gold standard of prospective randomized studies. Rather, propensity analysis and prospective randomized studies should be interpreted as complementary methods for finding the truth. Despite Nuttall and Houle’s assertion that randomized controlled studies, unlike propensity analyses, do not have “the limitation that remaining unmeasured confounding variables may still be present,”2both measured and unmeasured confounding variables may still be present. Randomized studies rely on the assumption (or hope) that these variables will be equally distributed between the groups. Who the anesthesiologist is or who harvests the saphenous vein may have a profound effect on outcome after cardiac surgery,3,4but random studies involving cardiac surgery rarely stratify by these factors or even measure them. Even small differences between groups in measured variables in randomized trials may lead to erroneous statistically significant outcomes.5
Prospective randomized studies may be limited by the inability to randomize for important variables. In evaluating an intervention, such as activated protein C on mortality of intensive care unit patients, it is necessary that nonrandom but important factors, such as which intensive care unit treats the patient, be controlled. Typically, this is done with severity scores such as the Acute Physiology and Chronic Health Evaluation and Mortality Probability Model. Although the word propensity is not used to describe the Acute Physiology and Chronic Health Evaluation or Mortality Probability Model, these scores are the likelihood or the propensity that a patient will die, and these scores are then included (the same as a propensity score determining the likelihood of receiving a transfusion would be included in a study of blood transfusion and sepsis1) in the analysis to partially control for some of the confounders in the randomized controlled trial.
Another limitation of randomized controlled trials is their lack of generalizability. In determining the benefits or harm of transfusion, Hébert et al. 6evaluated 6,451 persons to randomize 838 subjects (13%); 5,613 patients were excluded from their study. Physician belief in equipoise, the patient’s or family’s beliefs, or excluding patients based on age or comorbidities may produce nonrepresentative populations in randomized trials and severely limit the generalizability of the results.7–9In addition, crossover of subjects from one arm to the other arm of the trial or subject withdrawal may make the results hard to interpret.
Observational studies are not necessarily inferior to randomized studies. Both have advantages and disadvantages. Observational studies should be encouraged as a complement to randomized studies. They include a greater variety of patients, many of whom would be excluded by randomized studies, and can be performed for a small fraction of the cost. Sophisticated and innovative statistical techniques, such as multivariable analysis, propensity, and instrumental variables10should be used to help separate gold from fool’s gold.
University of Toledo College of Medicine, St. Vincent Mercy Medical Center, Toledo, Ohio. engoren@pol.net
