We appreciate the thoughtful attention that Dr. Hofer and colleagues have given our article describing the advantages and disadvantages of reliability, or “shrinkage” adjustment.1 Their title, “Current Quality Registries Lack the Accurate Data Needed to Perform Adequate Reliability Adjustments” may be accurate for the anesthesia data collections they mention but not for all surgical quality registries.
Dr. Hofer and colleagues’ message about the importance of measurement error cannot be understated. Measurement error, whether in administrative data or in registries, undermines both the validity and utility of quality measurement. When reliability adjustment is applied, unmeasured patient and case-mix factors leave “residual” variation that may be falsely attributed to hospitals or physicians rather than inadequate risk adjustment.2 With or without reliability adjustment, measurement error is critical when benchmarking quality across hospitals or physicians because federal mandates are linking payment to outcome-based performance measurement.
Physicians and hospital leaders already appreciate that meaningful outcomes comparisons are very costly to produce, particularly when accrued through a clinical registry and analyzed with the necessary statistical expertise. The first question is whether physicians believe that meaningful outcomes comparisons are important enough to pay for them. The American College of Surgeons (ACS) and the Society of Thoracic Surgeons (STS) staked out their positions on this issue decades ago and currently generate the highest-quality outcomes data in surgery while stewarding multiple measures in the National Quality Forum.3–5 Some may wonder why anesthesiologists have not taken a similar leadership position.6 However, it is important to consider that (1) participation in these registries is costly, (2) neither the ACS nor STS registry outcomes are part of current or proposed Centers for Medicare and Medicaid Services payment programs, and (3) the jury is still out on whether participation in ACS or STS registries improves quality.7,8
So how can anesthesiologists improve the quality of quality measurement? This is crucial because mandated links between payment and “performance” are moving forward with or (more commonly) without high-quality measurement science. Solutions are many: investing in anesthesia registries, fostering partnerships with surgeons to share the costs of registries, and uniting with surgeons and nurses for a stronger political voice. In brief, anesthesiologists must either “pony up” the financial and leadership costs of performance measurement or risk being left in the dust.
The authors declare no competing interests.