We appreciate the thoughtful attention that Dr. Hofer and colleagues have given our article describing the advantages and disadvantages of reliability, or “shrinkage” adjustment.1  Their title, “Current Quality Registries Lack the Accurate Data Needed to Perform Adequate Reliability Adjustments” may be accurate for the anesthesia data collections they mention but not for all surgical quality registries.

Dr. Hofer and colleagues’ message about the importance of measurement error cannot be understated. Measurement error, whether in administrative data or in registries, undermines both the validity and utility of quality measurement. When reliability adjustment is applied, unmeasured patient and case-mix factors leave “residual” variation that may be falsely attributed to hospitals or physicians rather than inadequate risk adjustment.2  With or without reliability adjustment, measurement error is critical when benchmarking quality across hospitals or physicians because federal mandates are linking payment to outcome-based performance measurement.

Physicians and hospital leaders already appreciate that meaningful outcomes comparisons are very costly to produce, particularly when accrued through a clinical registry and analyzed with the necessary statistical expertise. The first question is whether physicians believe that meaningful outcomes comparisons are important enough to pay for them. The American College of Surgeons (ACS) and the Society of Thoracic Surgeons (STS) staked out their positions on this issue decades ago and currently generate the highest-quality outcomes data in surgery while stewarding multiple measures in the National Quality Forum.3–5  Some may wonder why anesthesiologists have not taken a similar leadership position.6  However, it is important to consider that (1) participation in these registries is costly, (2) neither the ACS nor STS registry outcomes are part of current or proposed Centers for Medicare and Medicaid Services payment programs, and (3) the jury is still out on whether participation in ACS or STS registries improves quality.7,8 

So how can anesthesiologists improve the quality of quality measurement? This is crucial because mandated links between payment and “performance” are moving forward with or (more commonly) without high-quality measurement science. Solutions are many: investing in anesthesia registries, fostering partnerships with surgeons to share the costs of registries, and uniting with surgeons and nurses for a stronger political voice. In brief, anesthesiologists must either “pony up” the financial and leadership costs of performance measurement or risk being left in the dust.

Competing Interests

The authors declare no competing interests.

References

1.
Wakeam
E
,
Hyder
JA
:
Reliability of reliability adjustment for quality improvement and value-based payment.
Anesthesiology
2016
;
124
:
16
8
2.
Lawson
EH
,
Ko
CY
,
Adams
JL
,
Chow
WB
,
Hall
BL
:
Reliability of evaluating hospital quality by colorectal surgical site infection type.
Ann Surg
2013
;
258
:
994
1000
3.
Shahian
DM
,
Jacobs
JP
,
Edwards
FH
,
Brennan
JM
,
Dokholyan
RS
,
Prager
RL
,
Wright
CD
,
Peterson
ED
,
McDonald
DE
,
Grover
FL
:
The society of thoracic surgeons national database.
Heart
2013
;
99
:
1494
501
4.
Cohen
ME
,
Ko
CY
,
Bilimoria
KY
, et al:
Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus.
J Am Coll Surg
2013
;
217
:
336
46 e331
5.
Hyder
JA
,
Roy
N
,
Wakeam
E
,
Hernandez
R
,
Kim
SP
,
Bader
AM
,
Cima
RR
,
Nguyen
LL
:
Performance measurement in surgery through the National Quality Forum.
J Am Coll Surg
2014
;
219
:
1037
46
6.
Hyder
JA
,
Niconchuk
J
,
Glance
LG
,
Neuman
MD
,
Cima
RR
,
Dutton
RP
,
Nguyen
LL
,
Fleisher
LA
,
Bader
AM
:
What can the national quality forum tell us about performance measurement in anesthesiology?
Anesth Analg
2015
;
120
:
440
8
7.
Osborne
NH
,
Nicholas
LH
,
Ryan
AM
,
Thumma
JR
,
Dimick
JB
:
Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries.
JAMA
2015
;
313
:
496
504
8.
Etzioni
DA
,
Wasif
N
,
Dueck
AC
,
Cima
RR
,
Hohmann
SF
,
Naessens
JM
,
Mathur
AK
,
Habermann
EB
:
Association of hospital participation in a surgical outcomes monitoring program with inpatient complications and mortality.
JAMA
2015
;
313
:
505
11