“Although the possibility of continuously monitoring hemoglobin remains attr-active, particularly in actively bleeding patients, some progress has to be made by the manufacturers to improve the accuracy of the devices until we can safely use them in clinical practice.” 

IN patients with active bleeding during surgery, anesthesiologists are required to monitor hemodynamics to guide volume resuscitation1and to monitor hemoglobin levels to assess bleeding, maintain adequate oxygen and substrate delivery, and guide blood-transfusion decisions. Rapid and accurate intraoperative measurement of hemoglobin concentration is therefore essential. The reference method (cooximetry in the laboratory) requires venous or arterial blood sampling and is associated with an unavoidable delay (time for blood sampling, time for transport to the laboratory, time for biological measurement and its validation, time for the information to reach the physician). Therefore, the anesthesiologist usually uses a portable cooximeter and a single drop of blood (capillary-based method) to determine the hemoglobin level at the bedside, and its accuracy is considered appropriate. For quality control reasons, point-of-care devices linked to the central laboratories may also be used. However, now new devices are available, which enable continuous (or semi-continuous) monitoring of hemoglobin, using various methodologies. In this issue of ANESTHESIOLOGY, Coquin et al .2assessed one of them (NBM-200MPTM, Orsense, Nes Ziona, Israel) in a prospective equivalence study in critically ill patients who were admitted for gastroinstestinal bleeding. This innovative device combines low-perfusion cooximetry and occlusive spectroscopy. The study was prematurely interrupted after an interim analysis because the accuracy of the new device was significantly inferior to that of the capillary-based method, as compared with the reference method: the proportion of inaccurate measurements using this new device increased markedly (47 vs.  24%, P < 0.001) and should have been associated with an increased incidence of failed transfusions. The use of vasopressor agents did not significantly interfere with the accuracy. These negative results are very similar to those reported last year in ANESTHESIOLOGY and which used another technology, that is, pulse cooximetry (Radical-7 pulse CO-Oximeter, Massimo Corp., Irvine, CA).3These last results obtained in surgical patients were confirmed in critically ill patients with gastroinstestinal bleeding.4 

Although the possibility of continuously monitoring hemoglobin remains attractive, particularly in actively bleeding patients, some progress has to be made by the manufacturers to improve the accuracy of the devices until we can safely use them in clinical practice. It is important to consider that a new technique for the bedside measurement of hemoglobin should demonstrate that it is either superior to that one we use daily (e.g. , HemoCue®, Cypress, California) or at least equivalent if another advantage can be demonstrated, such as a continuous measurement. However, even a continuous measurement may not be such a clear advantage. Just remember the ongoing debate concerning the usefulness of continuous measurements of arterial blood gases or glycemia.5It is amazing that Rice et al .5indicated that “(continuous glucose monitor is) more of a direction of change than an absolute blood glucose monitor” because the “Bland and Altman plots will not suffice in substantiating accuracy.” Just replace “continuous glucose monitor” by “continuous hemoglobin monitor” and you reach the conclusion of Coquin et al.  2study. In the near future, we need to answer the following questions: (1) Do we need continuous hemoglobin monitoring? The answer is probably yes, but for which patients? (2) What price are we ready to pay to obtain trend information (decreased accuracy, increased cost)? Considering this last question, Coquin et al.  2performed a very interesting complementary analysis. Besides the accuracy of the monitor, they also studied the proportion of failed or inappropriate transfusions that should have resulted from the use of the device, using different hemoglobin level targets. This methodological approach is interesting, although it may provide less definitive conclusions as compared with a randomized study comparing two groups of patients, one with the device and the other without it.

These method comparison studies 2–4also raise some important issues concerning the methodological approach. It is obvious that we need high-quality studies to test these devices appropriately as Coquin et al.  2did. The Bland and Altman 6technique is now widely recognized as the appropriate method, although these articles still contain some correlation diagrams, which should probably be definitely abandoned. It should also be pointed out that more sophisticated statistical analyses are now routinely used to take into account the fact that most of these studies perform repeated measurements in the same patient.7 

However, these recent studies 2–4are landmarks that indicate an important progress in the way we are designing and reporting method comparison studies. In the modern era of clinical research, a unique and quantified hypothesis should be tested. Thus, it is mandatory to calculate the number of patients required a priori , according to that primary endpoint. Unfortunately, until recently, most method comparison studies did not fulfil these quality criteria (table 1), probably because available guidelines did not provide sufficient recommendations concerning this important methodological issue.8International guidelines such as the Consolidated Standards of Reporting Trials recommendations 9for randomized trials or the Standards of Reporting of Diagnostic Accuracy initiative10for diagnostic studies have indeed led to significant improvements after they were implemented in high-impact medical journals.11This means that from now on, authors should clearly state the type of their study (superiority, equivalence, noninferiority) and their primary endpoint, and the statistical analysis plan should be decided on before the onset of the study (table 1). Several methods can be used to calculate the number of patients needed. On his Web site, Martin Bland*proposes a method based on the estimation of the confidence interval of the limits of agreement. Coquin et al .2used the proportion of outliers to calculate the number of patients needed—an outlier being defined as the clinically unacceptable difference between the tested method and the reference method. Failed measurements—that is, when the apparatus does not provide any measure—may represent a noticeable proportion in some studies,2–4and one of the main interests of looking at outliers is that it includes failed measurements in the analysis. But a method must be chosen, whatever it is.

Table 1. Main Quality Criteria for a Method Comparison Study

Table 1. Main Quality Criteria for a Method Comparison Study
Table 1. Main Quality Criteria for a Method Comparison Study

Medical devices need the same scrutiny as drugs do, and leading journals must maintain a high level of methodology for the articles they publish. The international biostatistician community should provide us with recommendations concerning method comparison studies, as they did in the Consolidated Standards of Reporting Trials or Standards of Reporting of Diagnostic Accuracy recommendations.9,10In the meantime, authors who wish to submit their articles to ANESTHESIOLOGY should make sure that it fulfils the simple but important quality criteria listed in table 1, as Coquin et al.  2did.

Department of Emergency Medicine and Surgery, CHU Pitié-Salpêtrière, Assistance Publique-Hôpitaux de Paris, Paris, France. bruno.riou@psl.aphp.fr

1.
Cannesson M, Le Manach Y, Hofer CK, Goarin JP, Lehot JJ, Vallet B, Tavernier B. Assessing the diagnostic accuracy of pulse pressure variations for the prediction of fluid responsiveness: A “gray zone” approach. Anesthesiology. 2011;115:231–41
2.
Coquin J, Bertarrex A, Dewitte A, Lefèvre L, Joannes-Boyau O, Fleureau C, Winnock S, Leuillet S, Janvier G, Ouattara A. Accuracy of determining hemoglobin level using occlusion spectroscopy in patients with severe gastroinstestinal bleeds. Anesthesiology. 2013;118:640–8
3.
Lamhaut L, Apriotesei R, Combes X, Lejay M, Carli P, Vivien B. Comparison of the accuracy of noninvasive hemoglobin monitoring by spectrophotometry (SpHb) and HemoCue® with automated laboratory hemoglobin measurement. Anesthesiology. 2011;115:548–54
4.
Coquin J, Dewitte A, Manach YL, Caujolle M, Joannes-Boyau O, Fleureau C, Janvier G, Ouattara A. Precision of noninvasive hemoglobin-level measurement by pulse co-oximetry in patients admitted to intensive care units for severe gastrointestinal bleeds. Crit Care Med. 2012;40:2576–82
5.
Rice MJ, Coursin DB. Continuous measurement of glucose: Facts and challenges. Anesthesiology. 2012;116:199–204
6.
Bland JM, Altman DG. Agreed statistics: Measurement method comparison. Anesthesiology. 2012;116:182–5
7.
Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. 2007;17:571–82
8.
Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A, Roberts C, Shoukri M, Streiner DL. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64:96–106
9.
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang TCONSORT GROUP (Consolidated Standards of Reporting Trials). . The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Ann Intern Med. 2001;134:663–94
10.
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Moher D, Rennie D, de Vet HC, Lijmer JGStandards for Reporting of Diagnostic Accuracy. . The STARD statement for reporting studies of diagnostic accuracy: Explanation and elaboration. Ann Intern Med. 2003;138:W1–12
11.
Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: Interrupted time series analysis. BMJ. 2012;344:e4178