To the Editor:

Peyton and Wu1  have recently published a systematic review with meta-analysis that identified a relationship between the risk of postoperative nausea and vomiting within the first 24 h and the duration of the intraoperative exposure to nitrous oxide. Using data from over 10,000 patients in 29 randomized controlled trials of nitrous oxide administration (Peyton and Wu1 ; table 1), a meta-regression analysis found a statistically significant linear relationship between the increasing log risk ratio (RR) and increasing anesthesia/surgery duration with an estimated increased RR of 20% per hour of exposure after the first 45 min. They also reported a pairwise meta-analysis showing an increased RR of postoperative nausea and vomiting for nitrous oxide regardless of duration. I raise two concerns about their statistical analysis: (1) a failure to identify the claimed relationship using other meta-analytic models; and (2) the possibility of ecological (aggregation) bias.

Table 1.

Meta-analysis/Meta-regression of Nitrous Oxide Duration and PONV

Meta-analysis/Meta-regression of Nitrous Oxide Duration and PONV
Meta-analysis/Meta-regression of Nitrous Oxide Duration and PONV

Peyton and Wu performed their meta-analysis/meta-regression using the statistical software STATA 12 choosing a random effects analysis with inverse variance weighting (DerSimonian and Laird method of moments). Their original data were subjected to a secondary analysis using four alternative statistical approaches in the R statistical platform—namely (1) generalized linear mixed effects regression (conditional exact likelihood),2  (2) linear mixed effects regression (restricted maximum likelihood estimation),3  (3) higher order likelihood regression estimation,4  and (4) empirical Bayesian regression (exact posterior inference).5  All analyses were random effects or mixed effects models because of the considerable statistical heterogeneity; both RR and odds ratio effect sizes were used. Both a pairwise analysis and when possible a meta-regression was estimated by each statistical approach. Partial results are shown in table 1 with display of the effect size, 95% CI and 95% prediction interval. The 95% prediction interval estimated where 95% of true outcomes will fall in future studies. For meta-regression models, the effect size was predicted at 120 min. Full details of the data set, the statistical software, the function calls, and the statistical output are presented in Supplemental Digital Content 1, http://links.lww.com/ALN/B97.

No regression coefficient in any model reached statistical significance. For the pairwise comparisons, the RR effect sizes were 1.14 with the lower bounds of the 95% CI approaching the line of identity (table 1), similarly for the odds ratio effect size. For methods offering prediction intervals, future studies will have a wide range of values. For the regression predictions of the risk of postoperative nausea and vomiting at 120 min, effect sizes ranged from 1.02 to 1.19; the 95% CI extended widely across the line of identity. A sensitivity analysis of meta-analytic results using other statistical models has been recommended to explore the vulnerability of estimates to inherent assumptions.6  In the data of Peyton and Wu, any effect of nitrous oxide is not sufficiently robust to show clear evidence of effect in alternative statistical methods.

Peyton and Wu used a study level covariate in their meta-regression. Specifically, RR was regressed on the mean or median anesthesia/surgery time for all patients reported in each randomized controlled trial. Thus, the average of the sample (nitrous oxide exposure in an randomized controlled trial) was assumed to provide inferences about the likelihood of the response of a patient at an individual level. Using a study-level covariate is of course acceptable when patients are randomly assigned to receive or not receive nitrous oxide. But patients were not randomized to receive nitrous oxide for varying durations—a function of the surgical procedure. This duration could vary widely in studies. For example, in one of the largest studies (Myles et al.),7  the median anesthesia duration was 3.1 h with interquartile range (2.3 to 4.6); 25% of patients were exposed for less than 2.3 h and 25% were exposed for more than 4.6 h.

Stated formally, the correlation of aggregate quantities is not equal to the correlation of individual quantities. It is this assumption that constitutes the ecological fallacy/bias. This is not a hypothetical risk; “ecological bias rears its ugly head” in meta-regression.8  Only by using individual patient data in meta-analysis and meta-regression can the possibility of ecological bias be excluded. Using individual patient data, it has been demonstrated that ecological bias can either conceal a real treatment interaction8  or mistakenly indicate a clinically important treatment effect.9 

There are clearly data to question the benevolence of nitrous oxide. But the case against nitrous oxide remains incompletely proved.

Competing Interests

The author declares no competing interests.

References

1.
Peyton
PJ
,
Wu
CY
:
Nitrous oxide-related postoperative nausea and vomiting depends on duration of exposure.
Anesthesiology
2014
;
120
:
1137
45
2.
Stijnen
T
,
Hamza
TH
,
Ozdemir
P
:
Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data.
Stat Med
2010
;
29
:
3046
67
3.
Viechtbauer
W
:
Conducting meta-analyses in R with the metafor package.
J Stat Softw
2010
;
36
:
1
48
4.
Guolo
A
,
Varin
C
:
The R package metaLik for likelihood inference in meta-analysis.
J Stat Softw
2012
;
50
:
1
14
5.
Luo
S
,
Chen
Y
,
Su
X
,
Chu
H
:
mmeta: An R package for multivariate meta-analysis.
J Stat Softw
2014
;
56
:
11
6.
Liberati
A
,
Altman
DG
,
Tetzlaff
J
,
Mulrow
C
,
Gøtzsche
PC
,
Ioannidis
JP
,
Clarke
M
,
Devereaux
PJ
,
Kleijnen
J
,
Moher
D
:
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: Explanation and elaboration.
BMJ
2009
;
339
:
b2700
7.
Myles
PS
,
Leslie
K
,
Chan
MT
,
Forbes
A
,
Paech
MJ
,
Peyton
P
,
Silbert
BS
,
Pascoe
E
;
ENIGMA Trial Group
:
Avoidance of nitrous oxide for patients undergoing major surgery: A randomized controlled trial.
Anesthesiology
2007
;
107
:
221
31
8.
Berlin
JA
,
Santanna
J
,
Schmid
CH
,
Szczech
LA
,
Feldman
HI
;
Anti-Lymphocyte Antibody Induction Therapy Study Group
:
Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: Ecological bias rears its ugly head.
Stat Med
2002
;
21
:
371
87
9.
Riley
RD
,
Lambert
PC
,
Abo-Zaid
G
:
Meta-analysis of individual participant data: Rationale, conduct, and reporting.
BMJ
2010
;
340
:
c221