Background

Heterogeneity among reported outcomes from enhanced recovery after cesarean delivery impact studies is high. This study aimed to develop a standardized enhanced recovery core outcome set for use in future enhanced recovery after cesarean delivery studies.

Methods

An international consensus study involving physicians, patients, and a director of midwifery and nursing services was conducted using a three-round modified Delphi approach (two rounds of electronic questionnaires and a third-round e-discussion) to produce the core outcome set. An initial list of outcomes was based on a previously published systematic review. Consensus was obtained for the final core outcome set, including definitions for key terms and preferred units of measurement. Strong consensus was defined as 70% or greater agreement and weak consensus as 50 to 69% agreement. Of the 64 stakeholders who were approached, 32 agreed to participate. All 32, 31, and 26 stakeholders completed Rounds 1, 2 and 3, respectively.

Results

The number of outcomes in the final core outcome set was reduced from 98 to 15. Strong consensus (70% or greater stakeholder agreement) was achieved for 15 outcomes. The core outcome set included length of hospital stay; compliance with enhanced recovery protocol; maternal morbidity (hospital re-admissions or unplanned consultations); provision of optimal analgesia (maternal satisfaction, compliance with analgesia, opioid consumption or requirement and incidence of nausea or vomiting); fasting times; breastfeeding success; and times to mobilization and urinary catheter removal. The Obstetric Quality of Recovery-10 item composite measure was also included in the final core outcome set. Areas identified as requiring further research included readiness for discharge and analysis of cost savings.

Conclusions

Results from an international consensus to develop a core outcome set for enhanced recovery after cesarean delivery are presented. These are outcomes that could be considered when designing future enhanced recovery studies.

Editor’s Perspective
What We Already Know about This Topic
  • Enhanced recovery after cesarean delivery is potentially beneficial.

  • Heterogeneity among reported outcomes from enhanced recovery after cesarean delivery is significant. There has been a lack of consensus on the core outcomes to assess and compare values of enhanced recovery after cesarean delivery.

What This Article Tells Us That Is New
  • This study provides an international consensus of experts in multi-subspecialties and parturients on a core outcome set for enhanced recovery after cesarean delivery.

  • The core outcome set could be used to design future enhanced recovery studies, and assess and compare its value.

Cesarean delivery is now the most commonly performed inpatient surgery worldwide, and clinical and research efforts to optimize postpartum recovery are ongoing.1–4  Enhanced recovery after cesarean delivery protocols are being increasingly utilized within obstetric units in an effort to improve hospital efficiency, maternal quality of recovery, patient experience, and maternal satisfaction. The popularity of enhanced recovery after cesarean delivery protocols has become evident from the numerous guidelines and consensus statements that have recently been developed and published by professional societies endorsing its use, including the Society of Obstetric Anesthesia and Perinatology (Lexington, Kentucky), the Enhanced Recovery after Surgery Society (Stockholm, Sweden), and the American College of Obstetrics and Gynecology (Washington, D.C.).5–7  While enhanced recovery after cesarean delivery protocols have been shown to reduce hospital length of stay, maternal opioid consumption, times to mobilization, and urinary catheter removal, without compromising hospital readmission rates,8,9  there remains no consensus regarding optimal or fundamental metrics that clinicians and researchers can use to evaluate enhanced recovery after cesarean delivery protocols.

A recent systematic review of enhanced recovery after cesarean delivery studies identified significant heterogeneity in terms of reported enhanced recovery after cesarean delivery outcomes.8  Although there have been numerous studies evaluating the impact of enhanced recovery after cesarean delivery protocols, inconsistencies relating to outcomes used make them difficult to compare and limit the value of the available published research. The absence of consensus regarding how to assess the impact of enhanced recovery after cesarean delivery protocol adversely affects the interpretation and synthesis of data, slowing the progress and standardization of clinical care and research related to enhanced recovery after cesarean delivery. The lack of clarity due to heterogeneous reporting of results may also adversely influence the widespread adoption of enhanced recovery after cesarean delivery and therefore impact the quality of patient care delivered. In addition to the clinical benefits of standardizing outcome measures for audit, quality improvement, and benchmarking purposes, there is a need to standardize the endpoints used across research to enable improved comparison and combination of the results from diverse studies.10  The Delphi methodology has become a well-established method to harness expert opinion surrounding a topic in order to achieve consensus within an expert group.

The aim of this study was to seek expert and patient stakeholder consensus using Delphi methodology to develop a standardized enhanced recovery after cesarean delivery core outcome set that could be applied to future research and implementation studies evaluating the impact of enhanced recovery after cesarean delivery protocols.

An international consensus study was performed to develop a core enhanced recovery after cesarean delivery outcome set. This Delphi study was registered on the Core Outcome Measures in Effectiveness Trials initiative database on November 13, 2020 (https://www.comet-initiative.org/Studies/Details/1728) and follows the Core Outcome Set-Standards for Development and Core Outcome Set-Standards for Reporting recommendations.11,12  This study received institutional review board exemption from Stanford University (Stanford, California; No. 54128). The Delphi process involves an iterative process of multiple rounds, and includes generation of long lists, feedback, and voting.13–16  Modified Delphi methodology includes at least two rounds of electronic questionnaires followed by a final roundtable discussion and ratification round.15  This study follows the modified Delphi approach (two rounds of electronic questionnaires and an electronic roundtable discussion). The study was conducted by an Executive Committee (P.S., C.F.W., K.E., R.G., P.P., B.C.) who conceived, designed, and executed the study and a panel of stakeholders consisting of enhanced recovery experts and patients.

Scope

This project aimed to achieve consensus on (1) a core outcome set to evaluate enhanced recovery after cesarean delivery impact in future studies, (2) definitions of terms utilized within selected core outcomes (for example, length of hospital stay defined as time in hours from postanesthesia care unit admission to hospital discharge vs. delivery to hospital discharge), (3) optimal timing of measure (for example, postpartum opioid usage defined as opioid consumption during inpatient hospital stay vs. between 0 to 24, 0 to 48, or 0 to 72 h), and (4) optimal units to evaluate selected outcomes (for example, maternal satisfaction assessment using visual analog scale vs. numerical reporting scale or a Likert scale). For the purposes of this study, an enhanced recovery protocol was defined as a bundle of interventions implemented by a multidisciplinary team involving at least one intervention in the pre-, intra- and postoperative periods, which is implemented with the aim of improving patient recovery experience and the overall quality of care. Therefore, the core outcome set developed in this study would apply to all interventions implemented in the peripartum period (before, during, and after cesarean delivery).

Literature Search to Identify Enhanced Recovery after Cesarean Delivery Experts

Articles related to enhanced recovery after cesarean delivery were identified using a previously published search strategy.8  Literature searches of four databases (PubMed, CINAHL, Web of Science, and Embase) were performed on August 27, 2019, and repeated on June 19, 2020, and October 7, 2020, with no date limiters in order to identify published articles with corresponding authors that might be interested in participating in this Delphi study. The search strategy is provided in Supplemental Digital Content File 1 (http://links.lww.com/ALN/C861). Supplementary figure 1 (http://links.lww.com/ALN/C861) provides a summary of the number of enhanced recovery after cesarean delivery related studies identified from the search, and a list of the included references is provided in Supplemental Digital Content File 2 (http://links.lww.com/ALN/C861). Studies that implemented an “enhanced recovery protocol” as defined by study authors and compared recovery outcomes with and without enhanced recovery after cesarean delivery protocol utilization were included in addition to society recommendations, guidelines, consensus statements, review articles, editorials, and national surveys regarding the practice of enhanced recovery after cesarean delivery. Published letters, theses, and abstracts from scientific meetings were excluded.

Stakeholder Selection

In order to include key stakeholders involved in enhanced recovery after cesarean delivery implementation and delivery, representatives from obstetricians and maternal-fetal medicine specialists, anesthesiologists, midwife, nursing staff, and patients were sought. Corresponding authors of enhanced recovery after cesarean delivery implementation studies and guidelines or consensus statements written by professional societies about enhanced recovery after cesarean delivery were contacted. These publications were identified in a previously published systematic review of the literature.8  It was anticipated that identification of experts in this way would result in inclusion of an internationally representative breadth of experience and would result in inclusion of a broad range of ethno-demographic backgrounds. Furthermore, this approach would ensure representation from varied practices of peripartum care and cesarean delivery within different cultural and healthcare settings and ensure development of the most relevant and generalizable core outcome set possible for the majority of clinicians and researchers.

Through this process, a diverse panel of international stakeholders was invited including obstetricians‚ maternal-fetal medicine specialists‚ and anesthesiologists from North America (USA and Canada), the United Kingdom, France, Serbia, Africa, and China. Potential stakeholders were invited on January 19, 2021, via email and provided with details regarding study aims, a summary of proposed methodology, authorship agreement, and timelines of the study (Supplemental Digital Content, http://links.lww.com/ALN/C861). If potential stakeholders were unable or declined to participate, they were invited to recommend a suitable alternative enhanced recovery expert with experience in evaluating or assessing the impact of enhanced recovery after cesarean delivery. The Director of Midwifery Services at Stanford University (C.M., also a registered nurse) was invited to participate in order to represent the opinions of midwives and nurses involved in enhanced recovery after cesarean delivery implementation. Female anesthesiology faculty were also approached to represent the views of patients. In order to be considered for this role, the faculty members needed to have experienced childbirth in a hospital setting at least twice within the preceding 5 yr (between 2015 and 2020). A minimum of one patient representative was sought to participate in the planned Delphi process.

Round 1

The comprehensive list of outcomes was sent on February 1, 2021, to all stakeholders who agreed to participate in the study. Stakeholders were presented with 98 perioperative outcomes that were previously reported in a systematic review of enhanced recovery after cesarean delivery implementation studies.8  These outcomes were grouped by the Executive Committee as follows: general measures (length of stay, compliance, savings, readmissions, obstetric outcomes, and complications); maternal satisfaction, pain, breastfeeding, and side effects (nausea and vomiting, pruritus); process metrics (mobilization, oral intake, fluid therapy, staff follow-up); and neonatal outcomes (such as maternal-neonatal bonding assessment). The comprehensive list was presented in the form of a spreadsheet and distributed to stakeholders to score each outcome from 1 to 9 (1 to 3 indicating the outcome is “of limited importance or invalid,” 4 to 6 indicating the outcome is “important but not critical for inclusion or requires revision” and 7 to 9 indicating the outcome is “critical for inclusion”),17–19  as recommended by the Grading of Recommendations Assessment, Development and Evaluation working group for assessing the level of importance about research evidence.20,21  Definitions supported by published studies and agreed upon by the Executive Committee were also provided for each term utilized in the long list, and stakeholders were invited to amend or edit as they felt appropriate using free text. Predefined dropdown menus were used wherever possible with space to provide alternatives and general feedback comments. Responses were returned to the project administrator (P.P.) for anonymization and collation. A fully anonymized spreadsheet containing all comments and score selections were then analyzed by the Executive Committee, and revisions were made as deemed necessary for Round 2 along with explicit justification for the implemented changes. The criteria required for an outcome to proceed to Round 2 included either (1) a score of 7 or greater selected by 70% or more of stakeholders, or (2) a score between 4 and 6 selected by 70% or more of stakeholders (i.e., the number of scores 7 or greater and 4 to 6 scores were not combined, and strong consensus was required to proceed to the next round). Outcomes were excluded if they did not meet the above criteria or if a score between 1 to 3 was selected by 70% or greater of stakeholders. The cutoff value of 70% was selected after review of the Core Outcome Measures in Effectiveness Trials Handbook.22  This approach was also supported by a previously published study evaluating a long list of outcomes as proposed in this study.23  Consensus was achieved among the Executive Committee to utilize this cutoff value.

Round 2

On March 12, 2021, all stakeholders who participated in Round 1 received an anonymized summary of the results, including numbers of selections for scores between 1 to 3, 4 to 6, and 7 to 9 in addition to median (and 25th and 75th centile scores) for each outcome from the long list. In Round 2, stakeholders were once again invited to score each outcome (that met the criteria to proceed to Round 2) from 1 to 9 and were also invited to select preferred definitions (such as length of stay from time of delivery or postanesthesia care unit admission until hospital discharge) and units of measurement (such as visual analogue scale, numerical reporting scale, or Likert scale to assess satisfaction) for outcomes utilizing a series of dropdown menus. If a stakeholder felt that the appropriate option was not included in the relevant dropdown menu options, they were invited to enter free text alternatives that would be considered in the Round 3 discussion. The criteria for an outcome to proceed to Round 3 were the same as criteria to proceed to Round 2.

Round 3

After a Doodle poll (online scheduling tool; Zürich, Switzerland) to determine availability, stakeholders were initially invited to attend a recorded electronic roundtable discussion on April 12, 2021 (using Zoom Video Communications software, USA), aiming to achieve consensus on the final core outcome set, definitions, and preferred units of measurement. The session was chaired by an Executive Committee member (P.S.) and one cochair (B.C.). Comments and median scores from Round 1 and 2 included and excluded outcomes were disclosed to all stakeholders before the meeting. Outcomes that met the criteria for inclusion in Round 3 were discussed among available stakeholders. Discussion was limited to 5 min per outcome and guided by the following factors: (1) validity, whether it measures what it is supposed to measure; (2) reliability: stability of indicator when measurement repeated; (3) feasibility: practicality/ease of use; and (4) clarity of definition: ease of understanding. After discussion, stakeholders were invited to participate in live anonymized online polling, with stakeholders voting to either “include” or “exclude” each outcome. Using an iterative process, areas that warranted revision were modified, and subsequent voting was undertaken.

Once outcomes that met Round 3 criteria were discussed and voted upon, the stakeholders were given the opportunity to (1) reintroduce previously excluded outcomes from Rounds 1 or 2, and (2) introduce new outcomes that had not previously been used in enhanced recovery after cesarean delivery implementation studies, and therefore had not been considered thus far in the Delphi process. Any outcome that was reintroduced or introduced for the first time during Round 3 required a study participant to propose the outcome, and a separate participant to second the motion and provide brief justification to the remaining Delphi stakeholders, before it was considered and voted upon. The Round 3 electronic roundtable discussion was limited to 2 h (April 27, 2021), and therefore discussion was continued on May 19, 2021, in order to complete the proposed Round 3 tasks. In Round 3, outcomes were classified as a proportion of current stakeholders as follows:22,24 

  1. 70% or greater agreement: strong consensus; outcome accepted in the core outcome set

  2. 50 to 69% agreement: weak consensus discussed in the article as an outcome to consider when designing an enhanced recovery after cesarean delivery study or evaluating a clinical protocol

  3. Less than 50% agreement: outcome excluded as a core enhanced recovery after cesarean delivery outcome

Table 1.

Summary of Numbers of Votes and Strength of Consensus for Each Outcome Considered during Round 3 Discussion

Summary of Numbers of Votes and Strength of Consensus for Each Outcome Considered during Round 3 Discussion
Summary of Numbers of Votes and Strength of Consensus for Each Outcome Considered during Round 3 Discussion

Digital recordings of the Round 3 discussions were sent to all stakeholders who completed Rounds 1 and 2 but could not attend Round 3. All stakeholders were asked to approve the finalized core outcome set before publication.

Statistical Analysis

Cutoff values of 70% or greater and 50 to 69% agreement for strong and weak consensus, respectively, were decided upon based on Executive Committee consensus and consistent with previously published literature,23  since no broad agreement exists regarding what determines consensus.25  A minimum of 17 stakeholders was the desired target in this study, since the median [interquartile range] and range of participants in Delphi studies have previously been reported as 17 [11 to 31] and 3 to 418, respectively.25  Data were reported descriptively. Spreadsheets were developed for each round and circulated in Microsoft Excel (Excel for Mac; version 16.49, 2021; USA) spreadsheet format. All denominator values for percentages were based on responses, and percentage values reported signify the proportion of stakeholders in agreement with a particular option.

A total of 64 stakeholders were invited to collaborate in the Delphi process, and 32 agreed to participate. Among the 32 stakeholders who agreed to participate, 32 (100%), 31 (97%), and 26 (81%) participated in the first, second and third rounds, respectively. Figure 1 summarizes the numbers and respective specialties of stakeholders. Supplemental Digital Content File 3 (http://links.lww.com/ALN/C861) summarizes the country of practice for the stakeholders that completed Rounds 1 and 2 of the Delphi process. The healthcare workers who participated in Rounds 1 and 2 practiced in seven different countries, across four continents (North America, Europe, Asia, and Africa). The patient stakeholders who participated in the Delphi process experienced childbirth within the United States and United Kingdom. One stakeholder was suggested as an alternative participant by a U.S.-based obstetrician who was initially invited to participate.

Fig. 1.

Summary of stakeholder participation in study. Numbers in Round 3 discussion represent the total number that were present during some part of the discussion (numbers during each individual online poll varied). All patients are anesthesia faculty that have delivered at least twice in the past 5 yr. *A total of 26 stakeholders participated in either Round 3 Discussion 1 or 2 or both.

Fig. 1.

Summary of stakeholder participation in study. Numbers in Round 3 discussion represent the total number that were present during some part of the discussion (numbers during each individual online poll varied). All patients are anesthesia faculty that have delivered at least twice in the past 5 yr. *A total of 26 stakeholders participated in either Round 3 Discussion 1 or 2 or both.

Close modal

Rounds 1 and 2

A full list of outcomes considered in Rounds 1 and 2, with median scores and numbers of stakeholders scoring 1 to 3, 4 to 6, and 7 to 9, is provided in the Supplemental Digital Content, tables 1 and 2 (http://links.lww.com/ALN/C861). In total, 98 outcomes were considered in Round 1, and 22 outcomes in Round 2.

Round 3

In Round 3, discussion followed by voting occurred for 14 outcomes that met the criteria for inclusion in Round 3, four reintroduced outcomes (postoperative nausea or vomiting, cost savings, readiness for hospital discharge, and time until first solid intake) and three newly introduced outcomes (Obstetric Quality of Recovery-10, a composite patient-reported outcome measure of quality of recovery; requirement for neonatal intensive care unit admission and neonatal length of hospital stay greater than maternal length of hospital stay). A summary of the votes for the above 21 outcomes considered in Round 3 and the decisions to include or exclude and the strength of consensus (if included) are provided in table 1.

Table 2.

Final Core Outcome Set (with Definitions) to Be Considered in Future Enhanced Recovery after Cesarean Delivery Implementation Studies

Final Core Outcome Set (with Definitions) to Be Considered in Future Enhanced Recovery after Cesarean Delivery Implementation Studies
Final Core Outcome Set (with Definitions) to Be Considered in Future Enhanced Recovery after Cesarean Delivery Implementation Studies

Core Outcome Set

The core outcome set, which achieved a strong consensus during the Round 3 discussion (70% or greater votes to include), in addition to the definitions and units of measurement where applicable, is provided in table 2. In total, 15 outcomes met the criteria for inclusion in the final core outcome set for future enhanced recovery after cesarean delivery implementation studies.

Two outcomes (maternal satisfaction regarding cesarean delivery and requirement for neonatal intensive care admission) considered during the Round 3 discussions achieved weak consensus (Supplemental Digital Content, table 3, http://links.lww.com/ALN/C861). These outcomes (and their corresponding definitions) received between 50 and 69% of votes to be included in the core outcome set.

This Delphi study resulted in a core outcome set of 15 measures that could be adopted in future enhanced recovery after cesarean delivery studies, quality improvement, and audit projects. This proposed core outcome set, derived using Delphi methodology and involving international key stakeholders, represents key aspects of enhanced recovery after cesarean delivery, which can be used to implement and evaluate protocol success.

Optimal measurement and evaluation of postpartum recovery outcomes for research and clinical evaluations is variable, despite growing popularity of enhanced recovery protocols.1,2  A recent systematic review reported 11 published enhanced recovery after cesarean delivery implementation studies and an additional 36 abstracts presented at scientific meetings, comparing enhanced recovery after cesarean delivery to a control group.8  In total, 98 outcome measures have previously been used by studies evaluating the impact of enhanced recovery after cesarean delivery protocols.

Heterogeneity among outcomes reported in enhanced recovery after cesarean delivery implementation studies has limited the ability of researchers to pool results from these published studies. Furthermore, the majority of enhanced recovery after cesarean delivery studies utilize a before versus after implementation rather than a randomized controlled study design,8,9  which invariably results in the downgrading of levels of evidence for the outcomes reported in published enhanced recovery after cesarean delivery implementation studies, as rated according to the Grading of Recommendations Assessment, Development and Evaluation criteria.26,27  Important outcomes are either not reported or inconsistently defined among published enhanced recovery after cesarean delivery implementation studies. Length of hospital stay, for example, is the most commonly reported outcome among enhanced recovery after cesarean delivery implementation studies; however, it can be measured from time of hospital admission, time of delivery, or time from postanesthesia care unit admission until hospital discharge. Furthermore, several studies reporting length of stay present the proportion of women discharged on a specific postoperative day as stated in their enhanced recovery after cesarean delivery protocol (for example, the percentage of women discharged on day 1, 2, or 3 after cesarean delivery). This highlights the need for standardization in reporting among enhanced recovery after cesarean delivery studies. Findings from this study recommend that time from delivery to hospital discharge (in hours) could be measured and reported in future enhanced recovery after cesarean delivery studies.

Despite the heterogeneous outcomes reported in enhanced recovery after cesarean delivery implementation studies, several themes in this Delphi study were identified that can be used to measure success of an enhanced recovery after cesarean delivery protocol. For example, measuring length of length of hospital stay can identify reduction in duration of requirement for hospital level care and is therefore considered a marker of global health, recovery, and readiness for hospital discharge. Reporting of high overall compliance with individual enhanced recovery after cesarean delivery protocol components is likely to imply successful cultural “buy-in” from all multidisciplinary team members, and maternal morbidity associated with enhanced recovery after cesarean delivery protocols can be assessed by evaluating the need for hospital re-admission or unplanned outpatient or emergency room consultations. Other measurable improvements through the use of the proposed core outcome set include the provision of optimal analgesia (lower opioid consumption or requirement, higher maternal satisfaction with analgesia associated with enhanced recovery after cesarean delivery); a lower incidence of opioid-related side effects (nausea or vomiting); minimal preoperative and postoperative fasting times (to minimize the physiologic impact of surgery and promote the return of normal gut functioning); and reduced times to mobilization and urinary catheter removal (to minimize risks of thromboembolism and urinary tract infection). Finally, during the Round 3 discussion, members of the Delphi group acknowledged that postpartum recovery is a multidimensional construct, which led to the inclusion of a validated composite patient-reported outcome measure of obstetric recovery (the Obstetric Quality of Recovery-10).3,4,28–31 

Before introducing an enhanced recovery after cesarean delivery protocol, centers can consider developing the infrastructure required to capture some of the key outcomes proposed in this study in real-time during routine clinical care, without the need for additional research or administrative support. This process may involve the use of electronic healthcare records, and training of nursing staff to document times for key outcomes such as mobilization and urinary catheter removal. Routine collection of such data may facilitate and improve obstetric enhanced recovery research.

The Enhanced Recovery after Surgery Society (Stockholm‚ Sweden) and Enhanced Recovery after Surgery Society USA (Beverly‚ Massachusetts) recently created the Reporting on Enhanced Recovery after Surgery Compliance, Outcomes, and Elements Research checklist‚ which was designed to improve the quality of reporting of enhanced recovery after surgery studies.32  This tool consists of 20 items including best practices for reporting clinical pathways, compliance auditing, and formatting guidelines. However, this checklist is generic and not specific to cesarean delivery, a surgical model that has distinct differences (such as consideration of the newborn and predominantly neuraxial vs. general anesthesia) compared with other surgical models. This study has identified the measures of enhanced recovery after cesarean delivery considered to be of greatest importance to stakeholders and proposes a standardized way to evaluate individual enhanced recovery after cesarean delivery protocol success. The core outcome set developed in this Delphi study could be used in conjunction with the Reporting on Enhanced Recovery after Surgery Compliance, Outcomes, and Elements Research checklist in order to facilitate and improve standardized reporting of enhanced recovery after cesarean delivery implementation studies.

Evidence supporting enhanced recovery after cesarean delivery use remains low or very low for most outcomes.8,9  This is likely due to controversy related to the optimal study design to determine the impact of enhanced recovery after cesarean delivery protocols. Specifically, a randomized controlled study design is generally considered not ideal or practical in this setting to demonstrate the impact of enhanced recovery after cesarean delivery protocols. On the other hand, before versus after observational (implementation) studies are susceptible to higher risk of bias. High-quality levels of evidence supporting enhanced recovery after cesarean delivery use are lacking, but urgently needed. This may be best achieved through cluster randomization of entire hospitals or healthcare systems, or through prolonged periods of implementation in order to minimize potential confounding factors, or through time-sequenced introduction of interventions in order to identify the most impactful enhanced recovery after cesarean delivery protocol interventions that improve the recovery outcomes identified in this study.

This study was limited by the disproportionate numbers of each type of stakeholder (a comparatively low number of patients and nursing or midwifery staff), and the Round 3 discussion did not include any nursing or midwifery input. The limited number of obstetricians compared with anesthesiologists (8 vs. 18) in addition to the lower number of patient representatives (4), who were all from within the specialty of anesthesiology, may have contributed to selection bias in development of our final core outcome set. The rate of acceptance to participate after email invitations was higher among anesthesiologists than obstetricians (46% vs. 38%, respectively), but the reason for nonresponse to invitations to participate was not determined. However, after recruitment, the overall response rates were high (greater than 80%) for each round of the Delphi process, with all types of stakeholders contributing to Rounds 1 and 2, rounds that resulted in exclusion of the majority of outcomes. The aim of the study was to develop a list of core outcome measures that can be used to evaluate and improve enhanced recovery research. Therefore, by design, we sought greater representation from stakeholders who had evidence of research expertise in enhanced recovery after cesarean delivery. We also selected patient representatives with a medical background to facilitate this by instructing these stakeholders to provide input from a patient’s perspective, having recently experienced hospitalization for childbirth. We therefore feel that the Delphi process provided sufficient overall representation for the development of this core outcome set. Two stakeholders (P.S. and B.C.) were involved in the development and validation of Obstetric Quality of Recovery-10, but both participants abstained from voting for this item in Round 3, and online polling was performed anonymously in order to ensure ethicality. This Delphi is based on our current understanding of enhanced recovery after cesarean delivery, and this core outcome set may need to be updated as new evidence in this field emerges. While attempts were made to include authors from all settings, we acknowledge that there was underrepresentation of stakeholders from low and middle-income (as defined by the Wellcome Group which compiles its data from the Organisation for Economic Co-operation and Development) countries and some high-income countries such as Australia and New Zealand. Therefore, the core outcome set developed in this study may not be generalizable to all populations.

There is a large degree of heterogeneity among interventions implemented and outcomes evaluated among published enhanced recovery after cesarean delivery studies. This proposed enhanced recovery after cesarean delivery core outcome set addresses this heterogeneity by standardizing outcomes that will be evaluated in future studies. Importantly, there remains no consensus regarding what an enhanced recovery after cesarean delivery protocol requires as a minimum set of interventions. Institutions will ultimately implement interventions based on feasibility, perception of likely benefit, infrastructure, medication, and equipment availability. Consequently, “enhanced recovery after cesarean delivery” is unlikely to ever be a consistent intervention that one can readily compare between institutions or healthcare settings. Therefore, in order to maximize consistency and efficiency of future research and clinical efforts, hospitals could aim to implement as many of the Society of Obstetric Anesthesia and Perinatology and Enhanced Recovery after Surgery Society recommendations as possible within their healthcare setting. Future recommendations are required to define enhanced recovery after cesarean delivery and set the minimum number of interventions that are required for enhanced recovery after cesarean delivery to be met.

In summary, this Delphi study resulted in an enhanced recovery after cesarean delivery core outcome set of 15 items that could be used when designing and performing future studies that evaluate the impact of enhanced recovery after cesarean delivery protocols. Further work is needed to improve the quality of evidence supporting enhanced recovery after cesarean delivery implementation using this core outcome set in robustly designed high-quality studies, and future efforts are needed to define enhanced recovery after cesarean delivery and minimum intervention requirements that impact these outcomes.

Research Support

Dr. Sultan is an Arline and Pete Harman Endowed Faculty Scholar of the Stanford Maternal and Child Health Research Institute.

Competing Interests

Dr. George reports a financial relationship with Octapharma Inc. (Paramus, New Jersey), is an Associate Editor for the Canadian Journal of Anesthesia, and has received Expert Witness payments. Dr. Weiniger reports a financial relationship with Elsevier (Amsterdam, Netherlands), is an Editor for the International Journal of Obstetric Anesthesia, and is a consultant for Biomed JR (Tel Mond, Israel). Dr. Habib reports financial relationships with Pacira Biosciences (Tampa, Florida), Haisco (San Diego, California), Heron Pharmaceuticals (San Diego, California), Vertex (Boston, Massachusetts), MDoloris (Loos, France), and Trevena (Chesterbrook, Pennsylvania). Dr. Lim is Chair of the Advisory Board for Octapharma Inc. Dr. Zakowski is the owner of Quantum Birthing LLC (Yorba Linda, California). Dr. Nelson reports speaker/advisory fees from 3M, GSK, Abbott, and Medtronic outside of the submitted work, and is Secretary of the ERAS Society. The other authors declare no competing interests.

Enhanced Recovery after Cesarean Core Outcome Set, http://links.lww.com/ALN/C861

1.
Sultan
P
,
Sadana
N
,
Sharawi
N
,
Blake
L
,
El-Boghdadly
K
,
Falvo
A
,
Ciechanowicz
S
,
Athar
W
,
Shah
R
,
Guo
N
,
Jensen
S
,
El-Sayed
Y
,
Cella
D
,
Carvalho
B
:
Evaluation of domains of patient-reported outcome measures for recovery after childbirth: A scoping and systematic review.
JAMA Netw Open
2020
;
3
:
e205540
2.
Sultan
P
,
Sharawi
N
,
Blake
L
,
Ando
K
,
Sultan
E
,
Aghaeepour
N
,
Carvalho
B
,
Sadana
N
:
Use of patient-reported outcome measures to assess outpatient postpartum recovery: A systematic review.
JAMA Netw Open
2021
;
4
:
e2111600
3.
Sultan
P
,
Carvalho
B
:
Postpartum recovery: What does it take to get back to a baseline?
Curr Opin Obstet Gynecol
2021
;
33
:
86
93
4.
Sharawi
N
,
Klima
L
,
Shah
R
,
Blake
L
,
Carvalho
B
,
Sultan
P
:
Evaluation of patient-reported outcome measures of functional recovery following caesarean section: A systematic review using the consensus-based standards for the selection of health measurement instruments (COSMIN) checklist.
Anaesthesia
2019
;
74
:
1439
55
5.
Bollag
L
,
Lim
G
,
Sultan
P
,
Habib
AS
,
Landau
R
,
Zakowski
M
,
Tiouririne
M
,
Bhambhani
S
,
Carvalho
B
:
Society for Obstetric Anesthesia and Perinatology: Consensus statement and recommendations for enhanced recovery after cesarean.
Anesth Analg
2021
;
132
:
1362
77
6.
Macones
GA
,
Caughey
AB
,
Wood
SL
,
Wrench
IJ
,
Huang
J
,
Norman
M
,
Pettersson
K
,
Fawcett
WJ
,
Shalabi
MM
,
Metcalfe
A
,
Gramlich
L
,
Nelson
G
,
Wilson
RD
:
Guidelines for postoperative care in cesarean delivery: Enhanced Recovery After Surgery (ERAS) Society recommendations (part 3).
Am J Obstet Gynecol
2019
;
221
:
247.e1
9
7.
ACOG Committee opinion No. 750 summary: Perioperative pathways: Enhanced Recovery After Surgery.
Obstet Gynecol
2018
;
132
:
801
2
8.
Sultan
P
,
Sharawi
N
,
Blake
L
,
Carvalho
B
:
Enhanced recovery after caesarean delivery versus standard care studies: A systematic review of interventions and outcomes.
Int J Obstet Anesth
2020
;
43
:
72
86
9.
Sultan
P
,
Sharawi
N
,
Blake
L
,
Habib
AS
,
Brookfield
KF
,
Carvalho
B
:
Impact of enhanced recovery after cesarean delivery on maternal outcomes: A systematic review and meta-analysis.
Anaesth Crit Care Pain Med
2021
;
40
:
100935
10.
Bampoe
S
,
Cook
T
,
Fleisher
L
,
Grocott
MPW
,
Neuman
M
,
Story
D
,
Myles
P
,
Haller
G
:
Clinical indicators for reporting the effectiveness of patient quality and safety-related interventions: A protocol of a systematic review and Delphi consensus process as part of the international Standardised Endpoints for Perioperative Medicine initiative (StEP).
BMJ Open
2018
;
8
:
e023427
11.
Kirkham
JJ
,
Davis
K
,
Altman
DG
,
Blazeby
JM
,
Clarke
M
,
Tunis
S
,
Williamson
PR
:
Core Outcome Set-STAndards for Development: The COS-STAD recommendations.
PLoS Med
2017
;
14
:
e1002447
12.
Kirkham
JJ
,
Gorst
S
,
Altman
DG
,
Blazeby
JM
,
Clarke
M
,
Devane
D
,
Gargon
E
,
Moher
D
,
Schmitt
J
,
Tugwell
P
,
Tunis
S
,
Williamson
PR
:
Core Outcome Set-STAndards for Reporting: The COS-STAR statement.
PLoS Med
2016
;
13
:
e1002148
13.
Basson
R
,
Berman
J
,
Burnett
A
,
Derogatis
L
,
Ferguson
D
,
Fourcroy
J
,
Goldstein
I
,
Graziottin
A
,
Heiman
J
,
Laan
E
,
Leiblum
S
,
Padma-Nathan
H
,
Rosen
R
,
Segraves
K
,
Segraves
RT
,
Shabsigh
R
,
Sipski
M
,
Wagner
G
,
Whipple
B
:
Report of the international consensus development conference on female sexual dysfunction: definitions and classifications.
J Urol
2000
;
163
:
888
93
14.
Beattie
E
,
Mackway-Jones
K
:
A Delphi study to identify performance indicators for emergency medicine.
Emerg Med J
2004
;
21
:
47
50
15.
Diamond
IR
,
Grant
RC
,
Feldman
BM
,
Pencharz
PB
,
Ling
SC
,
Moore
AM
,
Wales
PW
:
Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies.
J Clin Epidemiol
2014
;
67
:
401
9
16.
Eubank
BH
,
Mohtadi
NG
,
Lafave
MR
,
Wiley
JP
,
Bois
AJ
,
Boorman
RS
,
Sheps
DM
:
Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology.
BMC Med Res Methodol
2016
;
16
:
56
17.
Harman
NL
,
Bruce
IA
,
Kirkham
JJ
,
Tierney
S
,
Callery
P
,
O’Brien
K
,
Bennett
AM
,
Chorbachi
R
,
Hall
PN
,
Harding-Bell
A
,
Parfect
VH
,
Rumsey
N
,
Sell
D
,
Sharma
R
,
Williamson
PR
:
The importance of integration of stakeholder views in core outcome set development: Otitis media with effusion in children with cleft palate.
PLoS One
2015
;
10
:
e0129514
18.
Bennett
WL
,
Robinson
KA
,
Saldanha
IJ
,
Wilson
LM
,
Nicholson
WK
:
High priority research needs for gestational diabetes mellitus.
J Womens Health (Larchmt)
2012
;
21
:
925
32
19.
Schmitt
J
,
Langan
S
,
Stamm
T
,
Williams
HC
;
Harmonizing Outcome Measurements in Eczema (HOME) Delphi Panel
:
Core outcome domains for controlled trials and clinical recordkeeping in eczema: International multiperspective Delphi consensus process.
J Invest Dermatol
2011
;
131
:
623
30
20.
Williamson
P
,
Altman
D
,
Blazeby
J
,
Clarke
M
,
Devane
D
,
Gargon
E
,
Tugwell
P
:
Developing core outcome sets for clinical trials: issues to consider.
Trials
2012
;
13:132
21.
GRADE
:
GRADE Working Group.
2021
.
Available at: https://www.gradeworkinggroup.org. Accessed March 25, 2021
.
22.
Williamson
PR
,
Altman
DG
,
Bagley
H
,
Barnes
KL
,
Blazeby
JM
,
Brookes
ST
,
Clarke
M
,
Gargon
E
,
Gorst
S
,
Harman
N
,
Kirkham
JJ
,
McNair
A
,
Prinsen
CAC
,
Schmitt
J
,
Terwee
CB
,
Young
B
:
The COMET handbook: Version 1.0.
Trials
2017
;
18
(
suppl 3
):
280
23.
Blazeby
JM
,
Macefield
R
,
Blencowe
NS
,
Jacobs
M
,
McNair
AG
,
Sprangers
M
,
Brookes
ST
;
Research Group of the Core Outcomes and iNformation SEts iN SUrgical Studies-Oesophageal Cancer; Consensus Group of the Core Outcomes and iNformation SEts iN SUrgical Studies-Oesophageal Cancer
:
Core information set for oesophageal cancer surgery.
Br J Surg
2015
;
102
:
936
43
24.
Wylde
V
,
MacKichan
F
,
Bruce
J
,
Gooberman-Hill
R
:
Assessment of chronic post-surgical pain after knee replacement: Development of a core outcome set.
Eur J Pain
2015
;
19
:
611
20
25.
Boulkedid
R
,
Abdoul
H
,
Loustau
M
,
Sibony
O
,
Alberti
C
:
Using and reporting the Delphi method for selecting healthcare quality indicators: A systematic review.
PLoS One
2011
;
6
:
e20476
26.
Balshem
H
,
Helfand
M
,
Schünemann
HJ
,
Oxman
AD
,
Kunz
R
,
Brozek
J
,
Vist
GE
,
Falck-Ytter
Y
,
Meerpohl
J
,
Norris
S
,
Guyatt
GH
:
GRADE guidelines: 3. Rating the quality of evidence.
J Clin Epidemiol
2011
;
64
:
401
6
27.
Guyatt
GH
,
Oxman
AD
,
Vist
G
,
Kunz
R
,
Brozek
J
,
Alonso-Coello
P
,
Montori
V
,
Akl
EA
,
Djulbegovic
B
,
Falck-Ytter
Y
,
Norris
SL
,
Williams
JW
, Jr
,
Atkins
D
,
Meerpohl
J
,
Schünemann
HJ
:
GRADE guidelines: 4. Rating the quality of evidence–Study limitations (risk of bias).
J Clin Epidemiol
2011
;
64
:
407
15
28.
Sultan
P
,
Kormendy
F
,
Nishimura
S
,
Carvalho
B
,
Guo
N
,
Papageorgiou
C
:
Comparison of spontaneous versus operative vaginal delivery using Obstetric Quality of Recovery-10 (ObsQoR-10): An observational cohort study.
J Clin Anesth
2020
;
63
:
109781
29.
Sultan
P
,
Kamath
N
,
Carvalho
B
,
Bansal
P
,
Elkhateb
R
,
Dougan
S
,
Whittington
J
,
Guo
N
,
El-Sayed
Y
,
Mhyre
J
,
Sharawi
N
:
Evaluation of inpatient postpartum recovery using the Obstetric Quality of Recovery-10 patient-reported outcome measure: A single-center observational study.
Am J Obstet Gynecol MFM
2020
;
2
:
100202
30.
Ciechanowicz
S
,
Setty
T
,
Robson
E
,
Sathasivam
C
,
Chazapis
M
,
Dick
J
,
Carvalho
B
,
Sultan
P
:
Development and evaluation of an obstetric quality-of-recovery score (ObsQoR-11) after elective caesarean delivery.
Br J Anaesth
2019
;
122
:
69
78
31.
Ciechanowicz
S
,
Howle
R
,
Heppolette
C
,
Nakhjavani
B
,
Carvalho
B
,
Sultan
P
:
Evaluation of the Obstetric Quality-of-Recovery score (ObsQoR-11) following non-elective caesarean delivery.
Int J Obstet Anesth
2019
;
39
:
51
9
32.
Elias
KM
,
Stone
AB
,
McGinigle
K
,
Tankou
JI
,
Scott
MJ
,
Fawcett
WJ
,
Demartines
N
,
Lobo
DN
,
Ljungqvist
O
,
Urman
RD
;
ERAS® Society and ERAS® USA
:
The Reporting on ERAS Compliance, Outcomes, and Elements Research (RECOvER) Checklist: A joint statement by the ERAS® and ERAS® USA Societies.
World J Surg
2019
;
43
:
1
8

Appendix

Expert Consensus Regarding Core Outcomes for Enhanced Recovery after Cesarean Delivery Studies: A Delphi Study CRADLE Investigators

Jessica R. Ansari, M.D.: Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Palo Alto, California.

Dan Benhamou, M.D.: Université Paris Saclay Hôpital Bicêtre - 78, Bicêtre Cedex, France.

Moris Baluku, M.D.: Department of Anesthesia and Critical Care, Kabale University, Kabale, Uganda.

Peter S. Bernstein, M.D., M.P.H.: Division of Maternal Fetal Medicine, Montefiore Medical Center, Albert Einstein College of Medicine, Jack D. Weiler Hospital, Bronx, New York.

Laurent A. Bollag, M.D.: Department of Anesthesia and Pain Medicine, University of Washington School of Medicine, Seattle, Washington.

Sarah J. Bowden, M.D.: Obstetrics and Gynecology, Imperial College London, London, United Kingdom.

Emily Fay, M.D.: Department of Obstetrics and Gynecology, University of Washington School of Medicine, Seattle, Washington.

Ashraf S. Habib, M.B.B.Ch., M.Sc., M.H.Sc., F.R.C.A.: Department of Anesthesiology, Duke University School of Medicine, Durham, North Carolina.

Sunil Halder, F.R.C.A.: Nuffield Dept of Anesthesia Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom.

Ruth Landau, M.D.: Columbia University Irving Medical Center, Department of Anesthesiology, New York, New York.

Grace Lim, M.D. M.Sc.: Department of Anesthesiology and Perioperative Medicine, Department of Obstetrics and Gynecology, University of Pittsburgh School of Medicine, UPMC Magee Womens Hospital, Pittsburgh, Pennsylvania.

Vincent Liu, M.D., M.S.: Kaiser Permanente Division of Research, Oakland, California.

Colleen Moreno, D.N.P., A.P.R.N., C.N.M.: Directory of Midwifery Services, Stanford University School of Medicine, Palo Alto, California.

Gregg S. Nelson, M.D., Ph.D., F.R.C.S.C.: Department of Obstetrics and Gynecology, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada.

Mark F. Powell, M.D.: Department of Anesthesiology and Perioperative Medicine, University of Alabama at Birmingham, Birmingham, Alabama.

Borislava Pujic, M.D., Ph.D.: Clinic for Anesthesia, Intensive Care Therapy and Pain Therapy, University Clinical Center of Novi Sad, Novi Sad, Serbia.

Nadir Sharawi, M.B.B.S., F.R.C.A., M.Sc.: Department of Anesthesiology, University of Arkansas for Medical Sciences, Little Rock, Arkansas.

Natasha Singh, M.B.B.S., M.D.(Res), F.R.C.O.G.: Department of Obstetrics and Gynecology. Chelsea and Westminster Hospital, London, United Kingdom.

Roger Smith, M.D.: Department of Obstetrics and Gynecology, Michigan Medicine, Ann Arbor, Michigan.

Emily Stockert, M.D., M.B.A.: Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Palo Alto, California.

Ellile Sultan, M.B.B.S., B.Sc., F.R.C.A.: Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Palo Alto, California.

Mohamed Tiouririne, M.D.: Department of Anesthesiology, University of Virginia, Charlottesville, Virginia.

R. Douglas Wilson, M.D., M.Sc. (Genetics), F.R.C.S.C. (Obstet Gynecol MFM): Department of Obstetrics and Gynecology and Medical Genetics, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada.

Ian J. Wrench, F.R.C.A.: Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom.

Romy Yun, M.D.: Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Palo Alto, California.

Mark Zakowski, M.D.: Department of Anesthesiology, Cedars-Sinai Medical Center, Los Angeles, California.