THE Maintenance of Certification in Anesthesiology (MOCA®) Simulation Course* is an important element of Part IV (Practice Performance Assessment and Improvement) of the American Board of Anesthesiology’s MOCA program. These Courses are offered at endorsed programs that form the American Society of Anesthesiologists’ (ASA) Simulation Education Network. Although the MOCA Simulation Course has been described previously,1  discussions with ASA members suggest that misunderstandings remain about several aspects of the MOCA Simulation Course (the “Course”) and its place in the overall MOCA program. We would like to clarify the nature and conduct of the MOCA Simulation Courses vis-à-vis the goal of honing the skills of board-certified anesthesiologists (“Anesthesiologists”).

Although the American Board of Anesthesiology sets the requirement for the MOCA program, the ASA’s Simulation Editorial Board (SEB) is responsible for overseeing the content and conduct of the simulation experiences. The SEB has established core Course requirements but provides latitude to the endorsed centers to “do what they do best” and to determine their own course scheduling and fees. The Course is an interactive experience designed to stimulate participants to create and subsequently engage in meaningful practice improvement activities. It is a 6- to 8-h immersive learning experience, held in an ASA-endorsed simulation center, that focuses on the management of challenging clinical events. The Course must address both the medical and technical skills of managing acute perioperative situations as well as the nontechnical skills of dynamic decision making and team management. A Course goal is to help participants identify possible system issues and approaches to improve patient care in their individual practices. Every participant is the primary anesthesiologist in at least one simulation scenario. During a scenario, they work with other participants as well as with role-playing instructors or staff as a clinical management team. Each scenario is followed by a detailed instructor-facilitated debriefing where participants reflect on what transpired and articulate lessons to improve their own practices. To achieve endorsement, among other criteria, a center must describe its various Course policies (e.g., confidentiality and cancelation) and provide evidence that its instructors can conduct simulations and debriefings of experienced clinicians with skill and sensitivity.

We emphasize that the MOCA Simulation Course is NOT A TEST. There are no individual or team scores or performance evaluations. Debriefing discussions address practice improvement, focusing on what lessons can be drawn from the scenario, and how they can be applied to actual patient care. The Course provides an opportunity for each participant to reflect on their own performance, and that of their peers, with constructive feedback from the instructors and other course participants.

The MOCA Simulation Course culminates in the creation of practice improvement plans by participants, to be implemented in the following 90 days. A goal of the American Board of Anesthesiology is for the learning that takes place during the Course to translate into subsequent practice improvement and behavior change. During the course, the instructors facilitate discussions on potential practice improvement activities stemming from the scenario experiences (e.g., “I am going to get my operating room team to practice managing an airway fire”).

Completion of the Course and the subsequent practice improvement activities qualify the participant for MOCA Part IV credit. Course participation may also qualify for Continuing Medical Education (CME) credit although this is site dependent. Currently, the Course is required once in each 10-yr MOCA cycle.

Conducting robust Courses for experienced anesthesiologists, with appropriate simulations and debriefings, is a complex process requiring extensive effort by faculty instructors, simulation specialists, pedagogy experts, and administrative personnel. Significant preparations are required before and during each Course.

Is MOCA Simulation Useful?

The MOCA Simulation Courses began in 2010. The SEB monitors Courses offered at endorsed sites to assure their value and relevance to ASA members. In addition to analyzing participants’ deidentified course evaluations and practice improvement plans, the SEB sometimes sends observers to Courses.

Experiences from the first 583 MOCA participants were evaluated in 2011 by the SEB yielding a peer-reviewed publication.1  The evaluations, practice improvement plans, and testimonials from MOCA Course participants have been overwhelmingly positive and represent powerful demonstrations of meaningful practice improvement at participants’ home institutions. As of early 2014, more than 2,700 anesthesiologists have completed a Course. More than 97% rated it as “relevant to my practice.” Ninety-six percent reported that the “course was a positive learning experience.” Almost all Course participants reported that, “what I learned will change my practice,” and that they would recommend the Course to their colleagues. Importantly, 94% of Course participants reported that they changed something in their practice. Few other CME activities lead to such robust practice reflection and change.2,3  A preliminary qualitative analysis of submitted deidentified practice improvement plans and accomplishments suggests deliberate efforts of many Course participants to conduct activities with real potential to impact the entire perioperative team.4 

Is MOCA Simulation Necessary for Anesthesiologists?

Our specialty was the pioneer in patient safety. Although we have made great strides during the past 3 decades, we must admit that patients are still harmed, even in seemingly simple, “routine” cases. Ample data show that approximately 30% of anesthetics contain “nonroutine events”5  and that a subset of these events represent significant physiological disturbances that, if not treated adequately, could lead to patient harm.6  Despite many advances in our care, the low but meaningful incidence of perioperative physiological disturbances7  has not changed appreciably since the 1980s.8  Although serious adverse events are uncommon, the ASA’s Closed Claims Project, the Anesthesia Quality Institute’s Anesthesia Incident Reporting System, and individual experiences include many examples of what should have been the “routine” anesthetics on healthy people that end tragically. The Anesthesia Patient Safety Foundation was formed in 1985 with the mission that “no patient shall be harmed from anesthesia,” a mission that clearly remains incompletely fulfilled.

For anesthesia, as for all human endeavors that strive to be “high reliability,” simulation training to hone skills and strategies is an important part of maximizing patient safety. Along with pioneering patient safety, anesthesiologists introduced simulation into medicine to address both technical and nontechnical aspects of decision making and patient care. This work led directly to the incorporation of various types of simulation into virtually all healthcare domains.

But Are Not Anesthesiologists Already Good Enough to Manage All Challenging Events?

The literature shows that even highly experienced anesthesiologists may still fail to optimally respond to challenging clinical situations.9,10  Murray et al.11  found that Anesthesiologists performed less than 20% of the indicated key actions during hyperkalemia and malignant hyperthermia scenarios. Currently, a team of 18 anesthesiology simulation experts have created four standardized high-fidelity simulation scenarios and delivered them to 300 anesthesiologists who volunteered to participate in a research study (funded by the Agency for Healthcare Research and Quality) grafted onto their MOCA Simulation Courses. State-of-the-art performance assessment metrics are being applied to evaluate performance of the volunteer individuals and teams (whose identity is kept confidential via code numbers). Domain experts have begun to review the video recordings and a germane preliminary finding is that indeed, as per previous research on smaller cohorts, the performance of anesthesiologists is quite variable. Although the vast majority of videos reveal skilled clinical practice, we have observed many simulation performances demonstrating room for improvement. Such suboptimal performances include both technical and nontechnical deficits and are seen in all four of the standardized scenarios. Only a small number of videos show exemplary performance proving that, as expected, no one is perfect. Similar patterns of performance during critical event management have been found for other highly skilled personnel in other arenas of high intrinsic risk, such as airline pilots and nuclear power plant operators.12,13  In anesthesiology, as in these other areas, when human life is at risk, exemplary performance is the goal and middling performance may not be good enough.

Could these findings just be a “simulation artifact,” that performance in the simulator does not reflect performance in the “real world”? Even though high-fidelity simulation has high face validity, it is not identical to actual practice. Nonetheless, 94% of Course participants rated the simulations as “realistic.” In a recent study, Weller et al.14  found that communication patterns were similar in actual and in simulated routine cases, thereby lending support for the contextual “validity of the simulation environment and its value in teamwork training.” To extend such findings to all components of decision making, communication, and clinical management during unusual but critical events in the simulated versus real world may be logistically impossible because the events most needing the acute interventional skills of the anesthesiologist will be uncommon.15  However, the types of performance gaps observed in MOCA Courses appear similar to events reported to the Anesthesia Incident Reporting System.

To date, the balance of evidence supports our assertion that simulation training enhances physicians’ clinical performance (and reduces complications) during actual patient care.16–19  Simulation training has been shown to improve outcome and decrease costs after central venous catheter insertion in medical intensive care units.20,21  With regard to nontechnical (i.e., behavioral or teamwork) skills, standardized patient-based simulations have improved the quality of actual handovers between anesthesia providers and recovery room nurses.22  Mannequin-based simulations targeting crisis resource management have been conducted for more than 2 decades and have become important components of anesthesiology resident training. The impact of in-person high-fidelity simulations conducted and debriefed by expert instructors is profound. For those who have not observed or taken part in such activities, the value of the experience is difficult to appreciate.

Issues about the MOCA Simulation Process

The 39 ASA-endorsed centers that offer Courses have worked hard to deliver high-quality education and practice improvement experiences to anesthesiologists. The process requires an individual to spend most of a day in the Course and, in some cases, significant travel is required. Endorsed centers are located in 21 states and more than 75% of anesthesiologists enrolled in MOCA live in those states. Most centers are located in urban areas proximate to the largest concentration of anesthesiologists in their state. New centers are continuing to apply for endorsement and the network is growing. However, for some individuals, attending a Course may be logistically inconvenient. Course tuition, although relatively high, is commensurate with its high intensity, the substantial infrastructure and preparations required, and the high instructor-to-participant ratio (no less than 1-to-5, and often greater than 2-to-5). Nonetheless, despite the cost and inconvenience, the more than 2,700 Course participants to date have been overwhelmingly positive about their experience.

Those on the SEB and in the simulation education community need to better inform our colleagues about the Course to allay concerns and anxiety about the experience. In fact, the SEB is quite sensitive about these issues. Several of the authors of this editorial, and many MOCA Simulation instructors, are themselves enrolled in the MOCA program and have already taken, or will need to take, a Course. However, the facts articulated earlier about there being no assessment, no evaluation, and no test do not seem to have been fully appreciated by many anesthesiologists. It is understandable that those who have never experienced simulation training may be concerned about “performing” in front of peers. The SEB requires endorsed programs to demonstrate their ability to conduct MOCA Simulation training in a way that addresses such concerns. The resoundingly positive responses by course participants to date show that these efforts have been successful.

Will One Simulation Course Every 10 Yr Make a Difference?

The frequency of simulation training is not the only determinant of its impact on anesthesia practice.23  The Course is designed to hone many general skills that are applicable to every case, and certainly to every challenging case. The lessons of the Course and of the practice improvement efforts are not limited only to the few scenarios run in a day, but should have lasting value. With any course or training intervention, there is a danger that skills learned will erode24  and, in a perfect world, simulation training should probably occur more regularly than once per decade. Judging from the experiences of other high-consequence industries, there is likely to be further benefit to more frequent simulation exercises. In aviation, simulation training occurs at least annually, and nuclear power operator teams typically spend an astounding 1 week in every five in simulation training. Although anesthesiology and other healthcare domains may never approach this level of commitment to proactive training of experienced professionals, to their credit some hospitals and anesthesia practices conduct their own regular simulation training. For now, the MOCA Simulation Course, attended once every 10 yr, is a credible step to providing anesthesiologists with a significant opportunity to improve their practices and hence it is an important component of the overall MOCA program.

The goal of MOCA overall, the Courses, and other patient safety initiatives is to save as many patients’ lives, hearts, and brains as possible. There are already anecdotes that these Courses have helped anesthesiologists to serve their patients and have even saved lives. Every life counts. There is a saying so profound that it appears in varying forms in both the Muslim Quran and the Hebrew Talmud, that “Whoever saves a life, it is as if he has saved all of mankind.” The public looks to board certification of clinicians as the mark of excellence. The MOCA program, and perhaps especially the simulation component, can reassure the public that anesthesiologists are taking seriously their responsibility to continually hone their skills to offer the care expected from those who are certified.

Simulation is one of the many innovations contributed by anesthesiologists that are used throughout health care every day to provide safer patient care. It is a willingness to do what is right even when it is not popular, even when it is difficult, even when it requires real effort from each of us, that demonstrates to all our healthcare colleagues, to our patients, and to the entire world, the leadership of anesthesiologists.

Acknowledgments

The authors thank the review and comment on this article by Jeff Cooper, Ph.D. (Department of Anesthesiology and Critical Care, Massachusetts General Hospital, Boston, Massachusetts), and Deborah Culley, M.D. (Department of Anesthesiology, Brigham and Women’s Hospital/Harvard Medical School, Boston, Massachusetts). The authors thank the other members of the Simulation Performance Study Group: Arna Banerjee, M.D. (Department of Anesthesiology, Vanderbilt University, Nashville, Tennessee), John (Jack) Boulet, Ph.D. (Foundation for the Advancement of International Medical Education and Research, Philadelphia, Pennsylvania), William McIvor, M.D. (The Peter M. Winter Institute for Simulation Education and Research, University of Pittsburgh, Pittsburgh, Pennsylvania), Robert L. Nadelberg, M.D. (Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts), Christine S. Park, M.D. (Department of Anesthesiology, Northwestern University, Chicago, Illinois), Eric Porterfield, M.S. (Department of Anesthesiology, Vanderbilt University, Nashville, Tennessee), Elizabeth (Lisa) Sinz, M.D. (Department of Anesthesiology, Pennsylvania State University, Hershey, Pennsylvania), Jason M. Slagle, Ph.D. (Center for Perioperative Research in Quality, Vanderbilt University, Nashville, Tennessee), and Laurence C. Torsher, M.D. (Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota).

Dr. Weinger’s effort was supported by the Department of Veterans Affairs Tennessee Valley Healthcare Systems’ Geriatric Research Education and Clinical Center (Nashville, Tennessee), by grant R18 HS020415 from the Agency for Healthcare Research and Quality (Rockville, Maryland), and by institutional resources of the Vanderbilt University School of Medicine, Nashville, Tennessee. Dr. Burden’s effort was supported by grant R18 HS020415 from the Agency for Healthcare Research and Quality (Rockville, Maryland), by the Cooper Medical School of Rowan University (Camden, New Jersey), and by Cooper Health System (Camden, New Jersey). Dr. Steadman’s effort was supported, in part, by grant R18 HS020415 from the Agency for Healthcare Research and Quality (Rockville, Maryland). Dr. Gaba’s effort was supported, in part, by grant R18 HS020415 from the Agency for Healthcare Research and Quality (Rockville, Maryland).

Competing Interests

The opinions or assertions contained in this article are the private views of the authors and are not to be construed as the official views or positions of the American Society of Anesthesiologists, the American Board of Anesthesiology, or the Department of Veterans Affairs. All of the authors are members of the American Society of Anesthesiologists Simulation Education Editorial Board (Park Ridge, Illinois). Drs. Weinger and Gaba are members of the Executive Committee of the Anesthesia Patient Safety Foundation (Indianapolis, Indiana) and have previously received grant funding from them for simulation-related patient safety research. All of the authors are MOCA simulation course instructors at their respective institutions and receive market-based compensation or other academic recognition for the time they spend teaching these courses.

*

Available at: http://education.asahq.org/sim/faq. Accessed March 5, 2014.

Available at: http://www.theaba.org/Home/anesthesiology_maintenance. Accessed March 6, 2014.

References

1.
McIvor
W
,
Burden
A
,
Weinger
MB
,
Steadman
R
:
Simulation for maintenance of certification in anesthesiology: The first two years.
J Contin Educ Health Prof
2012
;
32
:
236
42
2.
Domino
FJ
,
Chopra
S
,
Seligman
M
,
Sullivan
K
,
Quirk
ME
:
The impact on medical practice of commitments to change following CME lectures: A randomized controlled trial.
Med Teach
2011
;
33
:
e495
500
3.
Mazmanian
PE
,
Johnson
RE
,
Zhang
A
,
Boothby
J
,
Yeatts
EJ
:
Effects of a signature on rates of change: A randomized controlled trial involving continuing education and the commitment-to-change model.
Acad Med
2001
;
76
:
642
6
4.
Steadman
RS
,
Burden
AR
,
Huang
YM
,
Cooper
JB
:
Practice improvements implemented based on participation in a simulation program for maintenance of certification in anesthesiology. Presented at: Annual Meeting of the American Society of Anesthesiologists
.
San Francisco, CA
,
October 14, 2013
#LBC09
5.
Oken
A
,
Rasmussen
MD
,
Slagle
JM
,
Jain
S
,
Kuykendall
T
,
Ordonez
N
,
Weinger
MB
:
A facilitated survey instrument captures significantly more anesthesia events than does traditional voluntary event reporting.
Anesthesiology
2007
;
107
:
909
22
6.
Walsh
M
,
Devereaux
PJ
,
Garg
AX
,
Kurz
A
,
Turan
A
,
Rodseth
RN
,
Cywinski
J
,
Thabane
L
,
Sessler
DI
:
Relationship between intraoperative mean arterial pressure and clinical outcomes after noncardiac surgery: Toward an empirical definition of hypotension.
Anesthesiology
2013
;
119
:
507
15
7.
Slagle
JM
,
Anders
S
,
Porterfield
E
,
Arnold
A
,
Calderwood
C
,
Weinger
MB
:
Significant physiological disturbances associated with non-routine event containing and routine anesthesia cases.
J Patient Saf
2014
[Epub ahead of print]
8.
Forrest
JB
,
Cahalan
MK
,
Rehder
K
,
Goldsmith
CH
,
Levy
WJ
,
Strunin
L
,
Bota
W
,
Boucek
CD
,
Cucchiara
RF
,
Dhamee
S
:
Multicenter study of general anesthesia. II. Results.
Anesthesiology
1990
;
72
:
262
8
9.
Schwid
HA
,
O’Donnell
D
:
Anesthesiologists’ management of simulated critical incidents.
Anesthesiology
1992
;
76
:
495
501
10.
DeAnda
A
,
Gaba
DM
:
Role of experience in the response to simulated critical incidents.
Anesth Analg
1991
;
72
:
308
15
11.
Murray
DJ
,
Boulet
JR
,
Avidan
M
,
Kras
JF
,
Henrichs
B
,
Woodhouse
J
,
Evers
AS
:
Performance of residents and anesthesiologists in a simulation-based skill assessment.
Anesthesiology
2007
;
107
:
705
13
12.
Gaba
DM
:
Structural and organizational issues in patient safety: A comparison of health care to other high-hazard industries.
Calif Manage Rev
2001
;
43
:
83
102
13.
Gaba
DM
:
The thorniest issues in healthcare
in
Risk and Reliability in Healthcare and Nuclear Power: Learning from Each Other
. Edited by
Weinger
MB
,
Halbert
BP
,
Logan
MK
.
Arlington
,
Association for the Advancement of Medical Instrumentation
,
2013
, pp
pp 11
5
14.
Weller
J
,
Henderson
R
,
Webster
CS
,
Shulruf
B
,
Torrie
J
,
Davies
E
,
Henderson
K
,
Frampton
C
,
Merry
AF
:
Comparison of anesthesiologists’ communication patterns in real and simulated cases.
Anesthesiology
2014
;
120
:
1
7
15.
Gaba
DM
:
The pharmaceutical analogy for simulation: A policy perspective.
Simul Healthc
2010
;
5
:
5
7
16.
Bruppacher
HR
,
Alam
SK
,
LeBlanc
VR
,
Latter
D
,
Naik
VN
,
Savoldelli
GL
,
Mazer
CD
,
Kurrek
MM
,
Joo
HS
:
Simulation-based training improves physicians’ performance in patient care in high-stakes clinical setting of cardiac surgery.
Anesthesiology
2010
;
112
:
985
92
17.
Barsuk
JH
,
McGaghie
WC
,
Cohen
ER
,
O’Leary
KJ
,
Wayne
DB
:
Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit.
Crit Care Med
2009
;
37
:
2697
701
18.
Wayne
DB
,
Didwania
A
,
Feinglass
J
,
Fudala
MJ
,
Barsuk
JH
,
McGaghie
WC
:
Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: A case-control study.
Chest
2008
;
133
:
56
61
19.
Thomas
EJ
,
Taggart
B
,
Crandell
S
,
Lasky
RE
,
Williams
AL
,
Love
LJ
,
Sexton
JB
,
Tyson
JE
,
Helmreich
RL
:
Teaching teamwork during the Neonatal Resuscitation Program: A randomized trial.
J Perinatol
2007
;
27
:
409
14
20.
Cohen
ER
,
Feinglass
J
,
Barsuk
JH
,
Barnard
C
,
O’Donnell
A
,
McGaghie
WC
,
Wayne
DB
:
Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit.
Simul Healthc
2010
;
5
:
98
102
21.
Burden
AR
,
Torjman
MC
,
Dy
GE
,
Jaffe
JD
,
Littman
JJ
,
Nawar
F
,
Rajaram
SS
,
Schorr
C
,
Staman
GW
,
Reboli
AC
:
Prevention of central venous catheter-related bloodstream infections: Is it time to add simulation training to the prevention bundle?
J Clin Anesth
2012
;
24
:
555
60
22.
Weinger
MB
,
Slagle
JM
,
Kuntz
A
,
Banerjee
A
,
Schildcrout
J
,
Mercaldo
N
,
France
D
,
Speroff
T
,
Bills
J
,
Walston
K
;
and the PACU Handover Improvement Team
:
Improving actual handover behavior with a simulation-based training intervention.
Proc Hum Factors Ergon Soc
2010
;
54
:
957
61
23.
Weinger
MB
:
The pharmacology of simulation: A conceptual framework to inform progress in simulation research.
Simul Healthc
2010
;
5
:
8
15
24.
Kurrek
MM
,
Devitt
JH
,
Cohen
M
:
Cardiac arrest in the OR: How are our ACLS skills?
Can J Anaesth
1998
;
45
:
130
2