Stress Management Training Improves Overall Performance during Critical Simulated Situations: A Prospective Randomized Controlled Trial

This prospective randomized (1:1 allocation) controlled study with two parallel arms and a hypothesis of superiority was conducted in the Lyon teaching center for simulation in healthcare. The study protocol was preregistered on October 6, 2016 on clinicaltrial.gov (NCT02926599; Principal Investigator: Lilot Marc, Claude Bernard University, Lyon, France). The full trial protocol will be provided upon request. The study obtained approval from the Institutional Review Board of the Comité de Protection des Personnes SUD-EST II (2016-089). Two investigators (F.S. and G.P.) evaluated eligibility, provided information about the study, collected written individual informed consent, and enrolled residents. This study followed the recommendations of the International Committee of Medical Journal Editors and the results were reported using the Consolidated Standards of Reporting Trials guidelines.¹²  No data monitoring committee was assigned and no interim analysis was done for the primary endpoint.

This study involved all anesthesiology and critical care residents from the Lyon University who participated in high-fidelity simulation at the medical simulation center during the 2016 to 2017 academic year. These simulations were part of the resident educational program and were not used to certify these residents. No exclusion criteria were applied. Simulations were organized for the five postgraduate year residents as repetitive sessions of 4 to 5 h. Four different scenarios ran consecutively during each simulation so that each resident participated actively once in one scenario. Simulation followed the usual sequence: briefing, scenario, and debriefing.^9,13,14  Scenarios dealt with crisis situations in the emergency department, intensive care unit, and operating room (appendix 2). They were adapted to the training level of the residents. For a given scenario, the same instructor was embedded to play the role of a nurse acting as neutrally as possible and the same two instructors led all debriefings for that scenario. All instructors were anesthesiologists and intensivists certified in medical simulation or with at least 2 yr of instructor experience. Debriefings used Plus/Delta and Promoting Excellence and Reflective Learning in Simulation (PEARLS) models and lasted approximately 30 min.¹⁵  SimMan Essential and SimBaby manikins (Laerdal Medical AS, Norway) were used.

A blinded investigator (D.C.) performed a blocked and stratified randomization by postgraduate year, following simple randomization procedures (computerized random numbers) before the Tactics to Optimize the Potential training, by randomly picking concealed names to assign them to either the Tactics to Optimize the Potential or control groups.

Residents randomly assigned to the Tactics to Optimize the Potential group had a standardized stress management training based on strategies suitable for residents coping with stress induced by critical clinical situations. Residents from the same postgraduate year were convened in groups of 10 to 14. The training was provided in 60-min sessions, once a week during 5 consecutive weeks starting between 1 and 3 months before the scheduled simulation (appendix 3). Two Tactics to Optimize the Potential instructors (M.B. and A.G.-L.) led all of the workouts, which included controlled breathing control with relaxing and revitalizing breathing techniques,¹⁶  as well as mental rehearsal¹⁷  of upcoming actions to optimize the state of readiness and other techniques (mental imagery,^18,19  coherence cardiac biofeedback,²⁰  sensory relaxation²¹ ; appendix 1). Residents were not told they would be using Tactics to Optimize the Potential during the next simulation.

Residents were randomly allocated to a half-day session of simulation by residents from the same intervention group (Tactics to Optimize the Potential or control), in order to have at the end of each postgraduate simulation, for any given repeated scenario, the same number of residents from both groups. Residents (not informed of scenario assignment) were allocated by one investigator (F.S.) before the start of the simulation by randomly picking concealed names to assign them to the order of scenarios. All simulation instructors were blinded to the group allocation of each resident. During the simulation, residents from the Tactics to Optimize the Potential group had a specific reactivation of Tactics to Optimize the Potential immediately after the briefing and before involvement as active participants. The same investigator (G.P.) led the next active participant from the Tactics to Optimize the Potential group to sit down in a nearby isolated room. He guided each reactivation (standardized procedure for 5 min; appendix 4), by slowly reading a reactivation text starting with mental rehearsal for 4 min followed by revitalizing breathing for 1 min (appendices 5 and 6). At the end of the Tactics to Optimize the Potential reactivation, the active participant was brought into the simulation theater and the scenario started.

Residents from the control group had no training before the simulation. During the simulation, the same investigator (G.P.) led the active participant from the control group to sit down in the same nearby isolated room and to review normal printed laboratory test results unrelated to the scenario. The resident was asked, in a standardized manner, to read the results for 5 min to identify any abnormalities. Then the active participant was brought to the simulation theater and the scenario started.

For each scenario, a scenario-specific checklist was established beforehand to assess clinical performance. Using a Delphi-inspired approach, checklists were collaboratively developed by simulation instructors, taking into account available guidelines as well as the particularities of each scenario. These checklists have been described previously and contain multiple items corresponding to specific actions.²²  Each item is associated with a number of points so that the total was 100. When residents performed an action without suggestion by the embedded nurse, all points were awarded for the associated item. If residents did not perform an action essential to the scenario progression, the embedded-nurse instructor (linked to the chief instructor via a headset) suggested it implicitly to residents (implicit facilitation). If the essential action was not performed despite implicit facilitation, the instructor could suggest it explicitly (explicit facilitation). In case of implicit facilitation, only half of the points were awarded. No points were awarded if the action was not performed or if it was performed after explicit facilitation. The clinical performance score was obtained by summing all scored items. Two assessors (M.L. and J.-N.E.) blinded to group allocation evaluated all clinical performances independently, using the video recordings.

Nontechnical skills evaluation: For each scenario, two different performance evaluation scales were used to assess nontechnical skills by four independent assessors blinded to group allocation. The Ottawa Crisis resource management Global Rating Scale (Ottawa scale) was assessed by the same two clinical performance assessors (M.L. and J.-N.E.), using the video records. Ratings of the six criteria were summed, resulting in scores ranging from 6 to 42 points.²³  The Team Emergency Assessment Measure score was assessed independently by the two instructors prior to leading the debriefing. Ratings of the 12 criteria were summed, resulting in scores ranging from 1 to 54 points.²⁴ 

The average of the two assessors’ score for each performance scale was obtained for each resident.

At the beginning of the Tactics to Optimize the Potential training, residents were asked to complete a demographic questionnaire, the French validated translation of the Perceived Stress Scale, the State-Trait Anxiety Inventory-Trait (from 20 to 80 points) and the Fear of Negative Evaluation scale (from 0 to 30 points).^25–27  At the end of the Tactics to Optimize the Potential training, residents asked to report a Visual Analogue Scale (VAS) rating of Tactics to Optimize the Potential training (subsequently converted to a 0 to 100 mm numerical scale) assessing the perceived ease-of-use and efficiency-of-use of Tactics to Optimize the Potential.

On the day of simulation, residents were asked to complete a State-Trait Anxiety Inventory-State (from 20 to 80 points) immediately before the specific preparation.²⁶  VAS of stress were also completed interactively.²⁸  At the end of the debriefing, a VAS score of the effect of specific preparation for stress reduction (VAS-stress reduction) and a VAS score of effect of specific preparation for overall performance in simulation (VAS-simulation performance) were provided by residents (fig. 1).

The primary endpoint was the mean overall performance during simulation calculated as the sum of the clinical performance, the Ottawa scale and the Team Emergency Assessment Measure scores (maximum: 100 + 42 + 54 =196). This overall performance was then divided by 1.96 in order to obtain an overall score between 0 and 100. Secondary endpoints were clinical performance, Ottawa scale score, Team Emergency Assessment Measure score, VAS of stress, and specific VAS collected at the end of the debriefing.

No data were available to predict the effect of the intervention; no sample size was calculated a priori. However, in order to obtain a representative cohort from each year of residency, all anesthesiology and intensive care residents scheduled for simulation during the 2016 to 2017 academic year were invited to be included. Statistical analysis was performed on an intention-to-treat basis. All tests were two-tailed, and P < 0.05 was considered statistically significant. Shapiro–Wilk W test was used to confirm the normality of the distribution. Difference estimates with 95% CIs (95% CI) are provided for all comparisons. Effect size and 95% CI is reported using Cohen’s d for parametric data, Cliff’s delta for nonparametric data or partial eta squared (_pη²) for the analysis of covariance, using sjstats, as appropriate. The magnitude of Cohen’s d is considered small, medium, or large when around 0.2, 0.5, or 0.8, respectively. The magnitude of Cliff’s delta is considered small, medium, or large when around 0.2, 0.4, or greater than 0.5, respectively. The magnitude of _pη² is considered small, medium, or large when around 0.01, 0.06, or greater than 0.14, respectively.^29–31 

The primary endpoint was compared using independent t test. To refine the analysis, we performed an analysis of covariance with the scenario as cofactor.

Secondary endpoints: categorical variables were presented using absolute and relative frequencies and compared using Fisher’s exact test. VAS continuous variables were described; using mean ± SD or median [25th to 75th] and compared using independent t test, independent samples Mann–Whitney test, or paired samples Wilcoxon test as appropriate. Clinical performance, Ottawa scale and Team Emergency Assessment Measure scores, were compared using independent t test. We also performed an analysis of covariance with the scenario as cofactor. Inter-rater reliability of investigators was assessed by calculating absolute interclass correlation coefficients and 95% CI for single and average measures for same raters or random raters as appropriate. Correlation between nontechnical skills scores was evaluated using the Pearson (r) correlation index.

Data analyses were performed using MedCalc version 12.1.4.0 (Medcalc Software, Ostend, Belgium) and R version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria).

A total of 134 residents were randomized from November 22, 2016 to June 9, 2017. Of the 66 residents allocated to the Tactics to Optimize the Potential group, 61 received Tactics to Optimize the Potential training and 5 were unable to participate in the training. Of the 68 residents allocated to the control group, one declined to participate. One hundred twenty-eight residents were included in this intention-to-treat analysis (fig. 2). Characteristics of the two groups and the VAS of Tactics to Optimize the Potential training evaluation are presented in table 1.

The overall performance was 56 ± 10 out of 100 points. For all scenarios, the difference between minimum and maximum scores was 22 ± 9 points. Outcome data is presented in table 2.

The overall performance was 9% higher in the Tactics to Optimize the Potential group (59 ± 10) as compared with the control group (54 ± 10; difference, 5 [95% CI, 1 to 9]; P = 0.010; effect size, 0.50 [95% CI, 0.16 to 0.91]; table 2). Having removed the variability due to scenarios (analysis of covariance), the P value improved (P = 0.002; table 3).

The absolute interclass correlation coefficients for the two independent assessments of clinical performance was 0.97 [0.96 to 0.98] for single measures and 0.99 [0.98 to 0.99] for average measures. There was no significant difference between Tactics to Optimize the Potential and control groups with regard to the clinical performance (respectively, 48 ± 11 vs. 44 ± 11 out of 100 points; difference, 4 [0 to 8]; P = 0.073). Having removed the variability due to scenarios (analysis of covariance), the P value improved (P = 0.014; table 3).

The absolute interclass correlation coefficients for the two independent assessments of Ottawa scale performance was 0.90 [0.85 to 0.93] for single measures and 0.95 [0.92 to 0.96] for average measures. The Ottawa scale score was 11% higher in the Tactics to Optimize the Potential group (29 ± 6) than in the control group (26 ± 6 out of 42 points; difference, 3 [1 to 5]; P = 0.011). Having removed the variability due to scenarios, the P value improved (P = 0.004, table 3).

The absolute interclass correlation coefficients for the two independent assessments of Team Emergency Assessment Measure performance was 0.92 [0.89 to 0.94] for single measures and 0.96 [0.94 to 0.97] for average measures. The score was 8% higher in the Tactics to Optimize the Potential group (39 ± 9) than in the control group (36 ± 8 out of 54 points; difference, 3 [0 to 6]; P = 0.049). Having removed the variability due to scenarios, the P value improved (P = 0.018; table 3).

The Ottawa scale score was correlated with the Team Emergency Assessment Measure score (r = 0.62; P < 0.0001).

There was no significant difference between groups with regard to the VAS of stress and State-Trait Anxiety Inventory state before specific preparation (table 1). Immediately after specific preparation, the median VAS of stress was 17% lower in the Tactics to Optimize the Potential group (52 [42 to 64]) than in the control group (63 [50 to 73]; difference, −10 [−16 to −3]; P = 0.005; table 2). The analysis of VAS of stress within groups showed a reduction of VAS of stress after intervention in both groups that was greater in the Tactics to Optimize the Potential group (table 4). No difference was observed between groups for subsequent VAS of stress. The VAS–stress reduction and VAS–simulation performance were higher in the Tactics to Optimize the Potential group as compared with the control group (table 2).

This study shows that Tactics to Optimize the Potential, like a stress management training, improves the overall performance of residents during simulation by 9%. The associated quantitative effect size showed a magnitude of the phenomenon corresponding to a medium effect, which provides more significance.^29–31  One might question the practical implication of this increase in overall performance. The clinical importance of each additional performance point might vary depending on the scenario. However, it seems reasonable that an improvement in overall performance might be of interest if the results were applicable to the clinical setting. The increased performance score for residents in the Tactics to Optimize the Potential group provides new evidence for the use of cognitive stress management training to enhance participants’ performance.

With respect to secondary endpoints, the two nontechnical skills evaluation scales reported better scores within the Tactics to Optimize the Potential group. Before analysis of covariance, no significant difference was observed for the clinical performance between groups. Assuming that the reactivation was effective in reducing residents’ stress, one might conclude that the clinical performance is less affected by stress, and therefore less amenable to the benefits of Tactics to Optimize the Potential. Indeed, this stress management training cannot compensate for a lack of clinical knowledge, but it might help to mobilize resources already stored in memory. As opposed to clinical knowledge, nontechnical skills seem to be more influenced by Tactics to Optimize the Potential. Nontechnical skills refer to the “cognitive, social, and personal resource skills that complement technical skills and contribute to safe and efficient task performance.”³²  These skills include teamwork, communication, leadership, decision making, and situational awareness, and are increasingly considered for minimizing adverse events resulting from errors and poor performance.³³  Adverse events occurring in the operating room are caused not only by poor technical performance but also by deficits in nontechnical skills.^34–36  Previous studies reported a decrease in nontechnical skills with stressors or higher stress level,^6,11  and another study reported specific preparation related improvement of nontechnical skills during simulation.²²  The choice of using two different validated scales was justified by the subjectivity of qualitative assessment: the Ottawa scale would be more appropriate for the leader assessment and the Team Emergency Assessment Measure scale for the teamwork assessment. However, as the embedded nurse was acting as neutrally as possible, the two scales were used to perform a more objective assessment of nontechnical skills of the resident, and given approximately the same weight as the technical skills component in the overall performance score. Observing a score improvement in both scales, with heavy correlation between them and excellent inter-rater reliability (independent and blinded), provides strong evidence of the positive impact of the Tactics to Optimize the Potential.

Exploratory stress variables highlight the effects associated with Tactics to Optimize the Potential. First, no difference in anxiety scale was observed between groups before the reactivation. Therefore, Tactics to Optimize the Potential did not affect stress level or anticipatory anxiety before simulation. Despite the small effect size observed for the VAS of stress after intervention between and within groups, the VAS of stress after specific preparation showed a more significant stress reduction in the Tactics to Optimize the Potential group. These reductions might reflect the perceived beneficial effects of this stress management training before intervention, perhaps via better regulation of sympathetic activity. Further exploration of Tactics to Optimize the Potential effects may confirm the proposed mechanism of down-regulation of the amygdala observed with functional magnetic resonance imaging by other cognitive therapy.³⁷  The amygdala, part of the limbic system, is known to control the sympathetic nervous system.⁷  Stress activation–induced burst activity from the amygdala may be attenuated by this stress management training.

Tactics to Optimize the Potential as a stress management training was created by E. Perreaut-Pierre and was reported to reduce stress assessed by the perceived stress scale in a military setting.⁴  However, no evaluation of technical and nontechnical skills was reported. Autonomous practice of Tactics to Optimize the Potential as a stress management training was assessed in that study which reported poor compliance with no regular practice. In the current study, no assessment of daily practice training was performed, but it seems reasonable to expect greater benefits would result from regular practice. Another study in a military setting reported that a stress management training was effective in reducing stress with enhanced technical skills during simulated first aid.³⁸  The specific preparation was training in biofeedback and breathing during stress induced by a three-dimensions immersion first-person shooter video game. During subsequent simulation, a lower salivary cortisol response to stress was observed and the perceived ability to self-control stress was better in the stress management training as compared with the control group.

The current study provides new evidence for Tactics to Optimize the Potential as a toolbox to help practitioners structure stress preparedness in order to cope with difficult clinical situations. This is not a “stress inoculation” training technique to accustom practitioners to higher stress levels, but rather a toolbox to prevent, to anticipate, and to decrease a priori the consequences of stress activation. Therefore, a Tactics to Optimize the Potential intervention focusing on primary prevention of the negative effects of stress is an additional and nonexclusive strategy to be considered along with other existing options such as stress inoculation training and other stress management training. The potential of stress management training to reduce the incidence of stress-related long-term conditions while increasing well-being at work remains to be explored in further studies. The impact of excessive stress can lead to burnout, which is observed in up to 40% of intensive care and anesthesia practitioners, particularly younger clinicians working unconventional schedules.³⁹  Therefore, clinicians working in stressful medical disciplines with daily exposure to life-threatening situations should develop a mastery of stress management training. Tactics to Optimize the Potential also provides tools to improve recovery, especially sleep debt, using relaxation breathing (appendix 1), that may be useful for caregivers participating in overnight and weekend shifts.

This study has some limitations. The choice of a composite primary endpoint may be seen as a limitation. No standardization of the clinical competency of residents was performed before simulation. Therefore, to formally conclude about the effect of Tactics to Optimize the Potential on clinical performance, additional studies requiring prior standardization of residents’ clinical performance before simulation must be done. The choice of the two scales used to measure nontechnical skills merits discussion. As many different scales have been reported in the literature, one might suspect that different scores could have been observed with different scales.^40,41  One might acknowledge that even if the two nontechnical skills scales are somewhat different, they are partly composed of elements that could be measuring the same behavior and would be “double counted” in their presence or absence. This choice may have influenced the overall performance score.

Given that no difference in stress level could be detected between the two groups before participation in the scenario, it is possible that the reactivation alone might have led to attenuation of the stress response. Despite standardization of each specific preparation, there could be an associated bias in how the control group was treated as compared to the Tactics to Optimize the Potential group. The control intervention might be seen as a non–null event generating some stress and frustration in the residents. Except for the VAS of stress, which was also reduced to a lesser extent in the control group, no specific detailed data were collected to confirm this hypothesis. However, this control intervention has been used previously and was designed to mentally stimulate participants of the control group in a standardized and realistic manner.²²  Importantly, the 5 min required to reactivate Tactics to Optimize the Potential may prevent healthcare providers from doing the reactivation in unexpected situations where immediate care delivery is indicated. Nevertheless, many clinical situations might benefit from the reactivation without delaying care. These may include paramedics or medical assisted transport on the way to the patient, the trauma team waiting to receive a patient, expected difficult airway management, unusual or difficult clinical situation, or a complex procedure. Residents’ frequency of practicing Tactics to Optimize the Potential in the interval between training and simulation was most likely heterogeneous. Moreover, residents may have been more influenced by some specific tactics over others. Therefore, the specific reactivation could have been tailored to residents’ preference in order to potentially achieve a better performance. Finally, the reactivation-to-performance timing should be explored in order to optimize the timing applicability of Tactics to Optimize the Potential. Likewise, the training timeframe was arbitrarily decided between training instructors and the residents’ curriculum coordinators to better fit with residents’ schedule constraints and specific objectives of this training. The optimal modulation of Tactics to Optimize the Potential training (items and duration) in a competency-based learning strategy remains to be explored as it could influence the reactivation. A specific study should be performed to address this point because a shortening of the preparation interval might enhance the efficacy of this stress management training.

In conclusion, an increase of 9% in overall performance, probably resulting from a significant increase in the nontechnical skills scores, was observed during high-fidelity simulation after residents were trained in stress management techniques with Tactics to Optimize the Potential in addition to immediate Tactics to Optimize the Potential reactivation before the scenario. Confirming the effects on specific performance areas would be relevant before implementation as part of global team training for critical situations where 5 min of reactivation is feasible before delivering specific care. The exploration of Tactics to Optimize the Potential effects on patient care delivery and on team wellness might be particularly relevant.⁴² 

The authors thank: Jacques Escarment, M.D., Ph.D., (Regional Direction of French Military Health Service) for his support and encouragement for the Tactics to Optimize the Potential training study (Lyon, France); Carole Sage for her help with scheduling residents for Tactics to Optimize the Potential training and for simulation, Secretary of the Anesthesia Department (Bron, France); Darren Raphael, M.D., University of California Irvine Medical Center (Orange, California); Christopher Blakeley, M.D., Ph.D., Emergency Department, Westnine Medical Ltd. (London, United Kingdom); Pierre Duhamel, English teacher (Le Havre, France) and Malcolm Bickford, English native (Paris, France) for the substantial English revision of the manuscript; Pascal Roy, M.D., Ph.D., Hospices Civils de Lyon, Health Data Department (Lyon, France), Sophie Schlatter, M.P.H., Motricity Biology Inter-university Laboratory, University Claude Bernard Lyon 1 (Lyon, France), and Baptiste Balança, M.D., Ph.D., Hospices Civils of Lyon, Pierre Wertheimer Hospital, Anesthesia and Intensive Care Department (Bron, France); Inserm U1028, CNRS UMR 5292, Lyon Neuroscience Research Centre, TIGER Team (Lyon, France) for the substantial statistical help; and the students, simulation instructors, and simulation technicians (Lucas Denoyel and Sebastien Sygiel) of Université Claude-Bernard Lyon 1 Simulation Center (Lyon, France).

Support was provided solely from institutional (Université Claude Bernard Lyon 1) and/or departmental sources (Hospices Civils de Lyon, Departments of Anesthesia and Intensive Care, Lyon, France).

The authors report no competing interests.

Full protocol available at: marclilot@hotmail.com. Raw data available at: marclilot@hotmail.com.

“I feel in very good shape.

Able to give the maximum of myself for the following critical situation during high fidelity simulation.

I am calm and full of energy, ready to react to any stimulation.”

“You figure out the gestures, the sequences, the delicate phases, to anticipate the actions of the partners, anticipate different strategies… anticipate the unpredictable…

Get ready for the unexpected…

All this with the required psychologic qualities: energy, dynamism, efficacy, serenity and the pleasure of giving the maximum of yourself.”

“Now I feel in full control of my capacities, technically ready…

Physically at the maximum…

I’m ready!”

Revitalizing breathing is based on inspiration lasting four times longer than expiration. It was part of the tactics to optimize potential reactivation and was performed during 1 min just after the mental reactivation.