Impact Assessment of Perioperative Point-of-Care Ultrasound Training on Anesthesiology Residents

Point-of-Care (POC) ultrasound has been defined as portable ultrasound brought to the patient and performed “real time” by the provider.¹  Anesthesiologists have been leaders in the use of ultrasound technology for intraoperative transesophageal echocardiography,²  central vascular access,³  and regional anesthesia.⁴  Recently, the use of POC ultrasound has dramatically expanded in the areas of critical care,^5–7  surgery,⁸  and emergency medicine,⁹  and it is now clear that POC ultrasound has the potential to help the perioperative physician with far more than central venous access and regional anesthesia.^10,11  Specific POC ultrasound topics that may aid perioperative patient care include (1) cardiac, (2) pulmonary, (3) hemodynamics, (4) abdominal, (5) airway, (6) vascular access, and (7) intracranial pressure (ICP) assessment.^10–12  Because anesthesiologists redefine their role as leaders in coordinating care for surgical patients in the perioperative surgical home (PSH) model,^13,14  there is an urgent need to revise the current curriculum and include topics that are more relevant to the entire spectrum of perioperative care. Teaching anesthesiology residents on the use POC ultrasound is one way we can prepare anesthesiology trainees for this change in the perioperative environment.

However, training in the perioperative use and interpretation of POC ultrasound is challenging,¹⁰  and one is faced with the task of developing an effective education curriculum. Previously, simulation training has been demonstrated to improve acquisition and knowledge of transesophageal echocardiography use among anesthesiologists.^10,15–17  Indeed, we have also recently reported preliminary data that demonstrates the effectiveness of a simulation/model approach for resident education on POC ultrasound.¹⁸ 

The goals of this Foundation for Anesthesia Education and Research educational project were to develop a novel perioperative POC ultrasound curriculum for resident training and to assess the utility of a model/simulation-based education strategy for training anesthesiology residents on this curriculum. We submit that ultimately the objective of any clinical resident training curriculum should be improvement in patient care. As such we have decided a priori to use the Kirkpatrick four-stage model of evaluation,¹⁹  which is an assessment tool that is widely used to evaluate training. The Kirkpatrick instrument is a well-recognized measure that is used to evaluate the effectiveness of a new educational intervention on four ranked outcome levels: (1) the participants’ affective responses to training content (reaction and satisfaction), (2) the impact of the training itself on improving knowledge (learning and performance in a test), (3) the application of the new information (behavior and execution of the intervention), and (4) use of the new training to improve career-related activities (clinical impact on the organization).¹⁹  Although ideally any new educational intervention should achieve level 4 in Kirkpatrick instrument, this is not easily achievable. For this study, we hypothesized a priori that implementation of an innovative ultrasound curriculum would significantly improve resident training and organizational outcome as measured through the Kirkpatrick instrument, with our primary outcome being the impact of POC ultrasound on the clinical care in the organization (Kirkpatrick level 4).

The University of California Irvine Internal Review Board (#2012–8826) approved the study, and participating residents provided written informed consents. Support for the study was provided by a Research in Education Grant from the Foundation for Anesthesia Education and Research. This prospective study included 42 residents enrolled in clinical anesthesia (CA) years 1 to 3 at University of California-Irvine. All residents received the educational curriculum during the 2-yr study period and were enrolled in the study. However, four residents (two during study year 1 and two during study year 2 were excluded from testing secondary to having received previous ultrasound education in the topics listed in this curriculum and one resident was excluded (during study year 2) secondary to having an extended maternity leave, leaving the final number to 37. The study was extending over a period of 2 yr.

Development of the POC ultrasound curriculum for this study began with the creation of a task force of anesthesiologists, intensivists, and simulation experts (attending faculty who had received institutional certification and are certified simulation instructors for the American Board of Anesthesiology) approximately 1 yr before the study start date. This task force decided a priori that all development decisions would be data driven and based on the most current and state-of-the-art research. To develop the simulation topics, the task force conducted an extensive literature search and critical analysis and consulted with experts of other specialties (emergency medicine and critical care) on POC ultrasound. Construction of the curriculum by the taskforce was guided by the six Accreditation Council for Graduate Medical Education core competencies, offering residents a formal simulated experience to perform and practice behaviors necessary for patient care, medical knowledge, learning improvement, system-centric attentiveness, professionalism, and interpersonal skills and communication. The task force formulated the clinical objectives for the total body perioperative ultrasound examination (Focused periOperative Risk Evaluation Sonography Involving Gastroabdominal Hemodynamic and Transthoracic ultrasound [FORESIGHT]) with the following being the main areas of the curriculum: (1) cardiac, (2) pulmonary, (3) hemodynamics, (4) abdominal, (5) airway, (6) advanced vascular access, and (7) ICP assessment (fig. 1).

The focus on cardiac ultrasound was chosen because of the incidence of cardiac events in the perioperative setting and because of their potential impact on patients’ outcome. In addition, transthoracic examination of the cardiopulmonary system using bedside POC ultrasound technology has proven to be a reliable tool when compared with formal echocardiography²⁰  and can be taught to noncardiologist.^21,22  Recently, guidelines have been published for POC cardiac ultrasound by noncardiologists for the intensive care setting.²²  Considering the similarity between the intensive care unit and the operating room, the curriculum incorporated similar guidelines.

The high incidence of events involving the lung and the pleura in the perioperative period as well as the potential impact of these events on patients’ outcome lead us to incorporate pulmonary ultrasonography in the curriculum. Ultrasonography has been shown to be more accurate than auscultation or chest radiography for the detection of pleural effusion, consolidation, and alveolar interstitial syndrome in the critical care setting.^23,24  POC ultrasound has also proven to be a valuable tool for the detection of pneumothorax.^25,26  Consequently, assessment of pneumothorax, evaluation of air space disease, and evaluation of pleural effusion were included in the curriculum.

POC ultrasound incorporates several modalities that allow determining the ventricular filling pressures and fluid responsiveness, which are frequent concerns in the perioperative setting. Specifically, the collapsibility of the inferior vena cava and left ventricular (LV) end-diastolic area have been shown to be accurate measurements of reduced filling pressures^8,27–29  and were incorporated in the curriculum together with dynamic predictors of fluid responsiveness using ultrasound.^30,31 

Recently, POC ultrasound performed by anesthesiologists at the bedside has been used to assess gastric content and volume,^32,33  and a recent grading system based exclusively on the qualitative sonographic assessment of the gastric antrum has shown strong correlation with gastric volume.³³  Given the critical importance of gastric content to perioperative physician to prevent aspiration, we decided to incorporate this teaching in the curriculum. The Focused Assessment with Sonography for Trauma³⁴  is the most studied example of focused clinical ultrasound in trauma care⁸  and was also incorporated in the curriculum. For the perioperative physician who may be involved with the PSH, the application of this examination allows one to determine whether hemodynamic instability is secondary to injury of the pericardial and/or peritoneal space resulting in free fluid that can occur before postoperatively.

Unrecognized malposition of the endotracheal tube (ETT) can lead to severe patients’ complications and death.^35,36  The use of POC ultrasound for adjunct confirmation of tracheal versus esophageal intubation has been recently demonstrated,³⁷  and a recent study showed successful ability of POC ultrasound to verify correct ETT position in the trachea.³⁸  For these reasons, this technique was added to the curriculum.

The use of ultrasound to aid with vascular access has advanced beyond its now wide spread use for central venous access. Specifically, ultrasound has proven to reliably aid in the placement of difficult peripheral intravenous catheters^3,39,40  and intraarterial catheters.^41,42  The utility for this skill set is obvious to perioperative medicine because these are procedures that are performed every day, and this was part of the curriculum.

POC ultrasound has been shown to provide rapid assessment of elevated ICPs based on the assessment of optic nerve sheath diameter.⁴³  The relationship between the optic nerve sheath diameter and ICP has been well established. Because of the potential impact of increased ICP on patients’ outcome in the perioperative setting, it was decided to add this assessment to the curriculum.

After the curriculum was designed by the taskforce, baseline knowledge on the topics involved was assessed using multiple-choice questions. The pretraining examination (n = 37) consisted of 60 multiple-choice questions aimed to cover the following categories of the curriculum: Physics, Volume Status and Mechanisms of Hypotension, Cardiac, and Pulmonary Functions, and it was graded on a 0 to 100 scale (See Supplemental Digital Content 1, https://links.lww.com/ALN/B174). Excluding central venous access, no resident had received any education on the curriculum topics listed for this study during their residency period. Given the significant cost and resources for human model examinations and the fact that no resident had any significant hands-on training, it was decided not to perform pretraining model examinations and limit this phase to multi-choice questions.

Construction of this, 21-point, evidence-based, curriculum (fig. 1) followed the six Accreditation Council for Graduate Medical Education core competencies, offering residents a formal simulated experience to perform and practice behaviors necessary for patient care, medical knowledge, learning improvement, system-centric attentiveness, professionalism, and interpersonal skills and communication. A model/simulation-based education strategy was used for the curriculum lectures. This strategy was based on the previous departmental research showing that a model/simulation teaching strategy is more effective than traditional didactics for educating POC ultrasound.¹⁸  For the first year of the study, residents received a weekly 20-min focused lecture on 1 of the 21 objectives of the FORESIGHT examination listed in figure 1. Each lecture was immediately followed by a 25-min human model or simulation practice period. For the second year of study, the curriculum was adjusted to a 2.5-h monthly session, with approximately 60 min of lecture followed by 1 to 1.5 h of model/simulation practice. Simulation devices were used for the cardiac and venous access topics. In addition, at the end of each category of the curriculum, (1) cardiac, (2) pulmonary, (3) hemodynamics, (4) abdominal, (5) airway, (6) advanced vascular access, and (7) ICP assessment, a 10-min clinical scenario was simulated to emphasize the key components of the topic. The curriculum was structured to repeat every 6 months, for both years, so all topics were covered four times during the 2 yr of the study protocol (fig. 2). All residents experienced the same education curriculum outside of the pathology training.

For the study year 1, written and model-based examinations were conducted for CA-1/CA-2 residents at 12 and 18 months. These examinations occurred at the 11th month of the study for the study year 1 CA-3 residents and at the 24th month of the study for the study year 2 CA-1 residents (fig. 2). In addition, departmentally supported nonclinical time was used to conduct a perioperative ultrasound teaching service, providing supplemental hands-on training for the residents in the perioperative and intensive care setting. This service allowed for a faculty anesthesiologist experienced in POC ultrasound to lead POC ultrasound rounds every weekday morning with residents assigned to a designated perioperative POC ultrasound rotation as well those on research, postanesthesia care unit, and preoperative clinic rotations. Proficient attending faculty (n = 4) who were eligible to teach this curriculum and lead the service were defined as those who had personally performed at least 50 complete FORESIGHT examinations.

Vivid S6 (GE Healthcare, Norway) systems equipped with a linear (12 MHz), curved linear (5 MHz), and phased array (1.5 to 4 MHz) transducers were used for clinical examinations and teaching. Each lecture used instruction with a live human model and/or simulation mannequins (Blue Phantom, CAE Healthcare, USA), relevant to the particular objective goal. In addition, a transthoracic echocardiography simulator (HeartWorks, Inventive Medical, Ltd., United Kingdom) was used for instruction of relevant objectives as well.

CA-3 residents were evaluated for 9 months before graduating from the Anesthesiology program. Similarly, incoming CA-1 residents were evaluated for the last 12 months of the study because of matriculation into the program during the second year of the study. The remaining residents (CA-1/CA-2) were evaluated more than 21 months. For clarity, the timeline of this study (from curriculum development to evaluation of clinical transferability) is presented in figure 2. The curriculum was evaluated through Kirkpatrick levels 1 to 4 outcomes. The primary outcome was assessment for clinical impact of curriculum training (Kirkpatrick level 4). Secondary outcome markers were residents’ satisfaction surveys (Kirkpatrick level 1), assessment of curriculum content retention (Kirkpatrick level 2), impact of pathology specific training (Kirkpatrick level 2), and content application, image acquisition (Kirkpatrick level 3). All residents (n = 42) received the curriculum education; however, four residents (two during year 1 of the study and two during year 2 of the study) were not included in Kirkpatrick level 2 to 4 testing secondary to having previous ultrasound training and one was excluded (during study year 2) secondary to have an extended maternity leave, leaving 37 residents who completed the posttraining evaluation.

• Formative evaluation of curriculum by residents (Kirkpatrick level 1): Assessment of the curriculum content was measured by resident surveys for the first year of study in which they provided ratings of the quality of POC ultrasound instruction, as well as the relevancy to anesthesiology training. All residents for the first year of the study (n = 30) who were enrolled in the curriculum completed the survey, including the two residents who were excluded in Kirkpatrick level 2 to 4 testing secondary to their previous ultrasound training. The survey was based on eight questions (table 1), and each question was answered with the previously published and validated five-item Likert item scale^44,45 : strongly disagree, disagree, undecided, agree, and strongly agree.

• Content retention assessment (Kirkpatrick level 2): Secondary outcome of content retention was assessed using a before and after study design. After assessment of baseline knowledge at the beginning of the study period), evaluation of content retention (Kirkpatrick level 2) was assessed by comparison of multiple-choice questions (pretraining vs. posttraining) rated on a percent correct, 0 to 100 scale at the end of year 1 and 2. The posttraining tests were divided into four tests to cover the same topics and consisted of similar concept questions as the pretest (n = 37; Supplemental Digital Content 2, https://links.lww.com/ALN/B175).

• Pathology training randomization (Kirkpatrick level 2): After completion of year of training, residents were randomized, using an online random number generator⁴⁶  to receive either additional ultrasound training involving pathologic findings or the intervention outlined in figure 2. The additional training session was a video-based lecture showing clinical pathologic ultrasound examinations with specific topics including (1) hypovolemia, (2) abdominal bleeding, (3) LV failure, (4) right ventricular failure, (5) severe aortic stenosis, (6) pneumothorax, (7) severe pulmonary edema, (8) increased gastric volume, (9) esophageal intubation, and (10) increased ICP. The additional pathologic findings training session was planned before the final 3 months of the first year of the study. An image- or video-based pretest was performed by all residents. If the test topic had the possibility of degrees of severity, they were tested by having to rank pathologic images from least to worst (ranked numerical score). If the test topic was the presence or absence of pathology, the resident would answer yes/no (scored as 0 or 1) to the presence of pathology in the image/video. During the posttraining period, residents performed the same test again and all posttraining examinations occurred within 1 month from receiving training (n = 37).

• Content application/image acquisition assessment (Kirkpatrick level 3): This evaluation was conducted using a live human model examination at the end of the first and second years of the study. The examination was divided into the following main POC ultrasound topics: (1) volume status, (2) cardiac, (3) pulmonary, (4) vascular access, and (5) additional topics, which included gastric volume assessment, ETT location, and ICP assessment. Examinations were graded on anatomy identification, image quality, and image acquisition time using a numerical point system (Supplemental Digital Content 3, https://links.lww.com/ALN/B176). Image quality was rated on the following scale: 1 = no image; 2 = poor and unusable image quality; 3 = usable image quality; 4 = good image quality; and 5 = perfect image quality. An expert examiner, who was blinded to resident’s multiple-choice performance, graded all model examinations. The examiner scored as per the scale shown in Supplemental Digital Content 3, https://links.lww.com/ALN/B176, which was decided after determining agreement of the scoring scale by several experienced sonographers.

• Transferability to clinical management (Kirkpatrick level 4): After 1 yr of curriculum training, a clinical report (Supplemental Digital Content 4, https://links.lww.com/ALN/B177) was created to assess the clinical use of POC ultrasound in the perioperative setting. These reports were captured for residents who had completed the first year of training and whom had verification of completing at least 10 complete FORESIGHT examinations. Patients were identified at the request of the primary anesthesia provider to aid with clinical assessment and management of their patients in the preoperative, postoperative, or intraoperative setting. If the primary anesthesia team believes that they could get benefit from any component of the FORESIGHT examination, they would contact the resident or attending physician on the perioperative ultrasound service. As stated in the study protocol (fig. 2), the perioperative ultrasound service was implemented in August 2013 and was advertised to the department at several grand round meetings before this date. The clinical report contained diagnoses that were addressed by the ultrasound curriculum (Supplemental Digital Content 4, https://links.lww.com/ALN/B177); residents would document the preliminary diagnosis suggested by the primary anesthesia team, then perform a relevant ultrasound examination based on training from the FORESIGHT curriculum, and confirm/change the preliminary diagnosis based on the ultrasound findings. An experienced sonographer was present to confirm the results of the POC ultrasound examination. Reports were generated only for examinations in which there was an agreement between the resident and the experienced sonographer. Specifically, when there was an agreement, the findings were disclosed to the primary anesthesia team after which the resident physician interviewed the primary anesthesia team and then completed the report. For the report, the resident physician sonographer documented if the POC ultrasound examination affected clinical management of patient (after discussing findings with primary anesthesia team), and if answer was “yes,” then they would detail why it affected management: (1) obtained new pathology diagnosis, (2) verified current pathology, and (3) reassured by normal findings. Also, if the examiner identified a new diagnosis, it was categorized into a component of the FORESIGHT examination: (1) cardiac, (2) pulmonary, (3) abdominal, (4) vascular access, (5) ICP assessment, and (6) airway. These data were evaluated for clinical application of the curriculum by assessing the percentage of which POC ultrasound use impacted clinical care. Clinical examinations were performed from September 2013 to March 2014.

To assure confidentiality, all residents were assigned at the onset of the study a three-digit number that was used for all surveys, examinations, and analysis, with the first number identifying the CA-year of the resident (for data analysis) and the second two numbers assigned randomly.

1. Formative evaluation of curriculum by residents (Kirkpatrick level 1): POC ultrasound curriculum was evaluated by resident surveys at the end of the first year of the study, and data are reported as percent of the respective answer choices.

2. Content retention assessment (Kirkpatrick level 2): Content retention of the POC ultrasound topics of (1) ultrasound physics, (2) cardiac, (3) pulmonary, and (4) mechanism of hypotension/volume status were assessed via comparison of multiple-choice test (pretraining vs. posttraining).

3. Pathology training assessment (Kirkpatrick level 2): Training impact was assessed by comparing pretraining with posttraining score improvement between those who received additional pathologic findings training and those who did not. Effects of pathology training and class year on posttraining scores were tested with ANCOVA, including pretraining test scores as a covariate.

4. Content application/image acquisition assessment (Kirkpatrick level 3): All residents performed posttraining model examinations at the end of year 1 and 2 of the study. Descriptive statistics on anatomy identification, image quality, image acquisition time, and anatomy identification were performed at the end of year 1 and 2. This examination assessed anatomy identification, image quality, and image acquisition time, for each of the five main topics of model examinations: (1) evaluation of volume status, (2) cardiac, (3) pulmonary, (4) vascular access, and (5) additional topics (gastric antrum, ETT placement, and optic nerve diameter).

5. Transferability to clinical management (Kirkpatrick level 4): The primary endpoint was detection of a significant clinical impact of POC ultrasound after 1 yr of curriculum training. Previous studies have shown an incidence of unexpected pathology in the perioperative setting of 10 to 27%.^47,48  On the basis of the assumption that we expected the clinical impact of this curriculum training to be able to detect undiagnosed pathology at an incidence of 20%, we calculated the number of examinations needed to be 142, assuming a power of 0.8 and a level of significance of 0.05. Descriptive statistics were performed on the clinical report data. Specific data points of interest were (1) incidence of POC ultrasound affecting clinical management, (2) incidence of POC ultrasound obtaining new diagnosis, and (3) specific POC ultrasound topics that affected management.

All numerical data are presented as mean ± SD. Continuous data were compared using Wilcoxon (for paired comparisons) or Mann–Whitney (for unpaired comparisons) U tests as appropriate; all tests were two tailed. Proportions were compared using the Fisher exact test. A value of P < 0.05 was considered significant. Data were analyzed using SPSS 14.0 (SPSS, Inc., USA).

Residents positively reported “agree” or “strongly agree” to all survey questions for more than 90% of the time. The most positive response from the survey was reported for the question asking if the areas of ultrasound education taught in this curriculum were relevant for future anesthesiologists (>73% strongly agree and >23% agree). Further results related to the survey are shown in table 1.

Statistically significant higher scores were observed, for each resident class and overall, on all postlecture multiple-choice categories: (1) physics, (2) volume status, (3) pulmonary, and (4) cardiac. When compared with prelecture scores for both years of the study (fig. 3; n = 37), there was no difference noticed between resident class and posttest scores. No comparison was performed for the additional topics category, as the pretests did not contain questions on the topics in this section. Combined average scores for all tested categories showed significantly improved scores after training (48 ± 16 to 69 ± 17%, P < 0.0001), For year 1, the topic of pulmonary ultrasound showed the greatest improvement in percent correct scores (30 ± 16 to 71 ± 14%, P < 0.0001), followed by cardiac (43 ± 13 to 65 ± 14%, P < 0.001), volume status (55 ± 15 to 75 ± 15%, P < 0.0001), and physics (58 ± 10 to 71 ± 14%, P = 0.0033). Average scores improved from year 1 to 2 (69 ± 15 to 74 ± 14%, P = 0.0168). For year 2, the topic of cardiac ultrasound showed the greatest improvement (65 ± 14 to 74 ± 11%, P = 0.0036).

The pathologic finding training postlecture test scores were also higher in every residency class for those who were randomized to receive additional pathology training (n = 18) compared with those who did not receive the pathology training (n = 19; table 2). Higher class years that received pathology training showed more improvement in posttraining scores, but this was not significant (P = 0.06). Statistically significant differences were observed for: (1) LV systolic failure assessment (P < 0.0001), (2) RV systolic failure assessment (P = 0.0008), (3) assessment of aortic stenosis (P < 0.0001), (4) assessment of pulmonary edema (P = 0.0006), (5) gastric food content (P = 0.0002), and (6) increased ICP assessment via optic nerve diameter (P = 0.0007).

Summary descriptive statistics and analysis for model examinations are listed in table 3, and complete details are listed in Supplemental Digital Content 3, https://links.lww.com/ALN/B176. Residents showed improvement between year 1 and 2 in anatomical identification, image quality, and image acquisition time for all the five major categories: (1) evaluation of volume status, (2) cardiac, (3) pulmonary, (4) vascular access, and (5) additional topics (gastric antrum, ETT placement, and optic nerve diameter). Statistically significant improvements were as follows: (1) correct percentage anatomy identification for pulmonary (93 to 100%, P = 0.0499) and vascular access (83 to 92%, P = 0.0121), (2) image quality (0 to 5 scale) for cardiac (3.7 ± 0.5 to 4.22 ± 0.51%, P = 0.0046), evaluation of volume status (3.55 ± 0.73 to 4.44 ± 0.56%, P = 0.0016), and additional POC ultrasound topics (0 to 5 scale) (2.69 ± 0.86 to 3.17 ± 1.11%, P = 0.0498), and (3) image acquisition time for vascular access (25.24 ± 11.9 to 18.28 ± 12.2 s, P = 0.0409).

A total of 224 POC ultrasound examinations were performed on 150 patients during the study time period. The majority of examinations were performed in the operating room (n = 89; 60%), followed by preoperative care unit (n = 29; 20%), postanesthesia care unit (n = 18; 12%), and other locations (intensive care unit, obstetrics, radiology [n = 13; 13%]). Baseline characteristics for patients whose demographic information was captured (n = 63) as well as surgery classifications are listed in table 4. The main trigger for POC ultrasound examination was significant medical history (51%), followed by hemodynamic instability (14%), respiratory failure (13%), ETT location verification (13%), and peripheral venous access (9%). The primary anesthesia team reported that the POC ultrasound examination changed management in 76% of the cases. When asked how the POC ultrasound examination changed management, 31% responded that it was secondary to new diagnosis, 45% responded that it helped by verifying current known diagnosis, and 24% responded that it aided by confirming normal findings. New diagnoses were reported when the POC ultrasound examination results indicated findings that were new to the primary anesthesia team. Cardiac POC ultrasound was performed most often (39%), followed by pulmonary (21%), peripheral venous access (16%), ETT location (11%), abdominal (9%), and ICP assessment (9%; table 5). Cardiac POC ultrasound was performed most often and subsequently had the highest absolute number of new diagnosis (76%). When looking at the proportion of new diagnosis for each POC ultrasound topic, new diagnoses were found most often with abdominal (70%), followed by pulmonary (49%), cardiac (38%), ETT location (36%), and ICP assessment (22%). Further details of the specific findings of the new diagnoses are listed in table 5.

The PSH concept has been recently introduced in the United States.^13,14  This model, which extends the role of the anesthesiologists, will necessitate changes in the current anesthesiology curriculum. Under the conditions of this study, we found that (1) a “whole-body” POC ultrasound curriculum has a high degree of resident satisfaction (Kirkpatrick level 1), (2) use of a model/simulation learning strategy effectively trains anesthesiology residents on the FORESIGHT examination topics (Kirkpatrick level 2), (3) additional pathology training improves resident’s knowledge (Kirkpatrick level 2), (4) most components of the FORESIGHT examination can be performed in a relatively short time and residents can obtain adequate image quality/identify anatomy after 1 yr of training (Kirkpatrick level 3), (5) application of the perioperative FORESIGHT examination, at a major academic center, results in some degree of clinical impact with a substantial portion being secondary to new diagnoses (Kirkpatrick level 4).

This curriculum sought to cover common critical issues that are faced in the perioperative setting. With this in mind, we included significant components of transthoracic, abdominal, neurologic, vascular, and pulmonary ultrasound examinations. This examination was not meant to be compared with formal vascular, neurologic, transthoracic, pulmonary, or abdominal sonography. Rather, this examination introduces a new modality for the anesthesiologist to use POC ultrasound to help rapidly assess acute issues that occur in the perioperative setting. By using the FORESIGHT ultrasound examination, the anesthesiologist could quickly assess an unstable patient, evaluate a relatively broad differential diagnosis, and tailor treatment to the determined pathology.

Simulation training has gained interest as an effective method of education for anesthesiology residents.^17,49  This study builds from a preliminary study,¹⁸  showing that a model/simulation education strategy improved training more than standard didactic education. Regarding POC ultrasound training, use of simulation has proven to effectively teach transthoracic echocardiography⁵⁰  and regional nerve blocks to anesthesiologists.^51,52  Our study further supports the concept of educating POC ultrasound via a model/simulation strategy and expands its support to areas of POC ultrasound that have not been demonstrated before. To the best of our knowledge, assessment of a strategy to educate anesthesiology residents on the POC ultrasound topics of ICP, Focused Assessment with Sonography for Trauma examination, pneumothorax, ETT location, and gastric volume has not been previously shown. Our results suggest that anesthesiology residents’ knowledge on pulmonary ultrasound may be the easiest to improve, followed by cardiac, and then volume status. Understanding of physics showed a relatively high baseline score, which may be secondary to training on the use of ultrasound for central venous access that was integrated into the resident education curriculum before this study onset. Confirmation of interest for both the curriculum topics and its educational strategy was achieved with high survey scores. In addition, this study suggests that including video-/image-based pathology lectures improves content retention.

Our results highlight that the FORESIGHT examination can be performed relatively quickly and suggest that anesthesiology residents can successfully acquire images of useable quality after 1 yr of training. Among the main FORESIGHT topics, peripheral vascular access showed the highest image quality and greatest improvement in acquisition time from year 1 to 2. This is expected as anesthesiology resident training already includes ultrasound for central venous access.⁵³  Pulmonary ultrasound was the next highest category for image quality acquisition followed by cardiac, evaluation of volume status, and finally the additional topics (gastric antrum, ICP assessment, and ETT location). Given the fact that the topics under “additional areas” were the most novel of the FORESIGHT examination and therefore the most unfamiliar to anesthesiology residents, it is understandable that it showed the lowest initial scores and the lowest level of improvement in the model examinations. This point may also suggest that the residents had more interest in learning and performing other components of the FORESIGHT examination then those in this category, but this hypothesis will need to be tested in further studies.

These results suggest that pulmonary ultrasound topics (see Supplemental Digital Content 4, https://links.lww.com/ALN/B177) were the easiest for residents to learn anatomy and insonate an adequate image. This was followed by mechanisms of volume status and cardiac POC topics, both of which showed statistically significant improvement in image quality results from year 1 to 2. Of note all POC ultrasound topics showed improved results from year 1 to 2 with additional areas showing an average image quality score over 3 (defined as clinically usable). Overall, our results show that the curriculum did train anesthesiology residents to be able to, more often than not, correctly identify anatomy and obtain a clinical interpretable image within a brief period of time.

Our results suggest that the FORESIGHT curriculum can provide clinical utility, with our data showing that the primary anesthesia team received benefit from the POC ultrasound examination 76% of the time. Of note we demonstrated a new diagnosis rate of 31%, which is higher than what was required for our sample size calculation. As reported by others,⁴⁸  use of POC ultrasound in the perioperative setting often detects new cardiopulmonary pathology, and our data show similar results. Our results suggest that cardiac assessment contributes to the most frequent use of POC ultrasound and elicit the greatest frequency of new clinical diagnoses. This was followed by pulmonary assessment. As cardiopulmonary evaluation and management are crucial components to anesthesiologists, this study indicates that cardiopulmonary POC ultrasound can be effectively taught to anesthesiology residents and that this training may lead toward benefiting patient care. Of note, detection of diastolic dysfunction was the most common new diagnosis made within the cardiac POC ultrasound topics. This supports the literature indicating that diastolic dysfunction is grossly under diagnosed.⁵⁴  Regarding pulmonary POC ultrasound topics, detection of air space disease was the majority of new diagnoses. This may be secondary to the exquisite ability of POC ultrasound to detect air space pathology. Further studies would have to be done to assess relationship to postoperative pulmonary recovery.

Regarding the POC ultrasound topics of abdominal, ETT location, and ICP assessment, our results show that despite not being used as frequently, the proportion of clinical examinations that lead to a new diagnosis was quite high. In fact, the abdominal POC ultrasound category showed the highest detection of new pathology with a significant percentage being detection of gastric distention. Similarly, use of ultrasound for ETT location detected a substernal ETT cuff placement at a rate of 36%. Our results support the clinical utility of these less mainstream areas in POC ultrasound for the perioperative setting. In fact, there was no topic in the entire curriculum that did not provide some degree of positive clinical impact. This indicates that all components of the FORESIGHT examination if, appropriately trained, can be of benefit to perioperative physicians.

Our study was designed to introduce a comprehensive perioperative ultrasound examination. Potential difficulties and limitations include inability to guarantee the exact same exposure to curriculum for each resident given the off- campus and elective rotations. Resident surveys used a standard Likert item scale, but specific psychometric analysis for use of this scale was not performed. Also, we tested the subjects on a standardized patient of ideal body habitus. This study did not assess the ability of trainees to obtain images over a large range of patients of differing body habitus; although this was not one of the aims of this study, it does serve as a limitation. There was no charge to the patient for the POC examinations and evaluation for any potential economic impact was beyond the scope of the study. Also, the four experienced attending faculty instructors were required to have completed 50 FORESIGHT examinations and did not have further formal certification or credentialing. However, it is important to note that there is no current standard perioperative POC ultrasound training certification. Finally, although this study did show Kirkpatrick level 4 evidence for supporting the efficacy of this training curriculum, it is important to note that our findings are purely observational. New diagnoses were reported when the POC ultrasound examination results indicated findings that were new to the primary anesthesia team, but these findings were not verified to the patient’s medical records. Also reports were only generated for examinations that had agreement between resident and the experienced sonographers. This study did not capture rate of discrepancy between resident and expert interpretations. Additional, assessment of the POC ultrasound examinations resulting in additional testing for confirmation and/or alteration to postoperative management was not performed. Further studies will have to be performed to evaluate the clinical utility of any of the POC ultrasound topics toward impacting patient outcome.

This study highlights that a novel “whole-body” POC ultrasound examination (FORESIGHT) can be taught to anesthesiology residents using a model/simulation-guided curriculum, and with this appropriate training, one can impact clinical management of patients in the perioperative setting.

This work was supported by a Research in Education Grant from the Foundation for Anesthesia Education and Research, Schaumburg, Illinois (grant 40875; to Drs. Ramsingh and Cannesson); and Foundation for Anesthesia Education and Research Medical Student Research Fellowship (to Mr. Alexander).

The authors declare no competing interests.