You never really understand a tool until you know how it is misused and how it can hurt you.
Franklin L. Miller, Lt. Col., USAF
MY grandfather, a lieutenant colonel in the United States Air Force (USAF), was responsible for the training of all civilian mechanics in the USAF from 1948 to 1952. In this position, he received reports of injuries to his mechanics from around the world and knew well how tools were misused. I was reminded of his aphorism as I read the article by Schebesta et al. in this month's ANESTHESIOLOGY.1
“After acknowledging [the utility of airway simulators], [my grandfather] would tell me to be sure I understand how they are misused and how they might hurt or mislead us.”
The authors used computed tomography scans to compare the upper airway anatomy of four high-fidelity patient simulators and two airway trainers with normal humans. They found that none of the mannequins had anatomic dimensions that consistently replicated the sample of human subjects assessed. The authors state that because the calculated pharyngeal space is particularly important for the fit of supraglottic devices, the comparison of this volume in humans with the volume in mannequins was their primary outcome. They found that the calculated volume of the pharyngeal space in the mannequins varied from 2.2 to 5.1 times that of the human subjects.
The boundaries of the airway space in mannequins may be similar to those that we create when we manage an airway with a laryngoscope or a supraglottic airway, which may explain why people using these mannequins tend to report a satisfactory simulation experience.2This study, however, did not consider the compressibility of “tissues” in mannequins or the frictional interaction between airway appliances and mannequins. These properties, although harder to assess, contribute to the sense of realism when using mannequins and simulators.
What are we to make of these findings? Although mannequins and simulators are commercially successful and have become firmly integrated in medical education, most of us would agree that they are not 100% realistic. To assess the utility and limitations of these devices we should review some terms associated with fidelity. We usually think of fidelity as “adherence to fact or detail” or “accuracy, exactness,” but in the simulation literature, fidelity can be defined as the extent to which the appearance and behavior of a simulator match the appearance and behavior of the system simulated. We must distinguish engineering or physical fidelity (the degree to which the training environment replicates the physical characteristics of the real task) from functional fidelity (the degree to which the skills in the real task are captured in the simulation) and psychologic fidelity (how the simulation makes the user feel).3
The goal of a virtual training system is to enhance performance in a real-world task. Transfer of training is the process by which knowledge, skills, and abilities acquired through training are applied in an actual situation.4We can envision a spectrum of applications for simulators, from highly useful to less useful and possibly harmful. On the useful end of the spectrum, we have our medical students and junior residents practice basic airway skills on a mannequin before a first attempt on a patient. This application is an example of basic skill acquisition or transfer of training.
In addition to novice training, an anatomic model may be useful for intermediate or advanced training, such as learning the mechanics of manipulating a flexible bronchoscope or the intricacies bronchial anatomy. Simulators are helpful for practicing algorithms such as advanced cardiac life support or a response to a failed airway in the operating room. However, at least one study showed that anesthesia residents trained with a computer screen-based simulation performed as well as those trained with a high-fidelity simulator. The authors concluded that screen-based simulation may be acceptable for training in the technical aspects of crisis management whereas a full-scale simulator might be better for behavioral aspects like communication and leadership.5
On the other end of the training utility spectrum, there are circumstances in which simulators perform less well or may mislead. When tissue fidelity is important, plastic simulators leave much to be desired. I have instructed hundreds of anesthesiologists in emergency cricothyrotomy using fresh pig tracheas. Surgeons who have participated in this exercise tell me the tissue fidelity of the pig trachea is excellent and surpasses any commercial simulator or mannequin currently available. Commercial simulators also do not do a good job replicating tissue compressibility, secretions, blood, or an uncooperative patient, often elements of a challenging airway. Advanced simulators may fail when a good seal is needed with a supraglottic airway or may have anatomy unfavorable for fiberoptic tracheal intubation.
In addition to training, mannequins have been used for research and airway device evaluation. To study airway devices in mannequins rather than humans means institutional review board approval is not needed, data collection is quick, and confounding variables are minimized. There are some settings appropriate for mannequin studies, such as airway management of victims with limited airway access or those rescued from a simulated toxic environment.6,7There has been a proliferation of publications evaluating airway devices in airway simulators and mannequins. When Rai and Popat8found 57 publications in leading anesthesia journals describing mannequins used to evaluate or compare airway devices, they also found that few of these articles were followed by studies comparing the devices in humans. We must remember that performance of airway devices in mannequins is a means to an end, not an end itself.
In 2003 Cook proposed a three-step process for evaluation of an airway device. A new device is first evaluated on specifically designed mannequins. Then a safety and efficacy study is performed. After that the device is compared in a randomized controlled trial with currently used devices of the same class.9
Cook contacted the manufacturers of seven recently introduced devices and asked in how many humans had the device been tested before marketing began. The four companies that replied reported studies in 60–600 patients before marketing. I know from personal experience that a novel airway device developed using airway simulators required major revisions after being tested on cadavers. Given the discrepancy between human anatomy and the dimensions of the airway simulators, it is not surprising that these devices are a frustrating platform for prototype development. Perhaps one of the reasons the laryngeal mask airway was so successful was that it was based on anatomic specimens rather than tests with mannequins.10If the airway simulators and trainers tested in the study by Schebesta et al. had been used, it is unlikely that Brain would have found his device satisfactory.11
The authors of this month's article encourage the reader to question our assumptions about the role of airway simulators in training and research. In the end, we do the best with the tools we have. If my grandfather were alive, I believe he would recognize that airway trainers and simulators are powerful and effective tools for specific tasks. After acknowledging their utility, he would tell me to be sure I understand how they are misused and how they might hurt or mislead us.
P. Allan Klock, Jr., M.D., Department of Anesthesia and Critical Care, University of Chicago Medicine, Chicago, Illinois. aklock@dacc.uchicago.edu
