A Human Factors Engineering Study of the Medication Delivery Process during an Anesthetic: Self-filled Syringes versus Prefilled Syringes

INNOVATIVE technologies and medical devices may improve the effectiveness, efficiency, and quality of work in the anesthesia work systems, but it may also negatively impact the anesthesia providers’ cognition, behavior, decision-making, and therefore their work performance.^1,2  Human factors engineering (HFE) is a systems engineering approach to improve the healthcare quality and patient outcomes by investigating the fit of the implemented technology or device with human capacity, abilities, and limitations.³  Through an integrated and systematic lens, the system components and their interactions within healthcare settings, such as the workers, patients, technologies, physical environments, tasks, and organization, can be examined and optimized.⁴ 

It is estimated that one significant medication error occurs in every 133 medication administrations in the operating room (OR),⁵  including incorrect doses (36.5%), substitutions (25.0%), and omissions (19.2%).⁶  Recently, investigators have estimated that this rate may be as high as 1:20.⁷  Also, waste and unnecessary costs and workflow disruptiveness are contributing to the concerns of perioperative medication management process.^8,9  To address these concerns, prefilled Syringes (PFS) have been implemented in the OR as an alternative to the traditional self-filled syringes (SFS) drawing medication from a vial. PFS are believed to be superior to SFS in many ways. Medications in PFS are prepared by pharmaceutical compounders beforehand under standardized quality control.¹⁰  These syringes come with enhanced labeling, ready-to-use dosage, and extended beyond use dating in the OR.¹¹  Theoretically, the opportunities for selecting and administering a wrong or expired medication are reduced.¹²  Therefore, the Anesthesia Patient Safety Foundation has endorsed PFS as a tenant of safely delivering medication in the OR.¹³ 

Despite the potential benefits, the PFS superiority in terms of medication safety and efficiency has not been explored in a real-world study. We hypothesize that a system that includes all commercially available PFS is superior to a system that uses only SFS. We sought to compare the system vulnerabilities (SVs) in an SFS system versus a PFS system, which can be indicative of the impact PFS can have on medication safety and efficiency, in the context of anesthesiology medication delivery in ORs. We define SVs as an activity or event that has the potential to reduce safety, efficiency of provider workflow, or increase drug costs and waste.

This study was primarily a qualitative research design, with a small quantitative portion. This research was conducted in two phases in March 2013. First, a work system analysis (WSA) was performed in the ORs and OR pharmacy. Second, a proactive risk assessment (PRA) was conducted via a focus group. Both phases were approved by the hospital’s and university’s institutional review board, Charleston and Clemson, South Carolina.

Two human factors engineers (Y.Y. and G.P.) conducted WSA observations at a 700-bed academic medical center in Southeastern United States. A WSA is a qualitative research method using HFE principles to analyze a complex sociotechnical work system based on naturalistic observation in the real-world context.¹⁴  The two observers followed the WSA procedure described by Karsh and Alper,¹⁴  using the shadowing method to understand the anesthesia medication flows in the OR settings. Observations began with the anesthesia providers receiving the medication from the pharmacy and concluded upon the medication’s return to the pharmacy. During the observations, the focus was on how anesthesia providers interacted with the medication and how those interactions were impacted by surrounding system components such as the OR layout, workflow disruptions, wasting of medication, etc.

In total, 17 cases were observed over 7 days totaling 48.5 h. The participants of the WSA were anesthesia providers—the main end users of SFS/PFS who have interactions with the medication, including anesthesiologists, anesthesiology fellows, anesthesiology residents, certified registered nurse anesthetists, and OR pharmacists. Anesthesia providers were first shadowed during general surgery cases in which all medications were provided in the form of vials. In those cases, anesthesia providers had to draw up medications from the vial into the SFS before administration. Observations continued for eight cases until data saturation was reached at which point no significant variability was noted. Then, anesthesia providers were observed during general surgery cases during which all commercially available PFS were incorporated into the system, which meant almost all medications in the OR were PFS. All the vials or PFS were prepared and stored in special cassettes and kits specifically for this study by the OR pharmacists before the cases. Observations continued for nine cases until data saturation was reached. A list of the SFS medications and PFS medications is shown in table 1.

An open coding process was conducted.¹⁵  This process was completed in multiple rounds. First, two researchers (Y.Y. and G.P.), trained in qualitative analysis, combined their observation notes and coded the data independently, generating a list of descriptive codes that were related to the research aim. Second, the two researchers met together to put descriptive codes together based on their similarity and created themes of SVs. Third, two researchers selectively attached observational events to themes and modified the coding structure accordingly. Reliability checks were conducted throughout the process reaching acceptable intercoder reliability of higher than 0.85. Moreover, a third senior qualitative researcher (A.J.R.), who was not involved in data collection, made decisions for any discrepancy between the two coders. Finally, the coding process produced a list of SVs of both the SFS process and the PFS process. To enhance the validity of the identified SV, the SV list was reviewed by a group of subject-matter experts.^16,17  On the basis of their feedback, iterative refinements were made before developing the final list of SVs.

Based on the results of WSA, a PRA was conducted via a focus group. The PRA focus group had two purposes: first, it served as a member checking process to validate whether the observational data analysis was congruent with participants’ real experiences¹⁸ ; second, it followed an HFE methodology to rate the risk of SVs.¹⁹  We followed the PRA procedure described by Faye et al.²⁰  Following the guidance of Morgan,²¹  we recruited six participants from those who participated in phase 1 representing the different roles of anesthesia providers, including attending anesthesiologists, residents, certified registered nurse anesthetists, and OR pharmacists. The PRA focus group lasted 90 min in a conference room at the hospital.

After presenting each SV to the participants, they were asked to rate the failure mode of this SV by assigning a likelihood score (1 to 4) of its occurrence, severity, and disruptiveness. The rating instruction document is in the appendix. Free discussion facilitated by the research team followed the rating process.

The risk of each SV was determined by multiplying the scores (e.g., occurrence × severity × disruptiveness).²⁰  We determined an overall score higher than 16 as a high-risk SV to workflow or patients. We used 16 as the cutoff criteria because that score represents two out of the three categories (occurrence, severity, and disruptiveness) being rated a four (e.g., 4 × 4 × 1). We conducted a descriptive statistical analysis, calculated each SV’s mean and SD.

The focus group discussion was transcribed verbatim, and a thematic analysis was conducted using NVivo 10^© (QSR International Pty Ltd., Australia). We highlighted the data chunks and labeled them using one descriptive phrase. This process was repeated until meanings and insights of these descriptive phrases were described using several higher-level themes.²²  After the coding schemes were developed, one researcher (Y.Y.) selectively attached the data chunk to the themes. To enhance the validity of the thematic analysis, another researcher (A.J.R.) reviewed the coding structure and discussed any disagreement with the research team.

Based on the combined observation data of the two observers, we created a work system map and a flow chart (fig. 1). The figures depict the entire anesthesia medication flow process, starting from where medications are picked up by an anesthesia provider at the OR pharmacy and progressing in a stepwise manner from where they are prepared presurgery in an OR to where they are administered to a patient in preoperative holding room, to their preparation and administration to the patient during the surgery, to their transport to the postanesthesia care unit, and finally back to the OR pharmacy where they are returned. The work system map reflects the layout of the OR floor and the locations of the 11-step physical sequence of the perioperative medication management process. The flow chart explains the 11-step physical sequences of the medication flow and anesthesia providers’ tasks in each step.

Although the work system map and flow chart show similarities for both SFS and PFS systems, it is important to highlight some differences. For example, step 2, step 5, and step 7 are less complicated in the PFS system than in the SFS system. In steps 2 and 5, for the SFS system, anesthesia providers need to draw up medication from vials using a syringe, dilute and, when needed, reconstitute the medication, and finish the medication preparation process by labeling the SFS. In contrast, for the PFS system, these steps are unnecessary. Moreover, during an operation, the anesthesia provider often prepares medications for the current case and the next case. During step 7 in the SFS process, as shown in figure 2, if a medication has not been prepared beforehand, dilution and reconstitution is a complicated, operational process. There are also different labeling requirements of the syringes depending on whether the provider intends to give the entire dose immediately or in divided doses. In contrast, in the PFS process, as shown in figure 3, even if the PFS has not been prepared beforehand, there are fewer, less complicated steps to administer a medication. Preparing medications for the next case is also much more straightforward.

Four overarching themes of SVs were identified in the SFS system: the potential causes of medication errors, quality and efficiency, waste, and cognitive complexity during medication preparation. In comparison, only three themes were identified in the PFS system: the potential causes of medication errors, quality and efficiency, and waste. There were a total of 21 SVs identified under the four themes in the SFS system and 8 SVs under the three themes in PFS system (tables 2 and 3).

During the PRA focus group, each of the SVs was presented and rated by the participants. The combined scores of each SV are shown in tables 2 and 3. The SV with the highest score (41.0) in the SFS system is 21: OR pharmacist may need to manage medication products unfamiliar to them based on a product switch (concentration or manufacturer) or drug shortage. The SV with the highest score (20.5) in the PFS system is 7: OR pharmacist needs to check the expiration date of PFS more frequently because PFS have a shorter shelf-life compared with vials. There are 11 SVs in the SFS system that received a score higher than 16. Six of these highly rated SVs were categorized in the theme of cognitive complexity during medication preparation. There is only one SV (7) with a score above 16 in the PFS system.

We identified two overarching themes in the focus group discussion: (1) SFS advantages and disadvantages and (2) PFS advantages and disadvantages. SFS advantages included flexibility and autonomy to prepare medication and reduced waste in some circumstances. SFS disadvantages included illegible handwriting on labels and similar vial packaging. For example, one participant described like this, “I have terrible handwriting, like I can read it. But can the next person who comes in and relieves me read it? Maybe not.” Another participant described it like this, “They [vials] had the same color top, a blue top. So I went to pull up Zofran, and, I mean, I checked, but it was Vasopressin.”

PFS advantages included standard dosage and time saving. PFS disadvantages included similar colored packaging for different medications, complex wrapper removal process, and increased possibility for arbitrary storage. For example, one participant described like this, “the red package of both succinylcholine and vecuronium [PFS] look alike.” Another participant described it like this, “It [the wrapper of PFS] is actually very hard to remove. If you had to do it in a hurry, and the little tiny tab is not pulled….” Another participant described it like this, “some people will grab them, and then tuck them in the drawer and not turn them in [to pharmacy].”

When comparing the two work processes, results showed that PFS simplified the work processes and reduced the number and associated risk of SVs. The results support our hypothesis that a PFS system is superior to an SFS system with respect to medication safety and efficiency. Figure 1, A and B, established a clear understanding of the process of anesthesiology medication preparation and administration and the ORs context.²³  As shown in these figures, with PFS implemented into the system, anesthesia providers did not need to draw up medications into a syringe, label the syringe, or perform complex calculations for dilution and reconstitution. Rather, they simply organized the syringes on their work surface. The reduced attentional demands required of the task may improve the anesthesia provider’s work performance and quality of care, especially during the time-pressured situations or when engaged in multiple tasks simultaneously during surgery as shown in figures 2 and 3.²⁴  In addition, with less mental resources required for medication preparation and administration in the PFS system, anesthesia providers may be able to make better decisions and complete tasks with fewer opportunities of errors, stress, and fatigue during their work.²⁵ 

The reduced SVs and their failure mode scores (as shown in tables 2 and 3) also supported the notion of PFS being superior with regard to medication safety and efficiency. One SV theme in the SFS process, “cognitive complexity during medication preparation,” was eliminated in the PFS system. This theme contained six separate SVs in the SFS process, and each of these scored higher than 16, indicating a high likelihood of occurrence and high risk to the patients and/or workflow. This result is consistent with previous research, which identified anesthesia providers commonly experienced high mental workload in the SFS system due to the cognitive complexity.²⁵  High mental workload may be correlated to performance deterioration and potential errors.^26,27  The PFS system, with reduced cognitive complexity, may have enhanced system resiliency thereby creating a work environment in which fewer human errors may occur.²⁸ 

However, the full potential of PFS has not been realized. Based on the SV list, among the eight SVs in the PFS system, four of them were similar to SVs found in the SFS system (table 3; nos. 3, 4, 5, and 8), and the other four were SVs that were newly introduced into the system (table 3; nos. 1, 2, 6, and 7). This implies that although a large number of SVs have been eliminated in the PFS system, several remain as ineradicable SVs in the anesthesia medication management process.²⁹  Furthermore, PFS have introduced new complications into the system. Additional challenges for using PFS were identified during the focus group discussion, such as the packaging issues.

PFS product improvement, including multiple modality labels, distinguishing fonts, colors, symbols, and raised dots and dashes, may be useful to address these SVs.²⁹  However, based on the Systems Engineering Initiative for Patient Safety model, a sociotechnical system is composed of components such as persons, tasks, tools, and technologies, the physical work environment, and organizational structures.⁴  Therefore, we must take a macroergonomics approach to address these issues to ensure the compatibility among PFS with other components in the work system.^30,31  Therefore, the interventions may need to go beyond the product level.³²  Organizational-level interventions may be necessary to reconcile many of the SVs in the PFS system. Possible organizational interventions include standardizing processes, hospital-wide training, and communication between hospitals and the PFS vendors. With both anesthesia medication product improvement and the organizational-level interventions, the remaining SVs and the newly introduced complications may be resolved.

The limitations of this study include the following: (1) this is not a multisite study and this hospital may not be representative, so the results from this study may not be directly generalizable to other hospital settings. However, as this was primarily a qualitative research study, the transferability can be determined by the readers who can tailor the results that are applicable to their own work environment. (2) There were only six participants in the focus group PRA; although this number meets the requirement for a focus group, the risk rating did not have enough power to perform inferential statistics based on the scores. However, the value of this study is that we presented human factors research using a WSA and PRA focus group to examine the SVs in the systems of anesthesiology medication delivery with SFS versus PFS. These methods are helpful in identifying SV, which may help propose effective interventions at both the product- and organizational-level for successful implementation of new technologies or medical devices. Proactively applying these methods to other work systems in health care may save money and energy by avoiding the full implementation of new technologies/devices that are not compatible with other elements within the system.

In this research, using human factors methods, that is, work systems analysis and PRA focus group, we compared PFS systems to SFS systems. We conclude that with PFS, work processes have been simplified; and the number and associated risk of SVs were also reduced. Therefore, many aspects related to medication safety and work efficiency in the PFS system are superior to those in the SFS system. However, PFS do also introduce new complications, which need to be addressed by improvements to the PFS product design, as well as organizational-level interventions. Orser et al.³³  articulately urge that we build better systems. PFS are a component of that safer medication delivery system.

The authors thank Gary Palmer, Ph.D., Octo Consulting Group, Washington, D.C., for his contribution for data collection and analysis.

Pharmedium (Lake Forest, Illinois) provided support in the form of an unrestricted grant to the Medical University of South Carolina (Charleston, South Carolina) to cover the costs of performing the study and analyzing the results. This funding came at the request of our group of researchers.

The authors declare no competing interests.

Use the definitions in table below to rate the system vulnerabilities (SVs) for both the self-filled syringe process and the prefilled syringe process. For each SV, you must assign a rating of (1) occurrence (how frequently does that SV occur), (2) severity on patient (how serious is the potential effect of the SV on the patient), and (3) disruptiveness to workflow (how disruptive is the SV to the provider’s workflow). Assign a 1 to 4 rating in the spaces allotted in the rating document.