Best Practice Alerts Informed by Inpatient Opioid Intake to Reduce Opioid Prescribing after Surgery (PRIOR): A Cluster Randomized Multiple Crossover Trial

We conducted a cluster randomized multiple crossover trial¹⁵  to evaluate a real-time best practice alert embedded into electronic health records. The study design followed Pragmatic Explanatory Continuum Indicator Summary guidelines to maximize broad applicability and is reported according to Consolidated Standards of Reporting Trials Extension for Cluster Randomized Trials guidelines.^16,17  Institutional review board approval was obtained before patient enrollment (single institutional review board of record: COMIRB, protocol 19-3095). Patient and provider consent was waived because the alert supplemented the current standard of care, there was no requirement to respond, and the alerts were unlikely to be harmful and might have proven beneficial. The trial was registered on June 25, 2020, at ClinicalTrials.gov (NCT04446975).

The clusters were an academic medical center and three community hospitals, all part of the UCHealth system, which serves both inner-city and rural populations across Colorado, Nebraska, Wyoming, and beyond. Primary study subjects were credentialed prescribers at each site. Eligible secondary study subjects were postsurgical hospital inpatients who were hospitalized at least one night before discharge. There were no exclusion criteria for the prescribing providers, but patients needed to be at least 18 yr old.

Our previous work in three diverse samples of surgical procedures found that among available predictor variables that could be incorporated into an electronic decision-support tool, 24-h predischarge opioid intake was most associated with patient-reported postdischarge opioid intake.^11–13  Consistent with these findings, guidelines recommend a tiered approach to opioid prescription by categorizing in-hospital opioid intake on the day before discharge: (1) no opioids, (2) more than 0 to 22.5 milligram morphine equivalents (equivalent to three oxycodone 5-mg tablets), or (3) more than 22.5 milligram morphine equivalents.¹⁸ 

Centered on these findings and with local stakeholder input, we developed a best practice alert algorithm for commonly prescribed opioids based on previous day inpatient opioid intake (table 1). Two conditions were required for the alert to be displayed to a provider: the prescription had to be written during an active alert period and the initial prescription attempt had to exceed the alert threshold. During the 8-week periods when prescriber alerts were active, providers who attempted to prescribe higher-than-recommended doses were automatically notified on the order screen with a suggestion for a reduced prescription. Within the alert, a suggestion to prescribe nonopioid adjuncts was also displayed. Final prescription decisions remained at the discretion of the provider.

The best practice alert was assessed over a 48-week period from July 2020 to June 2021 in all eligible patient discharges within the four hospitals. The four hospitals were randomized to the starting configuration (active alert or inactive alert) to control for potential period effects and avoid within-hospital contamination. The clusters alternated between active alert and inactive alert conditions for four 8-week periods, each separated by a 4-week washout interval to minimize learning and augment masking (fig. 1). Randomization was implemented so investigators and data analysts remained masked to the condition designation. While alerts were displayed to providers during active periods if their prescriptions were above the algorithm alert threshold, the schedule was not shared with providers or patients.

The primary outcome was the amount of opioids prescribed at discharge. The opioids prescribed were recorded by type and total amount dispensed in oral morphine milligram equivalents.¹⁹  In 139 patients who had recorded medication names but no amounts recorded, we assumed that no medications were prescribed.

Secondary outcomes included opioid and nonopioid combination medications prescribed on the day of discharge and any additional opioid prescriptions written after discharge. Analgesic prescriptions were defined as a categorical outcome: opioids, combination opioid and nonopioid medications, or no opioid medications prescribed at discharge. To estimate the potential for underprescribing, outpatient opioids prescribed days 1 to 28 postdischarge by any health system provider were reported dichotomously for every admission based on the discharge date.

Other study measurements obtained from electronic health records included demographic and clinical characteristics, hospitalization-specific variables, and perioperative information, including surgical subspecialty, in-hospital opioid intake, and ordering provider profession and ordering provider sex (table 2). When a history of chronic pain was not recorded, we assumed that missing values represented not having chronic pain issues.

Baseline balance on potential confounding factors was assessed using absolute standardized differences, calculated as the difference in means or proportions divided by the pooled SD.²⁰  An absolute standardized difference of more than 0.1 was considered to indicate imbalance, and these variables were adjusted for all analyses.

Analysis was conducted at the individual patient level using a modified intention-to-treat strategy. Mixed (hierarchical) modeling procedures were utilized to account for the correlation within clusters and time and periods in the crossover cluster design.²¹  Because the effect was estimated within hospital due to the crossover design, it was adequate to specify fixed effects of the hospital, alert condition, time periods, and surgical subspecialty grouped according to previous work.²²  We also included a random effect for cluster periods to account for correlated observations within a cluster period by assigning each cluster period its own intercept.

The effect of the best practice alert on opioid prescription at discharge was evaluated using a linear regression model after log transforming the outcome. The treatment effect was reported as a ratio of geometric means (active alert or inactive alert) and the associated 95% CI. A geometric mean of less than 1.0 represents a decreased opioid prescription amount during the active alert condition relative to the inactive condition. Based on previous work,^12,13  we considered a one-third reduction in prescribed opioids to patients in the active alert period to be clinically meaningful.

The geometric mean and median are both appropriate summary measures for skewed data as they are robust to extreme values. Medians have the added advantage of a more intuitive interpretation, but in practice, quantile regression models (used to estimate conditional medians) with complex mixed-model specifications are difficult to implement. Thus, we reported the unadjusted median in each treatment group to aid in interpretation, but the treatment effect was reported as the ratio of geometric means. We note that geometric means and medians are equal when data are log-normal data but can diverge in other situations—as it did for our results.

The effect of the best practice alert on additional prescriptions after discharge was evaluated using a log-binomial regression model, including the previously mentioned fixed effects. The random cluster period effects were excluded as the mixed model failed to converge. The effect of the alert on the three-level analgesic categorical variable was assessed using a multinomial logistic regression model with a generalized logit link. Our prespecified mixed-effects model with heterogeneous variances between the treatment groups on the cluster period random effect and residual errors did not converge. We therefore report results from a simpler fixed-effects model.

We also conducted a post hoc secondary analysis to evaluate whether a differential treatment effect existed for the subgroup of patients qualifying for an alert (i.e., the initially considered opioid dose exceeded the one recommended by our algorithm) compared to the subgroup of patients that did not. The analysis was conducted by fitting the primary outcome model with an additional interaction between the treatment group and an indicator of whether the patient qualified for an alert. The interaction was considered significant if P < 0.15. Additionally, as the primary outcome data was not log-normal, a sensitivity analysis was conducted using a quantile regression model (instead of linear regression) adjusting for the fixed effects. All main effect analyses were conducted at α = 0.05, two-tailed, and both R 4.0.2 and SAS 9.4 were used to conduct the analyses.

We determined that it would be feasible to recruit 1,500 patients per cluster period, across four hospitals for four time periods. Although we considered a 33% reduction in mean prescribed opioids to be a clinically meaningful difference, we used a smaller effect size (11%) for the power analysis to account for providers not following the suggested dose in some patients. We estimated that the trial would have more than 99% power to detect a ratio of geometric means of 0.9 (active alert or inactive alert) for prescribed opioids or a difference of ˗0.12 (active alert or inactive alert) on the log-scale at the 0.05 significance level.

ClusterPower,²³  a flexible simulation-based package in R for estimating power and sample size in cluster randomized trials, was used. We randomly generated numerous data sets using a prespecified effect size under the alternative hypothesis and then determined the empirical power based on how often the null hypothesis was rejected. Further detail regarding sample size derivation is available in the Supplemental Digital Content (https://links.lww.com/ALN/D156).

We considered 21,864 patients from four hospitals for inclusion in the analysis during the 48-week period from July 2020 to June 2021. We excluded 175 patients because they were less than 18 yr old, yielding a final analysis sample size of 21,689 patients. Of these patients, 10,114 (47%) underwent general or orthopedic surgery. A total of 1,053 unique providers prescribed opioids to patients during the study period. Of these, 45% (472) were male, 41% (436) were attending physicians, 29% (309) were residents or fellows, and 29% (308) were in the nonphysician category (physician assistants, nurse practitioners, clinical nurse specialists, or nurse midwives). There were no clinically important differences in potential confounding factors (i.e., all absolute standardized differences were less than 0.1) in providers and patients between the active alert period (N = 11,003) and inactive alert period (N = 10,686). However, some confounders had a high percentage of missing values, including 46% of discharge pain scores. Sample characteristics by hospital are displayed in Supplemental Digital Content (https://links.lww.com/ALN/D157).

The median opioid dose prescribed at discharge was 75 oral morphine milligram equivalents [quartile 1, quartile 3: 0, 225] when the alert was active versus 100 oral morphine milligram equivalents [0, 225] when the alert was inactive. The ratio of geometric means (active alert/inactive alert) for opioids prescribed at discharge was estimated as 0.95 (95% CI, 0.80 to 1.13; P = 0.586) using a linear mixed model with fixed effects for hospital, treatment, time period, surgical specialty, and random effects for the cluster period (table 3). Thus, the best practice alert did not significantly affect the amount of opioids prescribed at discharge.

The percentage of patients without prescribed opioids at discharge was 35% in both the active and inactive alert groups. The number of patients with prescriptions exceeding 225 oral morphine milligram equivalents (the maximum dose recommended by our algorithm) was 2,208 (20%) in the active alert group and 2,201 (21%) in the inactive alert group (fig. 2).

Opioids in the first 28 days after discharge were prescribed in 2,046 (19%) patients in the active alert group and 1,870 (17%) patients in the inactive alert group. Alerts were not found to affect postdischarge opioid prescriptions, with an estimated odds ratio of 1.06 (95% CI, 1.00 to 1.12; P = 0.052). Furthermore, alert exposure was not found to affect the odds of being prescribed opioid and nonopioid combination preparations (odds ratio, 0.94; 95% CI, 0.86 to 1.03) or of being prescribed opioids (odds ratio, 1.00; 95% CI, 0.93 to 1.06) compared to receiving no opioid medications at discharge (table 4).

During active periods, best practice alerts were shown to prescribers in 28% (N = 3,074) of the cases. During the inactive alert period, 30% (N = 3,182) of the patients qualified for an alert, although none was presented, as per the study protocol (table 1). A total of 340 of the active alert patients qualified for an alert, yet an alert was not displayed. Whereas among inactive alert patients, 111 qualified for an alert, and the alert was displayed. In the post hoc secondary analysis, we assessed whether a differential treatment effect existed for the subgroup of patients qualifying for an alert compared to the patients who did not. The median opioid doses prescribed at discharge in the subgroup qualifying for an alert were 201 oral morphine milligram equivalents [quartile 1, quartile 3: 75, 338] during active alert periods and 225 oral morphine milligram equivalents [112, 338] during inactive alert periods. The treatment effect in the subgroup of patients eligible for alerts was an estimated ratio of geometric means of 0.89 (95% CI, 0.79 to 0.99; P = 0.027). For the 71% of patients who did not qualify for an alert, the estimated ratio of geometric means was 1.02 (95% CI, 0.86 to 1.22; P = 0.788). However, we did not find evidence for an interaction between whether patients qualified for an alert and the randomized treatment group (P = 0.822). Thus, displaying best practice alerts did not appear to change prescriber behavior (table 5).

In the sensitivity analysis to the primary outcome analysis, using quantile regression on all patients, the 50th percentile, 75th percentile, and 95th percentile difference in opioid prescription at discharge were all estimated to be 0 (P = 1.00), which is consistent with the results of our primary analysis.

In this randomized multiple crossover cluster trial of 21,689 adult surgical patients, those discharged during the active best practice alert period received a median of 75 oral morphine milligram equivalents, while those discharged during the inactive alert period received a median of 100 oral morphine milligram equivalents (P = 0.586). The embedded clinical decision-support tool did not significantly or meaningfully change the amount of opioids prescribed at discharge. Nor did opioid-prescription guidance alter the prescription of combination opioid and nonopioid preparations or the need for additional opioid prescriptions during the initial 28 days after discharge.

The national awareness of the opioid epidemic and its public health implications within the “opioid ecosystem,” especially in Colorado, are relevant to the context of this study.²⁴  The general and colorectal surgery departments were among the first to adopt enhanced recovery after surgery protocols initially developed by European academic surgeons. The protocols included limiting long-acting opioids with additional emphasis on opioid-sparing multimodal analgesic approaches.²⁵  General surgeons were also the first to marshal enhanced recovery protocols within the University of Colorado Health system,²⁶  and since 2018, new legislation, including Colorado Senate Bill 18-022, further limited prescription duration.²⁷  Although exceptions exist for acute postoperative surgical pain, the implementation of such policies may explain why 35% of patients were discharged without an opioid prescription and why only a quarter of the prescription attempts initially exceeded an opioid dose that our algorithm considered reasonable. In fact, even in the reference group, the median discharge prescription was only 100 morphine milligram equivalents, corresponding to about 13 oxycodone 5-mg tablets. Although opioid prescription rates have dropped over the last decade in the United States, surgical opioid prescribing at discharge still vastly differs from international practices. In a recent eight-country, 4,690-patient study of surgical patients after three common general surgical procedures, United States–based patients were 18 times more likely to be prescribed opioids at discharge than those in other countries.²⁸ 

Clinical decision-support systems for computerized provider order entry have demonstrated reductions in adverse events from drug–drug interactions,²⁹  improvements in clinician performance,³⁰  and lowering of pharmaceutical costs.³¹  Historically, successful best practice alerts have integrated extensive education in tandem with an actionable alert feature.³²  Although our best practice alert was designed and executed with the best intention of adhering to the five “rights” of clinical decision support (right information, to the right person, in the right format, through the right channel, at the right time), insufficient clinician engagement may have contributed to a lack of comprehension or awareness.³³  Moreover, we cannot presuppose that all incorporated electronic alerts will be advantageous or lead to positive change. There are several recent examples of innovative decision-support systems that did not garner the expected user attention or anticipated clinical outcomes, whether due to alert fatigue or presumed user irrelevance.^34–36  A corollary is that novel decision-support systems should be formally tested and rigorously validated, just like other medical devices, as there may be unintended consequences if implemented without discretion.³⁷ 

While emphasis has been placed on surgeon characteristics that predict differences in opioid prescribing after surgery,³⁸  we found that nonphysician clinicians comprised 29% of the prescribing providers but were responsible for 53% of the discharge prescriptions. This potentially represents a shift in responsibility within care teams. Any successful system for limiting opioid prescribing will thus need to include both nonphysician and physician prescriber buy-in. As physician anesthesiologists seek to add value to patient care, there is an opportunity in formulating perioperative multimodal plans and identifying high-risk individuals who would benefit from transitional pain management care.²⁴  National guidelines suggest nonopioid analgesics and nonpharmacologic modalities in addition to opioids as part of a balanced pain management plan in both surgical and nonsurgical settings.^39,40  In our study, only 16% of discharges included prescriptions for combination medications, possibly reflecting current guidance to use over-the-counter analgesics such as acetaminophen and nonsteroidal anti-inflammatory drugs separately.

A limitation of conventional cluster randomized crossover trials is the possibility of systematic effects of period on the outcome, such as learning or overall care improvements. That risk was diminished by our alternating cluster trial design, which repeated the study unit four times to minimize time-dependent confounding from background improvements in healthcare and regression to the mean.¹⁵  Carryover effects were minimized through the inclusion of 4-week washout intervals.⁴¹  Moreover, allowances were made to correlate outcomes within clusters and time periods. The Hawthorne effect was limited by continual observation throughout periods with and without intervention.¹⁵ 

Our study was limited to postsurgical inpatients. As such, our results should be cautiously extrapolated to ambulatory surgery or emergency departments. Prescribing clinicians were the primary subjects of the study, and given the pragmatic trial design, we did not collect patient-reported outcomes such as self-reported opioid intake after discharge. While we achieved adequate balance (absolute standardized difference of less than 0.1) on all potential confounders, some categories had a high percentage of missing values, although none were part of the primary analysis and were not considered to affect the power or efficiency of this study. Furthermore, analysis at the individual level was performed using a modified intention-to-treat strategy.

In summary, electronic opioid prescribing guidance embedded in an electronic ordering system did not significantly or meaningfully reduce discharge opioid prescribing for surgical inpatients. In the context of an analgesic education and awareness campaign, a clinical decision-support tool aimed at individualizing opioid prescribing at discharge did not lead to less opioid prescribing.

Supported in part by National Institutes of Health grant No. K23DA040923 (Bethesda, Maryland; to Dr. Bartels), Agency for Healthcare Research and Quality grant No. R01HS027795 (Rockville, Maryland; to Dr. Bartels), and the Data Science to Patient Value Dean’s Transformational Grant from the University of Colorado School of Medicine (Aurora, Colorado; to Dr. Ho and Dr. Kutner). The content of this report is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Agency for Healthcare Research and Quality. The National Institutes of Health and the Agency for Healthcare Research and Quality were not involved in study design, collection, analysis, interpretation of data, writing of the report, or the decision to submit the article for publication.

Dr. Fernandez-Bustamante received funding from the National Institutes of Health/National Heart, Lung, and Blood Institute (Bethesda, Maryland), the U.S. Department of Defense (Arlington, Virgina), the Merck Investigator-initiated Studies Program, and the Institute for Healthcare Quality, Safety and Efficiency for projects not relevant to the discussed work. The other authors declare no competing interests.

Full protocol available at: karbartels@unmc.edu. Raw data available at: karbartels@unmc.edu.

Supplemental Digital Content 1. Methods: Sample Size Justification, https://links.lww.com/ALN/D156

Supplemental Digital Content 2: Sample Characteristics by Hospital, https://links.lww.com/ALN/D157