Does the American Board of Anesthesiology BASIC Examination Really Affect Anesthesiology Resident Knowledge Acquisition?

We applaud Zhou et al. for their recent publication of American Board of Anesthesiology data suggesting that after implementation of the BASIC certification examination, anesthesiology residents’ performance improved on the subsequent in-training examination.¹  As opined by Murray in an accompanying editorial, increased transparency and sharing of data from the American Board of Anesthesiology is welcome and useful to the specialty, training programs, and community at large that physician anesthesiologists serve.²  Our program, as we suspect many others have, is focusing educational preparation for the BASIC exam over the two years of clinical base and clinical anesthesia year 1 training, an acknowledged potential benefit and goal.

Both the editorial and article discuss the small effect size (two points in scaled score) in this initial evaluation of the examination process restructure. In the mixed effects model, residents with in-training examination scores were considered, thus implying that a large proportion not taking the in-training examination during the clinical base year and any resident not sitting for subsequent in-training examinations was not accounted for. The method similarly confirms that only residents “who maintained a regular progression of training level” were included. Thus, it is likely that residents lost from the program through attrition (whether for medical knowledge, professionalism, or another competency) may have affected the small signal. This and an additional unintended consequence of the new examination structure is explored.

1. Most programs have incorporated success on the BASIC examination as an objective milestone measure of medical knowledge and many are offering residents only two unsuccessful opportunities, in the summer and fall of the rising clinical anesthesia year 2 year. As such, any deficiency will be apparent prior to the next spring in-training examination in the clinical anesthesia year 2 year and any loss of residents (who would naturally be presumed also to be poor performers on the in-training examination) may have de facto resulted in an apparent improvement in the cohort’s second compared in-training examination score.

2. Similarly, with appropriate increased academic attention and focus on the BASIC exam, it is likely that many clinical base and clinical anesthesia year 1 residents are more committed to the higher stakes first certification BASIC examination, which has implications for successful maturation through the program. The more specific curriculum for the BASIC exam and time required for preparation may unintentionally distract attention from the preceding in-training examination, which for many programs is not a high-stakes examination for satisfactory academic progress. Thus, the in-training examination in the clinical anesthesia year 1 year as the first comparison point may be artificially lower, this also appearing to accentuate the “improvement” in the subsequent in-training examination.

Addition of the BASIC exam as the first step in anesthesiology resident certification appears to be appropriate and useful to residents and programs in the milestone era. Optimism for objective markers of success should remain restrained, however, until the impact of unintended consequences in resident exam preparation priorities and residents missing from the in-training examination through attrition are accounted for. We eagerly anticipate continued distribution of data from the American Board of Anesthesiology on these and other certification processes.

The authors declare no competing interests.