Performance of a Large Language Model on the Anesthesiology Continuing Education Exam

As of August 2023, more than 180 million people have used the large language model Chat Generative Pre-Trained Transformer (ChatGPT).¹  There is emerging evidence that large language models can solve novel complex problems in a humanlike way.²  If proven accurate, large language models such as ChatGPT could facilitate real-time perioperative decision-making by giving anesthesiologists rapid access to evidence-based responses to clinical queries. ChatGPT performed “at or near the passing threshold” for the United States Medical Licensing Examination (USMLE) and displayed evidence of deductive reasoning.³  ChatGPT also performed well on board exam questions in various medical specialties, including neurosurgery and neonatology.^4,5  While questions from the American Board of Anesthesiology (Raleigh, North Carolina) questions are not publicly available, previous researchers have used questions from a board examination preparation book and found that ChatGPT also performs satisfactorily on these anesthesiology questions.⁶  One notable concern of large...