Assessing Radiology Residents
RadExam gives program directors an ongoing, specialized assessment tool.
Until now, there has been no standard tool available to assess a radiology resident’s knowledge base at the end of their rotations.” Petra J. Lewis, MBBS, FACR, professor of radiology and OB-GYN and vice chair of education at the Geisel School of Medicine at Dartmouth, is setting the scene for RadExam, a joint project between the Association of Program Directors in Radiology (APDR) and the ACR.
Lewis developed the content for this assessment tool along with Lawrence P. Davis, MD, FACR, vice chair and program director at the Zucker School of Medicine at Hofstra Northwell, Eric M. Nyberg, MD, assistant professor of radiology at the University of Colorado School of Medicine, and a large team of volunteer question writers and editors. The intent of RadExam — first piloted last September — is to provide radiology programs across the country with an objective method for the formative, continuous assessment of residents’ knowledge throughout their training.
“We created RadExam to give us a tool that would provide rapid resident evaluation (and feedback to them) at the end of a specific rotation,” Davis says. Before RadExam, the only national knowledge-based assessment came in the form of the ABR in-service exams (DXIT™) which are administered just once a year. “It’s important to give residents timely feedback on their accomplishments, or when there are areas that may need improvement,” he says.
RadExam uses ACR’s “radiology curriculum management system (RCMS),” now known as “Cortex,” to house assessment content and is designed to be user-friendly from both a resident and faculty perspective. “A program director can use one of the more than 80, 30-35-question, residency- level-specific exams developed by the RadExam exam developers, or in future will be able to build his or her own specialized exam using the thousands of questions from the RadExam database,” says Lewis. “So if you’re a breast imager, for instance, you could build an exam that is specifically related to breast MRI. RadExam gives us a metric so that we can intervene early in specific areas of weakness.”
DXIT exams test across all modalities and all systems. So within any one area, Lewis notes, at most you’re going to get 10 or 15 questions — with those same questions being applied to all four years of residency. “This means that somebody who has done one month of ultrasound and somebody who has done four months of ultrasound are taking the same exact exam, and not all questions may be appropriate for all resident levels,” she says. “This limits the power of the test to compare individual residents to others even at the same residency level.”
According to Lewis, program directors will likely administer RadExam assessments after each rotation, in addition to the annual DXIT exam. “The resident will take the exam that’s appropriate to both the specific rotation and the resident’s level of experience. They could be taking between 8-15 exams a year.”
RadExam content has been collected through crowdsourcing using a huge team of volunteers. Test item writers and editors were recruited from the Association of University Radiologists and the APDR to develop largely image-based, multiple-choice questions. Each question goes through three levels of editing by team members with both specialty and general assessment skill sets. Many of the contributors are also ABR-trained in exam development. The RadExam database system provides continuous, per-question feedback to the editors and writers to hone their skills for future test items. Approximately 4,500 questions have been developed or imported, and edited for RadExam.
“Until now, there has been no standard tool available to assess a radiology resident’s knowledge base at the end of their rotations.” –Petra J. Lewis, MBBS, FACR
RadExam is not intended to be a self-teaching tool, Lewis says. “It’s for assessment, but by having residents continuously answer high-quality questions based on the ABR core curriculum, including physics and non-interpretative skills, during their residency is certainly going to help them in their future test-taking skills. Program directors can identify specific knowledge gaps using RadExam to help direct resident learning.”
The pilot RadExam system went live in February and is free for any program director to access until July 2018. After that, there will be no charge to any program for each resident with a current DXIT subscription. Additional features, including the ability to make tailored exams, are planned for early 2019.
It will be important that radiology programs keep their annual DXIT subscriptions updated for every resident they may want to assess with RadExam,” Davis says. “RadExam is a great partnership between ACR and APDR that is providing an essential tool for both the radiology educator community and the residents.”
By Chad Hudnall, senior writer, ACR Press