News that the United States Medical Licensing Examination (USMLE) program will change its Step 1 scoring from a 3-digit number to pass/fail starting Jan. 1, 2022, has set off a flurry of shocked responses from students and physicians.
J. Bryan Carmody, MD, MPH, an assistant professor at Eastern Virginia Medical School in Norfolk, said in an interview that he was “stunned” when he heard the news on Wednesday and said the switch presents “the single biggest opportunity for medical school education reform since the Flexner Report,” which in 1910 established standards for modern medical education.
Numbers will continue for some tests
The USMLE cosponsors – the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME) – said that the Step 2 Clinical Knowledge (CK) exam and Step 3 will continue to be scored numerically. Step 2 Clinical Skills (CS) will continue its pass/fail system.
The change was made after Step 1 had been roundly criticized as playing too big a role in the process of becoming a physician and for causing students to study for the test instead of engaging fully in their medical education.
Ramie Fathy, a third-year medical student at the University of Pennsylvania, Philadelphia, currently studying for Step 1, said in an interview that it would have been nice personally to have the pass/fail choice, but he predicts both good and unintended consequences in the change.
The positive news, Mr. Fathy said, is that less emphasis will be put on the Step 1 test, which includes memorizing basic science details that may or not be relevant depending on later specialty choice.
“It’s not necessarily measuring what the test makers intended, which was whether or not a student can understand and apply basic science concepts to the practice of medicine,” he said.
“The current system encourages students to get as high a score as possible, which – after a certain point – translates to memorizing many little details that become increasingly less practically relevant,” Mr. Fathy said.
Pressure may move elsewhere?
However, Mr. Fathy worries that, without a scoring system to help decide who stands out in Step 1, residency program directors will depend more on the reputation of candidates’ medical school and the clout of the person writing a letter of recommendation – factors that are often influenced by family resources and social standing. That could wedge a further economic divide into the path to becoming a physician.
Mr. Fathy said he and fellow students are watching for information on what the passing bar will be and what happens with Step 2 Clinical Knowledge exam. USMLE has promised more information as soon as it is available.
“The question is whether that test will replace Step 1 as the standardized metric of student competency,” Mr. Fathy said, which would put more pressure on students further down the medical path.
Will Step 2 anxiety increase?
Dr. Carmody agreed that there is the danger that students now will spend their time studying for Step 2 CK at the expense of other parts of their education.
Meaningful reform will depend on the pass/fail move being coupled with other reforms, most importantly application caps, said Dr. Carmody, who teaches preclinical medical students and works with the residency program.
He has been blogging about Step 1 pass/fail for the past year.
Currently students can apply for as many residencies as they can pay for and Carmody said the number of applications per student has been rising over the past decade.
“That puts program directors under an impossible burden,” he said. “With our Step 1-based system, there’s significant inequality in the number of interviews people get. Programs end up overinviting the same group of people who look good on paper.”
People outside that group respond by sending more applications than they need to just to get a few interviews, Dr. Carmody added.
With caps, students would have an incentive to apply to only those programs in which they had a sincere interest, he said. Program directors also would then be better able to evaluate each application.
Switching Step 1 to pass/fail may have some effect on medical school burnout, Dr. Carmody said.
“It’s one thing to work hard when you’re on call and your patients depend on it,” he said. “But I would have a hard time staying up late every night studying something that I know in my heart is not going to help my patients, but I have to do it because I have to do better than the person who’s studying in the apartment next to me.”
Test has strayed from original purpose
Joseph Safdieh, MD, an assistant dean for clinical curriculum and director of the medical student neurology clerkship for the Weill Cornell Medicine, New York, sees the move as positive overall.
“We should not be using any single metric to define or describe our students’ overall profile,” he said in an interview.
“This has been a very significant anxiety point for our medical students for quite a number of years,” Dr. Safdieh said. “They were frustrated that their entire 4 years of medical school seemingly came down to one number.”
The test was created originally as one of three parts of licensure, he pointed out.
“Over the past 10 or 15 years, the exam has morphed to become a litmus test for very specific residency programs,” he said.
However, Dr. Safdieh has concerns that Step 2 will cultivate the same anxiety and may get too big a spotlight without the Step 1 metric, “although one could argue that test does more accurately reflect clinical material,” he said.
He also worries that students who have selected a specialty by the time they take Step 2 may find late in the game that they are less competitive in their field than they thought they were and may have to make a last-minute switch.
Dr. Safdieh said he thinks Step 2 will be next to go the pass/fail route. In reading between the lines of the announcement, he believes the test cosponsors didn’t make both pass/fail at once because it would have been “a nuclear bomb to the system.”
He credited the cosponsors with making what he called a “bold and momentous decision to initiate radical change in the overall transition between undergraduate and graduate medical education.”
Dr. Safdieh added that few in medicine were expecting Wednesday’s announcement.
“I think many of us were expecting them to go to quartile grading, not to go this far,” he said.
Dr. Safdieh suggested that, among those who may see downstream effects from the pass/fail move are offshore schools, such as those in the Caribbean. “Those schools rely on Step 1 to demonstrate that their students are meeting the rigor,” he said. But he hopes that this will lead to more holistic review.
“We’re hoping that this will force change in the system so that residency directors will look at more than just test-taking ability. They’ll look at publications and scholarship, community service and advocacy and performance in medical school,” Dr. Safdieh said.
Alison J. Whelan, MD, chief medical education officer of the Association of American Medical Colleges said in a statement, “The transition from medical school to residency training is a matter of great concern throughout academic medicine.
“The decision by the NBME and FSMB to change USMLE Step 1 score reporting to pass/fail was very carefully considered to balance student learning and student well-being,” she said. “The medical education community must now work together to identify and implement additional changes to improve the overall UME-GME [undergraduate and graduate medical education] transition system for all stakeholders and the AAMC is committed to helping lead this work.”
Dr. Fathy, Dr. Carmody, and Dr. Safdieh have disclosed no relevant financial relationships.
This article first appeared on Medscape.com.