Tania M. Jenkins, Ph.D.

On February 12, 2020, the United States Medical Licensing Exam (USMLE) parent organizations (the Federation of State Medical Boards, FSMB, and the National Board of Medical Examiners, NBME) approved a major change in the USMLE Step 1 exam—the basic science licensing test that is considered the single most important determinant for residency placement. The professional bodies have agreed to go from a three-digit score to making the exam pass/fail, effective January 1, 2022. The decision was reached after a long consultative process initiated by the FSMB, NBME, and other professional bodies such as the American Medical Association, the Association of American Medical Colleges, the Educational Commission for Foreign Medical Graduates to assess whether changes should be made to Step 1. To that end, they invited key stakeholders from graduate and undergraduate medical education institutions, state medical boards, medical students, and the public to a conference in March 2019 to primarily discuss the relative benefits of numeric scoring options versus pass/fail scoring. In the end, they decided to make Step 1 pass/fail, but to keep the three-digit score for the Clinical Knowledge portion of Step 2 (CK). 

​

What are we to make of this important change in the way physicians are licensed in the US?

​

Well, as with most things, this sociologist’s answer is: it’s complicated. There are undoubtedly some very real benefits to going pass/fail, but also multiple unintended consequences. And unsurprisingly, the most privileged individuals in the profession are likely to experience fewer of those unintended consequences than those who are already more stigmatized and subordinated.

 

Intended and Unintended Consequences

Let’s begin with the more privileged. On the surface, there appear to be clear benefits of a pass/fail grading scheme for US medical students’ mental health, as scholars have identified Step 1 as a specific determinant of trainee burnout (Jenkins et al. 2018). Because the exam is such an important determinant of matching, especially in competitive fields, the stress surrounding Step 1 has led to significant mental illness among some test-takers. This anxiety, in fact, was a primary reason behind the decision to make Step 1 pass/fail. Another reason had to do with the “parallel curriculum” that has emerged in medical schools surrounding the test. Students would rather invest considerable time and energy into acing this career-altering exam than focusing on other relevant subject matter, such as medical ethics or social determinants of health, which are often considered “low yield” topics because they are not on the test (Knopes 2019). Making Step 1 pass/fail has therefore undoubtedly led many current and future medical students to breathe a collective sigh of relief this week as their dreams of matching into their preferred specialty no longer ride on a single, nail-biting eight-hour exam.

​

But that momentary relief was no doubt swiftly followed by a more daunting uncertainty—if it is no longer Step 1 that determines how the most competitive residencies are meted out, then what will? One obvious answer to this question is Step 2 (Clinical Knowledge)—Step 1’s more clinically-oriented cousin which, at least currently, is not required from all residency programs prior to applying. That may change, however, as residency programs look to new markers of ability to create their match lists. As Step 2-CK becomes more relevant in the residency selection process, it is possible that it simply replaces Step 1 as the primary determinant of residency selection—and as the new stressor making medical students sick. And whatever currently constitutes high-yield topics will likely just shift from what’s on Step 1 to what’s on Step 2. For these reasons, making Step 1 pass/fail may not have the intended health and educational benefits that decision-makers hope.

 

Perhaps even more worrisome are the implications for less privileged applicants. My forthcoming book, Doctors’ Orders: The Making of Status Hierarchies in an Elite Profession (Columbia University Press, due June 2020), explores the question of why internal medicine residency programs are so segregated on the basis of medical pedigree, with large university programs disproportionately staffing American-trained MDs (USMDs) and smaller community programs disproportionately staffing international and osteopathic medical graduates (non-USMDs), who are routinely stigmatized in the profession. I spent three years observing two segregated programs to better understand the construction and consequences of such segregation for the residents’ training. While much of the book critiques the illusion of meritocracy that a three-digit Step score helps provide to applicants, a three-digit score still gave some non-USMDs a fighting chance of making it into competitive programs, provided they scored astronomically high on the test. The larger university program I studied, for example, routinely discarded almost every non-USMD application without review. Every year, however, there were a handful of cases that avoided the chopping block, in part thanks to superlative Step scores. As one program leader put it: “We don’t automatically reject people. You know if someone [a non-USMD] is at the top of their class and has done 270s on Step 1 and 2, and…we know the people they’ve done an elective with and they say this kid is better than any American grad I’ve ever seen, we’ll pay attention to that.” In other words, if a non-USMD scored two standard deviations above the national mean on their Steps and came with the ringing endorsement of a known faculty member, they possibly had a chance for an interview. Making Step 1 pass/fail could restrict that (small but existent) pathway to competitive residencies for some non-USMDs and would likely place more emphasis on informal resources, like network ties, which are more difficult for non-USMDs to come by.

 

Of course, Step-2 (CK) might very well replace Step 1 as the main determinant of residency positions, leaving open a small window of opportunity for non-USMDs to outshine their USMD counterparts. But instead of two data points (and two opportunities for non-USMDs to shine), there would only be one exam for program directors to consider, meaning it could all ride on Step 2-CK for non-USMDs hoping to break into competitive residencies. Instead of being able to see growth and development between Steps, program directors would only rely on a single standardized measure of competence. And if Step-2 CK does not become mandatory for all residency applicants, it is likely that old and new signals of “ability” and “fit” will simply grow in importance—making things like medical school of origin, research experience, and honor society membership all the more important during recruitment, clearly favoring USMDs over non-USMDs. There may also be consequences for minority USMDs for the same reasons; in fact, there is an important sociological literature on how recruitment decisions based on “fit” often reproduce broader social inequalities along the lines of class, race and gender (see for example Lorber 1984, Quadlin 2018, Rivera 2012, Rivera 2015).

​

Meaningful change in light of enduring social norms

Some have pointed out that three-digit USMLE scores only help international and osteopathic medical graduates in a world where program directors are flooded with applications. That if program directors only had to sort through fewer applications, they could more holistically evaluate every individual and make more informed decisions rather than rely on heuristics like the USMLEs. The solution they proffer is application caps, where applicants can only apply to a limited number of applications. These caps, however, would only help increase the holistic review of applications if they were accompanied by sweeping changes in professional norms and beliefs surrounding non-USMDs, which are routinely viewed as “incompetent” or “lazy”. Many programs also view it as a “badge of honor” to match exclusively with USMDs, as program leaders told me. In a recent study, my colleagues and I found that only 16 percent of internal medicine programs across the country can be described as “integrated,” meaning they staff between 30-65% USMDs, with over one-third of university programs staffing 90% or more USMDs (Jenkins et al. 2019). From my point of view, changing Step 1 to pass/fail may only worsen this segregation, as programs will have less information about candidates’ abilities and may be more inclined to worry about their reputations since they won’t be able to plausibly justify the recruitment of non-USMDs as being due to astronomically high Step 1 scores.   

​

If we really want to reduce stress among trainees and improve inequality between applicants, making Step 1 pass/fail is only an anodyne. To really help level the playing field, program directors could review applications blindly (without knowing the applicants’ medical school of origin) and compare USMDs and non-USMDs “numbers to numbers,” as one respondent put it. That would, of course, necessitate keeping Step 1 as a three-digit score—or perhaps, developing a new metric actually designed to measure applicants’ suitability to residency instead of Step 1, which is meant to signal minimum proficiency for licensure. Another approach could be to titrate recruitment to reflect the broader residency workforce, with American-trained allopathic physicians filling around 60% of residency positions nationwide. As for reducing stress among prospective residents, making Step 1 pass/fail does not address the broader hidden curriculum that paints some specialties as more competitive than others, nor would it attenuate the culture of having to constantly outperform one’s peers in medicine; instead, it will simply make space for new markers of distinction for students to obsess over. To really tackle these critical problems will require far more wide-reaching, structural change in medicine that any test can provide. I am pleased to see steps being taken to improve the trainee experience, but making Step 1 pass/fail may not represent the change we hope it will.    

 

​

*Tania Jenkins is an assistant professor of Sociology at the University of North Carolina – Chapel Hill and the author of the forthcoming book, Doctors’ Orders: The Making of Status Hierarchies in an Elite Profession (available for pre-order at https://cup.columbia.edu/book/doctors-orders/9780231189354).

 

For more information about the USMLE exam structure, visit https://www.usmle.org/

 

Works Cited

Jenkins, T. M., G. Franklyn, J. Klugman and S. T. Reddy. 2019. "Separate but Equal? The Sorting of USMDs and Non-USMDs in Internal Medicine Residency Programs." J Gen Intern Med. doi: 10.1007/s11606-019-05573-8.

​

Jenkins, Tania M., Jenny Kim, Chelsea Hu, John C. Hickernell, Sarah Watanaskul and John D. Yoon. 2018. "Stressing the Journey: Using Life Stories to Study Medical Student Wellbeing." Advances in Health Sciences Education 23(4):767–82. doi: 10.1007/s10459-018-9827-0.

​

Knopes, Julia. 2019. "Yields and Rabbit Holes: Medical Students’ Typologies of Sufficient Knowledge." Medical Anthropology:1-14. doi: 10.1080/01459740.2019.1640220.

​

Lorber, Judith. 1984. Women Physicians: Careers, Status, and Power. New York: Tavistock Publications.

​

Quadlin, Natasha. 2018. "The Mark of a Woman’s Record: Gender and Academic Performance in Hiring." American Sociological Review 83(2):331-60. doi: 10.1177/0003122418762291.

​

Rivera, Lauren. 2012. "Hiring as Cultural Matching: The Case of Elite Professional Service Firms." American Sociological Review 77(6):999-1022. doi: 10.1177/0003122412463213.

​

Rivera, Lauren. 2015. Pedigree: How Elite Students Get Elite Jobs. Princeton: Princeton University Press.