Strategies for Achieving Diversity through Medical School Admissions
The relative lack of diversity in medicine is a rate limiting factor in efforts to eliminate health care disparities. Many medical schools struggle to matriculate student bodies that reflect the diversity of this country. Actively recruiting is one tactic to diversify a medical school's applicant pool, but in isolation is not enough. Our medical school admissions committee made a number of programmatic changes that contributed to our current compositional diversity that may be instructive to others. This report from the field on the experience of one U.S. medical school describes several admissions committee initiatives that can be undertaken to increase the yield of students from groups underrepresented in medicine who matriculate to medical school.
Diversity, underrepresented minority, implicit bias, medical school, admissions
The relative lack of diversity in medicine is thought to be a rate-limiting factor in efforts to eliminate health care disparities.1,2 However, many medical schools continue to struggle to matriculate student bodies that reflect the diversity in this country. According to a 2012 publication by the Association of American Medical Colleges (AAMC), after excluding the historically Black medical schools and those located in Puerto Rico from the 127 member schools analyzed, only 28% had more than 15 African American matriculants across all four classes, 25% had more than 15 Hispanics, and only 20% matriculated at least one American Indian or Alaskan Native.3 This discussion will focus on admission committee initiatives that can be undertaken to increase the yield of students from groups underrepresented in medicine who matriculate to medical school. For the purposes of this paper and at our medical school we define underrepresented in medicine minorities (URM) as students who self-identify as Hispanic, Black or African American, American Indian or Alaskan Native, or Native Hawaiian or Pacific Islander. [End Page 9]
Activities to grow and strengthen the K–12 student diversity pipeline should also continue but are beyond the scope of this article. In 2009, Ohio State University College of Medicine matriculated a class of 45% women and 13% underrepresented minorities, similar to the national averages for both metrics. Motivated by our own institutional experience and Saha's paper describing the positive correlation of student body diversity with graduating physicians' self-rated preparedness to care for patients from underserved communities, we made diversity enhancement a major focus.4 By 2016, our entering class was 54% women and 26% underrepresented minorities and the U.S. News and World Report ranked our medical school as tied for ninth among nearly 140 medical schools for enrollment of African Americans,5 This evolution was the result of senior leadership alignment including the university board of trustees, president, medical center chief executive officer, dean and dean's staff in full support of diversity and inclusion. In addition, the associate dean for admissions completed the AAMC Holistic Review workshop facilitator training and the admissions committee participated in the Holistic Review on-site workshop and received implicit bias awareness and mitigation training. These efforts were complemented by collaboration with stakeholders including the Office for Diversity and Inclusion and student organizations such as the Student National Medical Association, the Latino Medical Student Society, and the Women in Medicine group. Since other medical schools attempting to enhance the diversity of their student bodies may benefit from reviewing the steps we took, we describe our efforts here and welcome comments. Figure 1 provides a graphic display of the time course of our admissions process vs increased class diversity.
1. Craft admissions mission/vision statement that speaks to diversity enhancement and keep the statement highly visible at all times
A primary tenet of a holistic admissions process is that it be mission-driven; that is, major decisions are based on an overarching mission that has buy in from key stakeholders. Not only can a mission statement provide a guidepost for each step on the front end of the medical education continuum—whom to interview, how to grade the interview, and whom to accept—but a medical school's mission statement can influence the practice patterns of its graduates.6 While medical schools and medical centers typically have wordy, multi-pronged mission statements, with regard to admissions we feel a succinct summary serves our purpose better. To craft a statement, we reached out to people in our medical center who have a vested interest in the quality of the students entering our medical school. The associate dean for admissions, the vice dean of education and the dean of the college of medicine met and defined key stakeholders as faculty (clinicians, educators, and researchers), administrators, current students, and alumni. Diverse representatives from these groups were asked to name three characteristics that, in their opinion, should be reflected in our matriculates. Compassion, intellect, diversity, and the potential to innovate and think critically were mentioned most often. The most commonly mentioned criteria were combined with the main theme of our medical center's mission and used to craft an admissions "mission/vision statement." Because the medical center's mission statement is approved by our university's board of trustees and is in turn encompassed in our admissions statement, we have alignment and unity [End Page 10] of purpose from our medical school admissions committee through the top leadership of our parent institution. The brief, two sentence statement is highly visible to those working in admissions: it is discussed at our annual training sessions for our admissions committee and screeners, it is printed inside the covers of the training manuals for both groups, and it is emblazoned on two large placards displayed at each voting session of our committee. Having this statement changes the critical question asked by voting committee members from "What do I think of this candidate's qualifications?" to "How well does this candidate fit our mission statement?"
2. Make voting anonymous with audience response system
Several years ago the voting sessions of our committee were open with hand-count voting. While this operated smoothly most of the time, it encouraged group-think and hampered chances of everyone voting their conscience. For our committee, anonymous voting has been critical to achieving the maximum benefit of the "wisdom of the crowd," which requires that each individual thinks independently and that private thoughts are grouped into one decision.7 Because our committee is large and has broad representation it is not unusual to have at the same meeting a clinical section chief, a basic science department chair, and a medical student. If a senior faculty member argues passionately that a candidate is not competitive it is unlikely that a medical student or a junior faculty member will raise their hand in opposition. In 2010 we switched to secret balloting using an audience response system. After the most heartfelt speeches in favor of or opposing a candidate's admission we now hold a secret vote with the results displayed for all to see. In this process all are free to vote as their judgement dictates and the audience response system (Turning PointTM) tallies a universal score for the candidate.
3. Put together a sizable group of faculty application screeners to minimize impact of individual biases
The initial "screening" process involves the use of trained faculty members to review medical school applications to select students to invite for interview. Since the pool of new medical students will be limited to those who are evaluated favorably during the screening process, this is arguably the most critical part of admissions. At our institution circa 2009 screening was performed by two people: one staff member and the associate dean for admissions, and conversations with colleagues around the country indicate that this was not unique then or now. With such a small number of screen-ers, the impacts of biases are amplified significantly. For instance, if there are only two screeners and one has a bias against non-traditional applicants, then half of all applicants in this category will be disadvantaged. The same is true for racial, ethnic, and gender biases. If multiple people are involved in screening, the impact of their individual biases is diluted and therefore reduced. It is also true that even with the best intentions, with two or three people reviewing thousands of files it is likely that the review is cursory and heavily based on academic metrics. After a critical review of our process we decided to expand our group of screeners. We now have 60 faculty members [End Page 11] participate in this process. Colleagues who ask, "How did you get such a large group of faculty to volunteer to be screeners?" are surprised by our answer: "We asked." We have found that physicians are eager to play a role in the selection of new physicians and will gladly volunteer their time when feasible. When we first sent a mass email to recruit faculty to review and grade medical school applications, we were surprised by the robust response. Several remarked that they had been wondering how to get involved with medical school admissions for years, but that it seemed to be a well-kept secret. We suspect that other medical schools will have a similar experience. Each year, screeners who review the most files are promoted to the full admissions committee, so the group of screeners has become something of a farm team for the admissions committee. The large number of screeners allows us to reduce the commitment; each screener is responsible for reviewing five files per week, a pace that allows time for a "deep dive" into each file. Our faculty screeners are trained annually in a two-hour session that covers the philosophy of holistic review (as defined by the Association of American Medical Colleges), the concept of implicit bias in admissions, and the specific enrollment goals of our college of medicine. We now believe that screening is far too critical to be delegated to a handful of people and are pleased to know that all interviewees have gone through a thorough, holistic examination of their application.
4. Adopt holistic review
The AAMC Holistic Review in Admissions project has been a critical component of our diversity enhancement strategy. Simply put, the strategy calls for placing an equal emphasis on the candidate's experiences, personal attributes, and academic metrics.8 In spring of 2011 we were one of several medical schools to host the AAMC Holistic Review in Admissions on-site workshop in which AAMC staff and admissions professionals from around the country led an eight-hour session on the use of holistic review in admissions. Although our approach was compassionate and somewhat holistic before, the workshop provided a starting point make formal the adoption of a holistic approach to the review of applications. Prior to the adoption of the formal AAMC holistic review process, committee members interviewed and presented candidates for deliberation and there was no specific instruction to place an equal emphasis on the candidate's experiences, attributes, and academic metrics. Rather, committee members decided on their own how to weigh these categories. While there was some initial hesitation by some of our faculty to support a system that seemed to de-emphasize academic metrics, they were quickly won over by the quality of the students entering our school under this new system. While intellectual prowess as measured by grade point average (GPA) and scores on the Medical College Admissions Test (MCAT) will always play an important role in the evaluation of medical school candidates, holistic review places academic metrics in the proper perspective. We use an evidence-based threshold for our academic metrics (i.e., an MCAT level below which students have tended to struggle in our curriculum), but students with MCAT scores significantly below our class average but above our threshold are frequently accepted, if they fit our mission and their experiences and attributes are clearly outstanding. Paradoxically, we found that when we moved academic metrics from its previous up-front position in our [End Page 12] admissions filter and placed it on equal footing with experiences and attributes, our class MCAT average increased. Others have written about a similar impact of holistic review on class metrics and diversity.9 As mentioned earlier, we believe that the most critical portion of admissions is screening; this is also the step at which holistic review has the greatest impact. If an institution screens largely based on GPAs and MCAT scores and reserves holistic review for the candidates that make it to campus for an interview, it is likely that hundreds or thousands of excellent candidates will be turned away.
5. Blind interviewers to academic metrics
Our medical school uses an evidence-based cutoff below which applicants of any race/ethnicity or gender are not considered further. In each entering class from 2009 to 2016 we have matriculated students with a wide range for GPA (3.0–4.0) and MCAT (24–41). In our previous system, committee members would review the entire file of the candidate, including grades and MCAT scores, prior to the interview. It became apparent to our leadership that the grading of the interview was being influenced by a priori knowledge of the candidate's academic record. If the interview question was, "Tell me what you learned about service and compassion when you volunteered at the nursing home last summer," and the interviewer was aware that the student's MCAT and GPA were significantly lower than our class average, even the most erudite answer was often deemed inadequate. On the other hand, superficial answers were more likely to be graded "outstanding" if the candidate had a 4.0 GPA and a very high MCAT score. As we tell interviewees in our welcome session, "We don't need to know your grades to determine if you are a good communicator, are compassionate, or if you have a deep understanding of your research project." Of all modifications of our system over the last few years, this one was met with the most resistance. Some interviewers felt that excluding the academic record from the file was depriving them of information critical to their being able to render a disposition. Proponents of providing grades and MCAT scores argued that the information was not used to profile candidates pre-interview, but to determine key attributes that we value in our medical school, such as persistence (the student who took the MCAT multiple times) or willingness to challenge oneself (the student who took a particularly challenging course that was not required for the major or for graduation). We acknowledged these points, but countered that these attributes can be judged elsewhere in our comprehensive evaluation process. Studies of metrics-blinded interviews in both undergraduate and graduate medical education support our concern that knowledge of the academic record prior to the interview influences the judgement of non-cognitive attributes.10,11 Beginning with the 2012–2013 cycle our interviewers evaluate the candidate's experiences and personal attributes without knowledge of their academic achievements. Through workshopping this concept at our annual training session and thoughtful discussion at our deliberation and voting sessions, our committee at large has become comfortable with this strategy. Interestingly, some interviewees remain skeptical; our interview-day survey asks our candidates, "How do you feel about the fact that at the time of your interview, your interviewer did not know your MCAT score or GPA?" Four years' worth of Likert-like responses on the anonymous survey indicate that approximately 70% feel "very positive" or "positive" [End Page 13] while 30% have a "negative" or "very negative" opinion about this strategy. We plan to explore this further in future surveys of our interviewees. We feel that this strategy is just, fair, and indispensable from our efforts to have a holistic admissions process.
6. Have committee take IAT, review aggregate results
The implicit association test (IAT) is widely used to detect biases outside of an individual's conscious control.12 A result on the Black-White IAT revealing that the subject unconsciously associates images of White people with positive words and Black people with negative words is referred to as implicit White race preference and indicates an unconscious preference for White people over Black people. The concept of implicit bias influencing behavior has recently gained a great deal of attention in academic medicine. Physicians appear to have the same implicit racial biases as laypeople,13 and several studies suggest that a clinician with implicit White race preference can adversely affect the quality of care that African Americans receive.14–16 If admissions committee members have the same biases, the implications could be important for efforts at diversity enhancement in the medical profession. In 2012 we had our admissions committee take three IATs; the Black-White race IAT as well as the heterosexual-homosexual and the gender-career stereotype IATs. Our recent publication discusses the results of the Black-White IAT, which revealed that the majority of our committee displayed moderate levels of implicit White race preference; men and faculty had the largest bias measures.17 Left unchecked, these biases may contribute to maintaining the status quo of severe underrepresentation of certain minority groups in the medical profession, which can be associated with racial health care disparities. We discussed the aggregate results at our annual admissions committee retreat/orientation and had a brief presentation on strategies to reduce the impact of unconscious biases. The class that matriculated after the IAT exercise was the most diverse in our school's history at that time. Though there are several alternative explanations, our findings suggest that this exercise in self-awareness may have resulted in a more inclusive climate perceived by medical school candidates resulting in an increased yield of URM students who decided to matriculate. We now provide reading materials, online presentations from the AAMC, and in-person lectures and workshops on implicit bias and bias mitigation strategies to our committee and faculty screeners. We are fortunate that our university has implicit bias expertise outside of the college of medicine, such as our psychology department and law school; consulting with these experts has proved valuable for creating the implicit bias training workshop for our admissions committee. We propose that all medical school admissions committee members be required to take the IAT and receive training in how to reduce the impact of these biases.
7. Remove photos from files when discussing applicants
Our candidates are instructed to bring a small photo of themselves on interview day to be attached to their file for identification purposes. While continuing to explore ways to promote objectivity in our admissions process, we had discussed removing [End Page 14]
Click for larger view
View full resolution
the photos for at least two years prior to making it official policy; one incident in particular made us act. A woman candidate was being presented to the committee by a woman committee member whose disposition was that the candidate was average in competitiveness. This was at odds with the evaluations of the candidate's two interviewers (in our system each candidate has two one-on-one interviews and days later a third committee member presents the candidate to a larger group for voting), who deemed the candidate to be outstanding. The interviewers were both men. When questioned about the discrepancy between the interviewers and the deliberations presenter, the presenter suggested that the applicant's attractiveness could have led to a lack of objectivity. The photo of the applicant was then held up for all to see and showed an attractive woman in what seemed to be a glamour shot with her long hair apparently being air-blown. There was a chuckle in the conference room and, it seemed, a sentiment that this was an average applicant whose interview score had been upgraded due to physical attractiveness. It is unclear how much this discussion affected the secret vote, but for the leadership this was the tipping point for deciding to remove photos from all applications. Since most biases, implicit and explicit, are triggered by visual stimuli, we decided that having the photograph of the candidate available when final dispositions are made is counter to our efforts to minimize bias. Our committee agreed readily to this change, with most reasoning that applicants who are obese or do not meet traditional standards of attractiveness are the most vulnerable to biases that could adversely affect their candidacy. Interestingly, a recent study indicates that, similar to [End Page 15] our example, those deemed attractive could also be at a disadvantage.18 While we still keep photos for identification purposes, they are kept separate from the applications until the candidate has been voted upon and has a final disposition.
8. Appoint women, minorities, and younger people (groups with less implicit racial bias) to committee
The Liaison Committee on Medical Education (LCME) standard 4.5 states that a medical school should provide opportunities "for professional development to each faculty member" in "program evaluation, student assessment methods, instructional methodology, and or research…" which can be interpreted to mean that each faculty member should have the opportunity to serve on committees, including the admissions committee.19 We agree with the spirit of this standard and seek nominations for a wide swath of the faculty to participate on the committee. Yet, given research findings that women, minorities, and younger people have less implicit racial bias,9 we also think it prudent to consider populating the admissions committee with the diversity we seek in the student body: if a class of 50% women is the goal, half of the committee should be women; if we seek a significant percentage of underrepresented minorities in the class, to the extent possible this should be reflected in the makeup of the committee. In our report we found that men and faculty members had the largest measures of White race preference, with women and medical students having lower measures.12 This finding—coupled with the fact that African American physicians tend to have less implicit racial bias than White, Asian, or Hispanic physicians10—informs our strategy of recruiting for diversity on the committee just as we recruit for diversity in the student body.
Support from senior leadership and alignment with our institutional mission have been critical to the implementation of a step-wise approach to improving medical student diversity. Increasing diversity in U.S. medical schools will result in enhanced diversity in a workforce that provides culturally competent health care to all populations. We join others in calling for a collaborative spirit among academic medical centers in addressing this issue.
QUINN CAPERS, IV, is Associate Dean for Admissions and Associate Professor of Medicine (Cardiovascular Medicine); LEON MCDOUGLE is Chief Diversity Officer and Associate Professor of Family Medicine; DANIEL M. CLINCHOT is Vice Dean for Education and Professor of Physical Medicine and Rehabilitation, all at The Ohio State University College of Medicine.
The authors would like to acknowledge former and current members of The Ohio State University College of Medicine admissions committee and especially the admissions committee chairs whose assistance with enacting the steps described above has been invaluable: Drs. Howard Werman, Robert Murden, Mark Debard, Anne Taylor, and Minka Schofield. [End Page 16]