Research in medical education

I had a crazy thought the other day,  why don't we do more research in medical schools. I know you're saying that there is a ton of research going on in the modern medical school.  But I bet if you are reading this you don't care that much about most of the research that is going on. Sure, there is plenty of basic science research. For instance, my good friend and co-conspirator/ co- director of our first medical school module is the famous Joe Fontes. He is in the Department of Biochemistry and he has two RO1s from the NIH.  Pretty impressive!  We also do some clinical research. One of my partners has an RO1 from the NIH to study colon cancer screening in minority populations.

But I am talking about research on and about the educational process. How much of what we do on a daily basis is evidence based?  I don't mean the knowledge content that is being delivered but the way that the information is delivered.  Is a lecture better than a small group?  Is an experienced lecturer better than one who is young and inexperienced?  Is a preceptor visit a better way to learn about professional behavior than lecturers. Are preceptorship visits better for teaching clinical skills or faculty physicians or trained patients?

These are all important questions. Why don't we know the answers to them?  

You might say that we know the answer to some of those questions. I would say that we probably know part of the answer to some but we don't know the whole answer to any of them. But does that matter? I think it does. We talk all the time about how we think evidence-based medicine is important but do we apply the same standard to the education of our students? I don't think so. I am not even sure that we could all agree on the definition of "better".

But don't you think we should have an evidence-based standard for educational practice just like we have for clinical practice?  This is what I propose. We need a medical school that is totally based on the best available educational research. Wait a minute, that won't work, will it?  We already know that there is not enough evidence for most of what we do. So, if we can't do it now maybe we could in the future. But how can we get there? Here is the radical part. We need to experiment on our students. (a big gasp was heard!)

The funny part is that I think we already do this. But right now we are not getting their permission.  Every time that we use an unproven educational method in our teaching, we are experimenting on them. We just don't get their consent.  I propose that we start asking for consent and randomize them to the standard educational method or to a well thought out intervention that is based on sound educational theory. This will require us to have a good grounding in educational theory which most physicians don't. And it will require an acknowlegement by us that our methods may not be right. We might need to change.

If I was a med student, I would be excited by this. Our students in Salina (see my blog about Salina) were told at the beginning that they would be the innovators, not guinea pigs. It was and is exciting to them. They were excited to be the first to prove that a medical school could be run in a small town in rural Kansas.  I think that the same would be true on other campuses with other methods. Let's be innovators.  

What do you think?

Empathy

A recent study Rosenthal, et al (1) in the journal Academic Medicine, studied the Jefferson Empathy Scale in medical students at the Robert Wood Johnson Medical School . This study was born out of a committee that included medical students, residents, and faculty. Their purpose was to design a curricular intervention that could be done in the third year of medical school that would help to preserve empathy throughout the year. 

The purpose of the study was "to evaluate JSPE-MS scores of two consecutive medical school classes in order to assess the impact of an empathy-preserving curricular innovation". At RWJ Med school they believed that there is a decrease in empathy across the third year of medical school. Anecdotally, I think that most medical educators know this to be true.

Wiggleton and colleagues (2) found moral distress, burnout, and depression in third year students. In this study, the authors described 50 potentially distressing situations which medical students might encounter in clinical rotations. Situations included: a patient had very advanced disease because they faced barriers to accessing care; a member of the team was disrespectful to someone below them in team ranking; optimal care was not provided as a result of alcoholism, drug use or homelessness; and our team provided care that only prolonged a patient's suffering. Over half of these situations had been experienced at least once. 35% of the situations caused mild to moderate distress.  

Diseker and Michielutte (3) found that empathy decreased before and after clinical experiences in the third year. The author's administered the Hogan empathy scale to all medical students. They found the empathy scale to be negatively correlated to MCAT scores. (see my previous blog about the MCAT). And they found that there was a significant decline in the empathy score from the beginning of medical school to the end.

Hojat and colleagues (4) found empathy significantly declined across the third year of medical school. They found that empathy did not really change that much in the first and second year, but the decline during the third year persisted until graduation. Interestingly, 27% of the students did not have any decline in empathy.

So, in this study the authors designed a curricular intervention that was given in the clinical year. Their intervention was six interactive sessions that were given during each of the required third-year clerkships. The sessions were one hour long and included time for debriefing on intense experiences, reflective essays/blogs,  and discussions of role models, patient care, morally distressing events, and the students' reactions.

This intervention seems fairly simple and similar to a longitudinal experience that our school has in the third year of medical school. The authors were able to document a lack of change in empathy across the third year. There was no statistically significant change in pretest/posttest empathy scores (pre = 115.4, post = 113.9, p =.135). One interesting fact was that student who entered family medicine, internal medicine and pediatrics had significantly higher empathy scores than those entering other specialties such as surgery, urology, otolaryngology, and anesthesiology.

Hopefully, there will be more interventions designed to help our students during the third year. It is hard. They are vulnerable and are often treated like crap. Dr. Steven Kanter (5) in his opening editorial in the March issue of Academic Medicine, reminds us that we need to think with our head as well as our heart to provide the best patient care. If we damage our students they will no longer have the ability to think with their heart, to care for their patients.  OK, I will get off the soap box for a while.

REFERENCES

(1) Rosenthal S, et al Humanism at Heart: Preserving Empathy in Third-Year Medical Students. Acad Med 2011;86(3):350-358.

(2) Wiggleton C, et al. Medical students' experiences of moral distress. Acad Med 2010; 85:111-117.

(3) Diseker RA, Michielutte R. An analysis of empathy in medical students before and following clinical experiences. J Med Educ. 1981;56:1004–1010.

(4) Hojat M, et al. The devil is in the third year: A longitudinal study of erosion of empathy in medical school. Acad Med. 2009;84:1182–1191.

(5) Kanter S. Think With Your Head and With Your Heart. Acad Med  2011;86(3):273.

Professional dress: does it matter any more?

Sorry, I have been absent for a couple of weeks.

We started a new class of students off on their medical journey last month. The first year students arrive on campus bright-eyed and bushy tailed as it were. At our school most are fresh out of college. They have spent the last four or five years as Biology or Chemistry majors. They went to class or maybe they didn't. Most college courses don't have attendance requirements. They are generally allowed to dress however they want, this time of year shorts, flip flops and ball caps are the norm.

In 2005, there was a big flap (or should I say flip/flap) when the national championship women's Lacrosse team from Northwestern University was invited to the White House to meet President George Bush.  The scandal began when several people noticed that a picture taken of the ladies showed four of the nine players in the front row wearing flip flops.  Now these ladies were dressed up in skirts and nice clothes. After all they were meeting the President, but dressing up apparently did not include changing shoes.

 

So here we are with another class of new medical students. I am the co-director of the first module so I am sitting there in class most every day. The students are polite, they almost always address me as Dr Delzell, and so far I have not seen any rude behavior in class. But at least half the class is wearing flip flops. T-shirts are the norm. Many extolling their undergraduate school or their fraternity/sorority. Many of the guys wear ball caps. 

Now don't get me wrong, I like to dress casual. As soon as I get home from work I put on shorts and a t-shirt. I love to wear flip flops. I would love to have a job at a medical school that is located on the beach so I could wear casual Hawaiian-style shirts and flip flops every day.  But I don't.  And neither do our students. 

I know that there are some schools that require professional dress whenever the students are on campus. Dr. David Steele, Senior Associate Dean for Medical Education at the Paul L Foster Texas Tech School of Medicine in El Paso has told me that at their new medical school the faculty decided to require students to dress professionally every day. Even during the basic science lectures. And last year, we were invited to be visiting professors at the Yerevan State Medical University in Yerevan Armenia. We visited the campus and toured one of their large lecture halls that would hold about 600 first year medical students. It had hard wooden benches and no air conditioning and the students were required to dress up (suit and tie for the men) each day for lecture.

I don't know if it makes a difference. We talk all the time about how Millennials-Generation Y is different from past generations. I am sure that in the sixties when the hippies started their first day of medical school, they were wearing bell bottoms and tie-dye shirts. I am sure the professors were concerned about the lack of professionalism that those students displayed.  This is to some extent a generational issue that is seen every year. But where is the line?  When is it a generational issue-where the younger generation have a different set of internal rules and values that guide them in different ways than a previous generation? And when is it a maturation issue-where you need to learn behavior from those that are your teachers and mentors?

New Salina med school campus unique in U.S.

The following is an excerpt (posted with permission) of an article by Dave Ranney from the Kansas Health Institute News Service that was originally posted on July 5, 2011 on the KHI website. Dave Ranney graciously gave permission for this reposting.

SALINA — Next month (actually tomorrow), the University of Kansas School of Medicine will open a four-year, fully accredited school – officials prefer to call it a campus – next door to the Salina Regional Health Center.

“This will be the smallest medical school campus in the country and Salina will be the smallest city in the country - outside of a few major suburbs - to have a medical school campus,” said Dr. Heidi Chumley, senior associate dean for medical education at KU Medical Center.

The first class will have eight students, seven of whom are from Kansas.  KU Medical Center officials said they plan to add eight students a year at the campus in each of the next four years.

More primary-care docs

“The goal is to develop more primary care doctors for rural Kansas,” Chumley said. “We’re shooting for 75 percent (of the new school’s eventual graduates) choosing primary care, and 75 percent rural.” Much of the new school’s curriculum will be tied to the classroom offerings at the KU Medical Center’s campuses in Kansas City and Wichita. “We completely redid our curriculum about six years ago. It’s very computerized now,” said KU Medical Center Executive Vice Chancellor Dr. Barbara Atkinson. “All the lectures are podcasts. They’re all going to be teleconferenced (in Salina), though some will be generated on-site.”

The cost of remodeling the three-story building has been picked up by Salina Regional Health Center. And the hospital’s foundation and several private donors are covering many of the operational costs and scholarship offers. “So far, we’ve not asked the state for any money for this because we understand the financial situation the state is in,” Atkinson said. “But we have asked donors to support it and they’ve been very, very generous. Incredibly generous.” Salina-area benefactors, she said, hope to raise $2.5 million over the next four years. They’ve already raised $1.5 million with $1 million coming from the Salina hospital.

Salina Regional Health Center has hosted a residency program for KU Medical School graduates for about 30 years.  Most of its residents went on to start or join rural practices.

Model for other states

“What the University of Kansas is doing, I think, will be a template for having a positive impact on the number of medical practitioners in rural communities,” said Brock Slabach, senior vice president at the National Rural Health Association. “Other universities will be watching because, really, for a major medical school to commit itself to meeting rural-community needs like this is truly novel. It shouldn’t be, but it is.”

Slabach said he’s long been baffled by the fact that medical school officials in many rural states’ fail to see the connection between their states' shortages of health care providers and their students leaving for big cities in other states. “I wouldn’t include Kansas in that group,” he said. Last year, a national survey of how well medical schools were fulfilling their “social mission to train doctors…” ranked KU School of Medicine fifth in the nation. The school was ranked eighth for its percentage of graduates (44 percent) practicing in underserved areas.

Earlier this year, a Kansas Department of Health and Environment survey found that 51 of the state’s 105 counties had less than one physician per 2,695 residents and were considered medically underserved.

Looming retirements

“The shortage of health care professionals has been a critical issue for a long time,” said Dr. William Cathcart-Rake, director at the KU-Salina campus. “It’s not new, but what is new is that now we have a number of physicians who are nearing retirement age, and a good number of them are practicing in the rural communities.” Generating enough new doctors to replace those who are retiring – especially in rural areas - will be difficult, he said. “We have to do something,” Cathcart-Rake said. “We can’t keep doing what we’ve always done. It’s not enough.”

The Salina campus, he said, will be geared toward allowing students from small towns to complete their studies and residencies in a small-town environment. “There is evidence that shows that if someone is trained in a rural area they are more likely to stay in a rural area,” he said. “So the idea is that from day one we’ll be training our students in in non-metropolitan settings and exposing them to all the good and the bad that comes with life in rural Kansas. After that, we’ll hope for the best.”

Cathcart-Rake, who grew up in small town in Orange County, Calif. (“…back when there was still a small town in Orange County”), has been practicing medicine in Salina for 32 years. “The perception is that if you go to a small town you’ll work yourself to death, you’ll never get a vacation and you won’t get to spend time with your family,” he said. “The way to get around that is to be with a group of physicians so you can cover for each other so you don’t have to do everything for yourself.

Going where no medical school has gone before....

On the front page of the Saturday (July 23, 2011) New York Times, an article by A.G. Sulzberger proclaimed Small-Town Doctors Made in a Small Kansas Town.  This week, the University of Kansas School of Medicine is opening a new campus in Salina, Kansas.  Salina will be the smallest town in the US to host a four-year medical campus. The stated purpose of the new campus is produce primary care doctors that will practice in rural Kansas. Dr. Heidi Chumley, the KU Senior Associate Dean for Medical Education and Associate Vice Chancellor for Educational Resources and Interprofessional Education, is leading KU's development of this campus with strong support from Chancellor Bernadette Gray-Little and EVC / Executive Dean Barbara Atkinson. The fact that this article ran on the front page of the NY Times is a testament to the uniqueness of this new campus. The LCME reviewers of the Salina program thought that it was "an interesting model" that "could be a stimulus for other schools".

As you might expect, the program has generated a lot of interest:

Dave Ranney from the Kansas Health Institute had a nice article on the KHI web site on July 5, 2011, New Salina med school campus unique in U.S. I will be reposting an excerpted version of this article in the next few days.

Elana Gordon of KCUR, the University of Missouri-KC NPR affiliate, did a nice article about the new campus, Tiny School to Create Tiny Town Docs. This ten-minute interview with Dr. Chumley ran on the air and a podcast is posted at this link.

Roger Cornish, from KWCH 12 Eyewitness News (a Wichita CBS affiliate), has a great interview with Kayla Johnson and several of the other new first year medical students at the Salina campus. You can watch the video of the interview titled, Classes begin at K-U Med / Salina.

Lily Wu from KAKE (another Wichita TV station) did an interview of the students and staff at the new campus, Medical Students Interested in Serving Small Towns.

I am sure that there will be more press on this over the next few days and weeks. Congratulations to Dr. Chumley and the University of Kansas for this historic endeavor.

Follow up

I wanted to follow up on some issues that I have blogged about recently.

On February 4, 2011, in my post Why do we put so much import on the MCAT , I wrote about the value of the admissions test for medical school. The Medical College Admissions Test is one of several screening mechanisms used by medical schools to determine who should be allowed to pursue medical training. A recent editorial (1) in the New England Journal of Medicine by Joshua Tompkins (Science Journalist and Medical Student at the USC Keck School of Medicine) discussed the expense of MCAT and Board prep courses. According to Mr. Tompkins, medical students are taken advantage of by a "multi-million-dollar industry in commercial exam-preparation assistance". These (for-profit) companies play on student's fears--fears of failing, of not getting a high enough score to get into their residency of choice, their fear of not matching in any residency.  The problem is that there is little to no evidence that these board prep courses actually do anything to affect a student's score on the MCAT  or USMLE step 1.  A systematic review of commercial test preparation (2) found that "current research lacks control and rigor" and the evidence to support these courses was weak or non-existent. According to Mr. Tompkins, pre-clinical medical students "focus on obscure minutiae and rare conditions" and "spend less time studying the common diseases they will face during clerkships, residency, and practice".

And while we are talking about medical school admissions. In another recent blog, Personality traits that predict success in medical school , I wrote about using personality profiles to chose which students should be in medical school. An article in the July 10, 2011 New York Times by Gardiner Harris, New for Aspiring Doctors, the People Skills Test reports on the use of a specific type of medical interview that is being implemented at Virginia Tech Carilion School of Medicine in Roanoke, VA. The Dean at Virginia Tech Carilion is a family doctor named Dr. Cynda Johnson. She happens to also be a former resident and faculty member from the University of Kansas. Virginia Tech Carilion (and several other schools) are using the multiple mini interview or MMI to screen potential medical students. The test is administered to students as part of their medical school interview and is designed to "assess how well candidates think on their feet and how willing they are to work in teams". According to Dean Johnson, "if people do poorly on the MMI, they will not be offered positions" in the medical school class.  The MMI is also a pretty good predictor of performance in the preclinical years of medical school. (3)

Way to go Virginia Tech. This is definitely a step in the right direction.

References:

(1) Tompkins J.  Money for Nothing? The Problem of the Board-Exam Coaching Industry. NEJM 2011; 365 (2): 104-105.

(2) McGaghie WC, Downing SM, Kubilius R. What is the impact of commercial test preparation courses on medical examination performance?  Teach Learn Med 2004; 16 (2): 202-11.

(3) Eva KW, et al.  The ability of the multiple mini-interview to predict preclerkship performance in medical school. Acad Med  2004; 79 (10 Suppl): S40-2.

A new blog: A Journey to Family Medicine

I would like to introduce you to one of my students. Brooke is a brand new fourth year medical student. I have known Brooke since the first month of medical school. She was assigned to my PBL (problem-based learning) group. This means that she and nine of her classmates were going to be meeting with me every month or so to work our way through a clinical case. Every basic science module has PBL cases. There are multiple purposes for these PBL groups: to learn about clinical medicine, applying basic science concepts to a patient case, to learn about the professional aspects of medicine, and to foster and promote interactions with medical school facutlty members. We get to do all of that and more.

The PBL groups are definitely one of my favorite parts about being a teaching faculty member. Our students come into school with such high expectations and aspirations. They are not cynical or jaded (yet). They have a lot of different motivations for medicine but generally have one thing in common--they want to help people. Usually, by the end of two years, I know these students pretty well. I learn about their families. It is great getting to know them individually as people, not just as students. I don't try to talk them into going into primary care, but they do get two years of listening to my opinions (for what that is worth).

So Brooke was in the last Family Medicine clerkship this year. By the time, I saw her, she had already taken Internal Medicine, Surgery, Geriatrics, Neurology, Psychiatry, Pediatrics, and OB/GYN. The students at the end of the year are an interesting group. Clinically, they are very accomplished. But often they are stressed. They are trying to decide what residency program they should try to get into. They are tired after a year of long clinical rotations. And, honestly, they are often a little cynical about the educational process and medicine. There is data that says that students' empathy and emotional intelligence falls throughout the third year.

But Brooke was excited. She could not wait to come to Family Medicine. She told me the first week that she had been thinking about it for a long time and had decided that Family Medicine was the specialty that she was going to choose. She is smart. She could do anything that she wanted to do. She is confident and poised. She is a class leader. Not formally, but informally. The other students on the rotation looked to her for leadership which she provided quietly without any fanfare. And she decided that she wanted to be a family doctor. I must tell you, that is one of the greatest thrills!

So, I asked her if she would write about her experience. She is going to blog from now until Match day and maybe beyond. She is going to write about her life and being a medical student. She is going to write about the process of applying to residency, interviewing, and the Match. I hope that if you work with medical students you will tell them about her blog. The name is: A Journey to Family Medicine.

How much should medical school cost?

It does not seem that many people care very much about the cost of medical education in the US. But we should be concerned. Did you know that the average debt for a medical student on graduation from a state funded medical school is about $150,000. To put that into perspective, the average cost of a four bedroom house in suburban Kansas City, where I live, is $225,517 not much difference.(1)

Many people will see that and say,"who cares, doctors are rich". I can't argue with that fact, doctors are rich. Physician salaries are in the top 5 percent of all jobs in the US. A recent blog from Dr Joshua Freeman described the top 10 paying jobs in America. Physicians held 9 of the spots. There are many ways to fund medical school, including loans, repayment plans, and my personal favorite-having a rich dad. I remember the first day of my medical school experience and one of the women in my class asked a question during the financial aid lecture. "Can you send the bill to my daddy?" Repayment programs can be very effective. Here in Kansas, we have one of the oldest and best. Students that commit to practice in an underserved county (that is most of Kansas), choose a primary care specialty such as family medicine, internal medicine or pediatrics, and maintain their academic standing are given a full tuition scholarship and a monthly stipend. Over its 50 year history the Kansas Medical Student Loan program has placed hundreds of students in communities across Kansas.

So back to why should we care about this? Well, as I discussed in a previous blog, Production of Primary Care Doctors, [ student debt is one of the primary deterrents to students choosing primary care specialties such as family medicine. The equation is fairly straight forward. More student debt leads less students choosing family medicine residency training (and other primary care specialties). Less students in family medicine residencies means less primary care physicians in practice. The data at this point is pretty clear. If there are less family doctors per capita in a county in the US, that county is less healthy. The citizens die more frequently, they are hospitalized more often, and over all they are less healthy.

Wow, all of that because medical school costs a lot? No, not all but enough that a recent Op-Ed in the NYTimes should be given some serious thought. Drs Bach and Kocher proposed that medical school tuition should be free. Then how would we pay for it? Their idea is a good one. Tax the students that want to choose specialty practices. Basically, the idea is that medical school is free if you choose to practice in a primary care field like family medicine or general pediatrics. If you decide that you want to be a dermatologist or a radiologist or another specialty then you have to pay for your training.

That may seem unfair to the students that want to go into dermatology, but remember the state and national government (and we the people) have a huge investment in medical education and the health care field. $1,400 for every car you buy from General Motors is for health costs, medicaid accounts for 1/3 of the budgets in most states, and even private medical school receive millions of dollars from the federal government to support biomedical research. And we don't need more dermatologists. We do need a strong and vibrant primary care infrastructure. The beauty of this idea is that it could be changed as the physician practice population changes. If we need more anesthesiologists then we make the payback less. If we need more general surgeons, then we make the pay back less to do general surgery. If we need less plastic surgeons then we just raise the payback more. How much? How ever much you need. Would a student choose to do orthopedic surgery if medical school was going to cost them a million dollars? How about 2 million? You get the idea?

Will this ever happen? Unlikely, but we can hope...

References
(1) http://money.cnn.com/pf/features/lists/hpci_data/index.html

The new (and improved?) MCAT

Some of you may have heard about the proposed revisions to the Medical College Admissions Test. What do you think about the changes?

The AAMC released the preliminary report from their MR5 advisory committee. This task force was appointed in 2008 with broad representation from medical school deans and administration, basic science and clinical faculty, premed advisors, and included a resident and a student. This group, chaired by Steven Gabbe, MD (Senior VP for Health Sciences THE Ohio State University) was given the task of making recommendations for changes and revisions to the MCAT "that are likely to increase the exam’s value to medical school admissions committees and examinees".(1) The last time that the MCAT was significantly revised was in 1991.

The MR5 Committee got feedback from faculty at medical schools, deans and administrators, resident physicians, and medical students.  In spite of this, some of the changes have gotten significant criticism. They have made fourteen recommendations, including an increased emphasis on Behavioral and Social Sciences Principles and Critical Analysis and Reasoning Skills. (2) 

As I wrote about in a previous blog, Why do we put so much import on the MCAT? , the MCAT has major problems when used as a decision point for medical school.  The MCAT may even be a negative predictor for some important characteristics of physician behavior, such as verbal fluency, breadth of interests, and the ability to communicate. (3) 

So, it seems that changes to the test would be welcomed. I don't know. Honestly, my major problem is not the composition of the test so much as the way that the test is used by medical schools across the country.  The MCAT is the primary determinant for students' admission to medical school, often overshadowing other important factors such as communication skills, altruistic intentions, and a service orientation.

I was surprised by some of the comments on the AAFP's website . These were comments that were posted by readers after an announcement about the MR5 recommendations. Some physicians seem to feel that the changes that are recommended by the MR5 committee will make students less qualified for entry into medical school. "I was always better at Chemistry and science than in fuzzy subjects that didn't require logical thinking" and "Medicine is a scientific profession which requires the high-caliber minds who can think scientifically and objectively". And then there was my favorite, "Is this another attempt at the academia trying to brainwash students?"

I think much of this concern comes from a misunderstanding of the value of the test. It also may have something to do with changes in medical education and the practice of medicine that many physicians are not ready to embrace.  Medical school needs to do a better job of preparing students for practice in environments that are team-based and collaborative, and use information at the point of care.  The days of a physician who acts and practices alone and in a vacuum are gone. We need students who can work with nurses, pharmacists, physical therapists, social workers, and lay patient educators. (4)  The days of a physician who knows everything about everything that they do are gone. There is too much information.  We need students who can access information at the point of care and interface with electronic resources at the same time as they interact with a patient.

So, what should be in the MCAT. I would like MCAT to test some of the important aspects that are beyond the science. Science is not that hard to teach. Make the science portion pass / fail. If you meet this level, we think you can pass USMLE Step 1.  But many other important topics are harder to teach. Why can't the MCAT have more emphasis on ethics?  It could have more questions about health policy and the politics of medicine.  I would love to see a personality profile measure built into the test. What about service and altruism. What about team work? Are there measures of how collaborative people are in team situations? Now some people will say a test can be scammed for those kind measures. I am sure that is true, but with the numbers of people taking the MCAT and the amount of resources that are available, I bet a valid and reliable test could be constructed that would measure more than just knowledge.

References:
(1)  https://www.aamc.org/initiatives/mr5/
(2)  https://www.aamc.org/download/182662/data/mr5_preliminary_recommendations.pdf
(3)  Gough HG. Some Predictive Implications of Premedical Scientific Competence and Preferences. J Med Educ  1978; 53: 291-300.
(4)  http://www.newyorker.com/online/blogs/newsdesk/2011/05/atul-gawande-harvard-medical-school-commencement-address.html

Production of primary care doctors

COGME has just released their 20th report, Advancing Primary Care. Unless you are a real geek like me, you probably don't know what COGME stands for. COGME is the Council on Graduate Medical Education. COGME was authorized by Congress in 1986 to provide an ongoing assessment of physician workforce trends, training issues and financing policies, and to recommend appropriate federal and private sector efforts to address identified needs. The legislation calls for COGME to advise and make recommendations to the Secretary of the U.S. Department of Health and Human Services (HHS), the Senate Committee on Health, Education, Labor and Pensions, and the House of Representatives Committee on Commerce. The Health Professions Education Partnerships Act of 1998 reauthorized the Council through September 30, 2002. Since then, the Council has been extended through successive annual appropriations governing the Department of Health and Human Services.(1)

Now, there is some history here.  In January 1994, COGME released its Fourth report, Improving Access to Health Care Through Physician Workforce Reform. In that report the authors took the position that the physician workforce was not meeting the needs of the US healthcare system and the public need. The report concluded that we needed more generalist physicians.  Their recommendation was that 50% of all graduates should enter practice as a generalist physician (Family Medicine, General Internal Medicine, and General Pediatrics). Their goal was to attain this by 2000. As we know that did not happen. In fact, it went the other direction.

But interestingly in their Sixteenth report, Physician Workforce Policy Guidelines for the United States, 2000-2020 (2005), COGME reversed themselves and said that market forces should determine the proportion of students in an in specific specialty. The exact recommendation was: The distribution between generalists and non-generalists should reflect ongoing assessments of demand; therefore, COGME does not recommend a rigid national numerical target.  Wow, what an amazing mistake that was.  Since that time, the proportion of students choosing family medicine and primary care in general has continued a downward trend. In 2008, the total proportion was down to 32%.  Clearly market forces are not working.

Dr Jerry Kruse, Chair of Family & Community medicine at Southern Illinois University who I wrote about in a recent blog was Chair of one of the writing groups for the COGME report.  According to Dr. Kruse, market forces didn't work because there is not a traditional supply and demand market in the US healthcare system. What we have is really a supply and supply market.  We have a virtually unlimited amount of care that can be delivered (whether it should be is a different discussion) and a funding pot that rewards doctors, hospitals, DME companies, etc for doing more (See Dr. Lin's post at the Common Sense Family Doctor). The rewards are financial and huge. So, the market is designed to reward doctors financially in specialties that do more technical and procedural things. (A more detailed discussion of this can be found in a recent post by Matthews and McGinty on the Wall Street Journal Health blog.) These are specialties such as, Radiology, Orthopaedic Surgery, Anesthesiology, and Dermatology. The "ROAD", as some medical students have taken to calling it.

Medical students are not stupid. They figured out that the financial rewards of practicing in the higher paid specialties were extraordinary. The median lifetime income gap between a student choosing primary care versus a specialty is 3.5 million dollars.(2) Currently, US primary care doctors earn about 55 percent of what specialists earn on average. When primary care doctors' salaries dropped as a proportion of specialists' salaries, interest in family medicine and other primary care areas also drop. The key number seems to be around 70 percent. If the income of primary care doctors as a proportion of the income of specialty physicians goes up then student interest goes up as well. The Altarum Institute estimates that increasing primary care income to 80 percent of specialty income would double medical student interest in primary care. This would increase the percentage of students choosing primary care to about 40 percent.(3) 

Canada had the same problem. From 1998 to 2004, they had a 25% drop in students choosing family medicine and it worried the Canadian health ministry.(4)  In a country that has universal coverage, it is vital that the primary care base is adequate. In Canada, they understand and believe that they need a specific number of family doctors in order to be able to take care of the populace. To address this national crisis (their words not mine) they invested in student interest by building up and supporting medical school family medicine interest groups (FMIG).  And they raised the salary of family physicians. They did not make family medicine and specialist salaries the same, but they raised the proportion to 87%.  By 2006, the median income of Canadian family physicians was $212,000 per year compared to the median annual specialty income of $245,000.(4)  That was enough. Interest started going back up. Medical students choosing family medicine has increased by 27 percent each year since 2004.

Why can't we do that in the US? There are so many reasons that I don't think I could even begin to cover them all but let me hit some of the highlights. We don't have a national universal coverage system.  Doctors are mostly self-employed or work for large healthcare organizations (like hospitals). The government does not directly decide how much doctors are paid.  (For more on this, read Dr. Freeman's post Outing the RUC: Medicare reimbursement and Primary Care) 

The most important reason is probably that we don't see this as a national crisis.  Most people think that we have the best healthcare system in the world. Unfortunately, the data does not support that. We have a mediocre healthcare system compared to the rest of the world by any measure. The primary care base is the key to the system. We need to and should adopt the COGME recommendations. If we don't have enough primary care doctors (translate=enough students choosing family medicine), the US population will be less healthy. There is no question. Our population will be less healthy, people will die prematurely, and it will cost more.(5)

References
(1) http://www.cogme.gov/whois.htm
(2) Wilder V, Dodoo MS, Phillips RL Jr, Teevan B, Bazemore AW, Petterson SM, Xierali I. Income disparities shape medical student specialty choice. Am Fam Physician. 2010 Sep 15;82(6):601
(3) Altarum Institute. (2009). Updates to BHPR phy­sician supply and requirements models. Presenta­tion to COGME, p. 15. Rockville, MD, from COGME 20th report
(4) Canadian Institute for Health Information. (2007, December 13). Physicians in Canada: Average Gross Fee-For-Service Payments, 2005-2006. Re­trieved May 11, 2010, from http://secure.cihi.ca/ cihiweb/products/FTE_APP_2005_Eng_final.pdf
(5) Phillips RL Jr, Bazemore AW.  Primary care and why it matters for U.S. health system reform. Health Aff (Millwood). 2010 May;29(5):806-10

Using interviews to select medical students

This is the fourth in a series of posts about entry of students into medical school. In the last post, I blogged about personality traits that may be better suited to being a medical student. Unfortunately, most schools in the US do not use personality profiles as a screen for incoming medical students.

At most schools, the closest that we come to this is the admissions interview. Many believe that a 30 minute interview is a good way to weed out bad apples. I am not sure that this is true. Just on the surface, it seems like an experienced interviewer may be able to identify highly dysfunctional people. By dysfunctional, I mean traits that would be obviously detrimental to their function as a physician. These obvious dysfunctional traits are things like: students who have difficulty talking to others, students who have flaws in their ethical approach to life, and students who have problems with their reasons for entering medical school. But what does the literature say?

Powis, et al (1) used a case-control design to study students who were admitted to medical school but did not graduate. They retrospectively analyzed 56 paired cases and controls. The cases were students who had left medical school due to failure or withdrawal, while the controls were students who had completed medical school. The controls were all students who had excelled in the their academic performance. The students who left medical school had all been rated lower at their admission interview. Effect sizes were statistically significant in the Overall rating (ES=2.17), self-confidence (ES=2.59), perseverance (ES=2.98), and tolerance of ambiguity (ES=1.04).

The Powis study used a objective and structured interview and they compared the students who left or were dismissed from medical school to those who received Honors in medical school. It is not clear that the admission interview would distinguish between failing students and anyone who would not fail. Admission interviewers have widely variable reliability. Powis found the inter-rater reliability varied from .23 to .63 for seven different qualities assessed by two faculty members. Other studies have found that reliability data is better for interview programs that use a structured interview process (.82 to .84) while with unstructured interviews the reliability is .61 to .75. (2)

So, reliability is not great, but seems to be better with more structure. Part of what provides structure is giving interviewers training and giving them types of questions to ask. But (and this is a big but)...I think that the interviewer has to be experienced as an interviewer. They have to be able to sort through the information presented by the student. They have to be willing to ask probing questions and be willing to make the student uncomfortable. Questions about ethical grounding or hypotheticals about decision-making are difficult. Interviewers can be blinded by other characteristics. Like MCAT scores．

For instance, I have heard interviewers say, well they didn't interview very well but they have great MCATs, so they will do fine. I am not kidding, I really heard a faculty member say that. And they were being serious! I know that is not supported by the data, but you still hear it a lot. The interviewer has to be experienced enough to ask tough questions. Not just, "tell me about your fraternity activities in college" but hard questions about ethics, hypotheticals, and dilemmas. They have to ask about motivation, why do they want to come to medical school. They have to get beyond the pat and prepared responses that students practice during their mock interviews and really push the student to get at internal motivations and thought processes.

And what about medical students? In my experience the medical students that we ask to interview are pretty good at sniffing out the bull. But their problem is that they feel so happy to actually be in medical school and almost finished (our interviewers are fourth year students) that they tend to be a little easy on the score sheets. Gutowski and colleagues,(3) looked at current medical student interviewers. They found that when compared to faculty interviewers, students wrote more about applicants' motivation, personality, communication skills, and interests. Student wrote more in the overall evaluation sections (p<0.001) and gave more examples on the motivation section (p<0.0011) and communication skills section (p<0.0035).

So, I guess the bottom line is that there is no easy way to figure out who are the right students to admit to medical school. We should push for multi-dimensional models that minimize the MCAT and utilize personality characteristics. We should ask the admissions committee (and the dean) to define what they think are the qualities and characteristics of the students that should matriculate to our medical school. And maybe most importantly the admissions committee should be held responsible for the results of their work.

References
(1) Powis DA, Neame FLB, Bristow T, Murphy LB. The Objective Structured Interview for Medical Student Selection. BMJ. 1988;296:765-768.
(2) Albanese M, et al. Assessing Personal Qualities in Medical School Admissions. Acad Med 2003;78:313–321.
(3) Gutowski CJ, et al. Current medical student interviewers add data to the evaluation of medical school applicants. Medical Education Online 2010;15:5245.

Posted using BlogPress from my iPad

Location:Common St,New Orleans,United States

Personality traits that predict success in medical school

This is the third in a series of posts about my concern with the students that we are bringing into medical school. I am constantly reminded that not everyone believes that there is a problem. Some think that we are getting the right students into medical school. My question. The burning question for this series of posts is: if we are getting the right students in, why is the end product not meeting the needs of America? Or the needs of our individual states? There was a recent op-ed in the New England Journal of Medicine by Stephen R. Smith, MD, MPH from the Warren Alpert Medical School at Brown University.(1) Dr Smith said "medical schools must recognize the current factors that discourage medical students from pursuing primary care careers and then devise ways to overcome these barriers."

My first post in this series, Why do we put so much import on the MCAT? was about the MCAT and why (in my opinion) we need to de-emphasize the MCAT as a criterion for entry into medical school. I am not the only one saying this. Dr. Smith said, "The first test of this commitment will come in the way in which admissions are handled. The little evidence that is available on factors predicting career choice indicates that students who express a desire to serve underserved populations, who demonstrate altruism, and who are committed to social responsibility are more likely to go into primary care." Dr. Smith went on to say that "admissions criteria need to be broadened beyond scores on the Medical College Admission Test (MCAT) to include these personal attributes. The school should adopt an “MCAT-blind” admissions policy, dictating that students whose MCAT scores are at or above a predefined minimum that predicts a likelihood of success in medical school should then be considered further for admission without the reporting of their MCAT scores to the admissions committee."

In my last post, Characteristics of Future Physicians, I blogged about characteristics that I believe that we should want in graduates of our medical schools. There are several factors that medical school faculty have rated as extremely important but very hard to teach. My argument is that we should select students who already have those traits prior to matriculation. Some of the traits listed are obvious: (1) is emotionally stable; (2) is a person of unquestionable integrity; (5) is unusually intelligent; and (6) has sustained genuine concern for patients during their illness.  Some are less obvious but seem really important: (9) is motivated primarily by idealism, compassion, and service; (14) is able and willing to learn from others; (17) is observant; and (18) is adaptable.

So the question for today is: are there personalities that do better in a medical curriculum and those that do worse? If you read the article by Dr Sade (2) I am sure that it is no surprise that there are some personalities that do better in medical school and some that seem to struggle more. Medical school is an extremely stressful environment. Many of the students that come to medical school, have never struggled academically in their lives. I have heard folks say, medical school is a pressure cooker.

The pressure cooker brings out all of the problems. It stresses them. It pushes them in ways that many of these students have never been pushed. 28 percent of physicians report that they have stress that affects their ability to provide clinical care.(3) A recent systematic review found that medical students perceive similar levels of stress to physicians.(4)

I see students struggle because they have never been in this kind of situation. Students are young, they are often coming straight from undergraduate school. Students that come to medical school have often not had any real life experiences. They have not had a job, or had extensive life experiences. Then we put them into an academically stressful environment that is extremely competitive and at some schools even cutthroat. The good news is that some personalities may be better at handling the stress of medical school, and then potentially life as a physician. McManus (5) studied stress in UK students over a period of 12 years. The authors found that stress could exacerbated or even caused by personality factors, specifically by high levels of neuroticism, low levels of extraversion and low levels of conscientiousness. These traits are dimensions that are part of the Five Factor model of personality. This model is well accepted and validated. The model is used by many of the studies of personality contributing to success in academic settings.

Another study from the UK (6) looked at Goldberg's "Big Five dimensions of personality" to study 176 medical students at the Nottingham Medical School. The authors compared the students' personality scores and academic assessments in Years 1 through 5. The conscientiousness dimension was a significant predictor of academic performance in the pre-clinical years but interestingly in the clinical years (Years 4 and 5) it correlated with poorer performance.

So, there are personalities that may be more suited for medical school. And even more importantly, these personality traits can affect success as physicians. The Aussies looked at this a few years ago. In this study, Knight, et al (8) studied whether personality characteristics, measured by the Hogan Developmental Survey (HDS), were associated with academic performance in 139 medical students. They found that borderline/schizoid and narcissistic/antisocial characteristics were negatively correlated with academic success. That one seems pretty obvious. One of the subsections of the HDS, the ‘Diligent syndrome’, were found to be positively correlated to higher exam scores. The Diligent syndrome is students who have a tendency to be attentive and good with details, orderly, rational, careful and well organized.

What about here in the US? The folks at Jefferson Medical College are the only ones who have studied this. Hojat, et al(8) studied six personality measures. They also asked about the students' relationships with their parents and their general health. They compared these assessments with global faculty ratings of competence in the clinical clerkships (family medicine, internal medicine, obstetrics and gynecology, pediatrics, psychiatry and surgery). The ratings used a 4-point scale (‘high honours’, ‘excellent’, ‘good’, ‘marginal competence’). The students in the lowest group had significantly lower levels of self-esteem and sociability, they were lonelier, and had less satisfactory relationships with their parents.

So, what should we do? Conscientiousness seems to be an important predictor of success in the preclinical years, but it may also lead to vulnerability to stress. And it is related to worse performance in the clinical years. Extraversion, self-esteem, and sociability seem to be more important in the clinical years. Maybe we should do like the Australians did and have every applicant fill out a personality profile. I am not sure how this was used, but it does not seem that it was used to choose students. Next time, I will write about some of the ways to screen incoming students.

References
(1) Smith S. A Recipe for Medical Schools to Produce Primary Care Physicians. New Eng J Med 2010; 364(6).
(2) Sade M, et al. Criteria for selection of future physicians. Ann Surg. 1985 February; 201(2): 225–230.
(3) Firth-Cozens J. Doctors, their well-being, and their stress. BMJ 2003;326:670–1.
(4) Dyrbye LN, Thomas MR, Shanafelt TD. Systematic review of depression, anxiety, and other indicators of psychological distress among US and Canadian medical students. Acad Med 2006;81 (4):354–73.
(5) McManus IC, et al. Stress, burnout and doctors’ attitudes to work are determined by personality and learning style: a 12-year longitudinal study of UK medical graduates. BMC Med 2004;2:29.
(6) Ferguson E, et al. Pilot study of the roles of personality, references, and personal statements in relation to performance over the five years of a medical degree. BMJ 2003;326 (7386):429–32.
(7) Knights JA, Kennedy BJ. Medical school selection: impact of dysfunctional tendencies on academic performance. Med Educ 2007;41 (4):362–8.
(8) Hojat M, Callahan CA, Gonnella JS. Students’ personality and ratings of clinical competence in medical school clerkships: a longitudinal study. Psychol Health Med 2004;9 (2):247–52.

The characteristics of future physicians

In my blog from 2/4/11 "Why do we put so much import on the MCAT?", I discussed some of the negative characteristics that can be associated with a higher MCAT. The conclusion of the article by Dr. Gough¹ was that students with higher MCATS and a scientific orientation were found to "less adept in interpersonal skills, less articulate, narrower in interests, and less adaptable than their fellows". 

Wow!  I don't think those are characteristics that I want in my doctor.  What about you?

What are the characteristics that we want in our medical students? We want them to be great at science, right?  On average, academic performance in undergraduate classes only predicted about 9% of the variance in medical school performance.² What about MCAT? We want them to have a high MCAT, right? Well, a high MCAT is good at predicting performance on the USMLE step 1 and preclinical grades,³ but as someone who is really smart once told me "we are not trying to make step 1 passers".

Sade and colleagues asked this same question a few years ago. Their specific question was to identify the specific characteristics that are important qualities of a superior physician. They also asked which of these qualities are hardest to teach in the medical curriculum. They based their work on a study by Price, et al⁴ who had previously generated a list of positive traits associated with a superior physician.  

Dr. Sade took this list of traits and showed them to the faculty of the College of Medicine at the University of South Carolina. The faculty were asked to rate the personal qualities on a scale of 1-10, where 1 is non-teachable and 10 was easily teachable. The survey was sent to all of the faculty at the college of medicine. They also asked a select group of experienced medical educators to take the survey. There was remarkable agreement between the faculty, greater than 80% inter-rater reliability. There was also a high correlation between the basic science faculty's ratings and the clinical faculty's ratings of the importance of characteristics (r=0.87, p<0.001) and the teachability of characteristics (r=0.93, p<0.001). 

The outcome of this survey was a list that ranked the characteristics from 1 to 87.  Each characteristic was given a rank for importance and for difficulty in teaching. The authors converted the rankings to a Z-score. (***Note: This was my favorite line in the manuscript...)  "The teachability Z-score was subtracted from the importance Z-score, and the combined Z-scores were multiplied by 10 and added to 50."  This gave a combined score that they called the NonTeachable-Importance Index (NTII). The NTII gives you a list of characteristics that are ranked from highest to lowest based on importance and the difficulty of teaching it to medical students.

That sounds like a good list of pre-matriculant variables to me.  If we can't easily teach it but it is important then obviously we should select students that have these characteristics before coming to medical school.

Using the NTII ranking, some of the characteristics are obvious: (1) is emotionally stable; (2) is a person of unquestionable integrity; (5) is unusually intelligent; and (6) has sustained genuine concern for patients during their illness.  Some are less obvious but seem really important: (9) is motivated primarily by idealism, compassion, and service; (14) is able and willing to learn from others; (17) is observant; and (18) is adaptable. The list goes on from 1 to 87. 

The characteristic that was ranked as the most difficult to teach was: is unusually intelligent. The next four were: (2) is naturally energetic and enthusiastic; (3) is imaginative and creative; (4) has a warm, friendly, outgoing personality; and (5) is motivated by sheer liking of people.

So why are we still choosing medical students based on the MCATs and GPAs? Maybe, we should be looking at these factors.

Next time, I am going to write about personality factors that influence medical student performance.

References

(1) Gough HG. Some Predictive Implications of Premedical Scientific Competence and Preferences. J Med Educ  1978; 53: 291-300.

(2) Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ  2002; 324: 952–7.

(3) Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med  2007; 82(1): 100-6.

(4) Price PB, et al. Measurement and predictors of physician performance: two decades of intermittently sustained research. Salt Lake City: Aaron Press, 1971; 121-149.

Why do we put so much import on the MCAT?

Every year thousands of potential medical students spend a lot of time and money to study for the MCAT. Students spend a lot of money, for example,  $1,749 to take the Kaplan Complete MCAT Preparation and $1,000 for the Princeton Review. Why?? They spend this money preparing themselves to sit for this life changing examination. The test itself is actually comparatively inexpensive-- $235 dollars. 

The worst part is that the MCAT is probably not measuring any of the most important variables for our matriculating medical students. I have been on the admissions committee at two different medical schools.  Both were State supported medical schools with a strong commitment to graduating students interested in primary care, rural practice, and practice in underserved areas. But at the Admissions Committee level there is a serious lack of understanding of the importance of pre-matriculant noncognitive data and the variables used to select students to enter the school. Many of my colleagues (physicians and basic scientists) over the years have held the belief to some extent or another that higher pre-admission scores lead to better medical students which leads to better doctors.

Nothing could be further from the truth. In fact, for most of the variables there is very little correlation. And even more worrisome for many of the important characteristics of being a physician, there is an inverse relationship.  At this point, many of you are thinking, "Delzell is completely off his rocker!". In fact, one of my basic science colleagues said as much last year.  Well, that may be true, but I do have evidence to back up these statements.

Let's go back. Way back, to 1978. Harrison Gough, PhD, a psychologist at the university of California-Berkley, published one of the most fascinating studies (1) that I have seen in the medical educational literature. It is also one of the best written papers that I have ever read, and as an editor for a major medical journal I get to read a lot of manuscripts. Dr. Gough collected data on medical students from the University of California-San Francisco from 1951 to 1977. Wow! That is like the Framingham study of medical students. A longitudinal study of a medical school and its medical students.  This study reports on 1,195 UCSF students from 1972 to 1977. Data collected included MCAT scores, undergraduate GPA, and a measure to assess scientific preferences.  He created a composite index score by adding the "measures of scientific talent" (MCAT Science subtest and Undergraduate Science GPA) to Science Preference.

Science Preference is a fascinating concept that was developed by Goldstein (2) and modified by Dr. Gough. Students rated the three subjects from college that they liked best and the three that they liked least. The average score for the three least liked subjects was subtracted from the average score for the three subjects liked best. This gives an overall score, which was termed the students' preferences for science.  

The next step was to measure correlations between these measures and performance during medical school. He looked at grades in all four years of school. As you would expect, there was a significant correlation between the composite index and GPA in year 1 (Pearson Product- Moment correlation .34; p < 0.01) and year 2 (Pearson Product- Moment correlation .21; p < 0.01). But, there was no significant correlation with GPA in years 3 and 4. More importantly medical school faculty ratings of clinical competence and general competence were not significantly correlated (Pearson Product- Moment correlation .01; p = ns).

Ok, so maybe MCAT and GPA are not great at predicting things after the first two years but those years are important. Right???  Someone very smart once told me, "they would be really important if we were trying to make Step 1 passers, but we are not trying to make Step 1 passers, we are trying to make doctors."

Fortunately, Dr. Gough didn't stop there, there was another part to his study. He selected 70 students for an intensive study of personality at the UC- Berkley Institute of Personality Assessment and Research. The students were evaluated by 10-15 trained assessment staff members. Students were observed closely for an entire day and the staff members described them using a 300 item Adjective check list. These descriptors were then correlated with the four previously evaluated science predictors for each student.  

There was no statistical correlation between MCAT scores and personality descriptors. But, Science GPA was correlated with "painstaking"(r =.26) and "silent"( .26). There were negative correlations with adjectives such as "poised" (r = - .32), "self-controlled" (- .30), and "interests wide" (- .27).  When compared to the composite index there were also several significant correlations, such as "awkward" (r = .27), "cautious" ( .23), and "conservative" ( .27).  The composite index was negatively related to several descriptors, such as "stable" (r = - .28) and "relaxed" (- .28).  

Each of the students was also judged by all of the observers. The reliability of their judgment is striking. The inter-rater reliability was greater than .80.  The students with higher composite index were rated lower in their ability to communicate (r = - .28), breadth of interests (r= - .35), self-acceptance (r = - .26), and verbal fluency (r= -.29). All of these correlations were significant. By the way, the MCAT by itself also had a negative correlation with every measure, with r values between - .11 and - .20.  Dr Gough's conclusion was that scientifically oriented students were "less adept in interpersonal skills, less articulate, narrower in interests, and less adaptable than their fellows.". 

Wow! That is amazing. Why didn't anybody tell about this when I joined the admissions committee? For my next blog, I am going to write about some important characteristics of future physicians and how we can better select students that will have those characteristics.

References

(1) Gough HG. Some Predictive Implications of Premedical Scientific Competence and Preferences. J Med Educ  1978; 53: 291-300.

(2) Goldschmid ML.  Prediction of College Major by Personality Tests. J Counseling Psychol  1967; 14: 302-308.