The characteristics of future physicians

In my blog from 2/4/11 "Why do we put so much import on the MCAT?", I discussed some of the negative characteristics that can be associated with a higher MCAT. The conclusion of the article by Dr. Gough¹ was that students with higher MCATS and a scientific orientation were found to "less adept in interpersonal skills, less articulate, narrower in interests, and less adaptable than their fellows". 

Wow!  I don't think those are characteristics that I want in my doctor.  What about you?

What are the characteristics that we want in our medical students? We want them to be great at science, right?  On average, academic performance in undergraduate classes only predicted about 9% of the variance in medical school performance.² What about MCAT? We want them to have a high MCAT, right? Well, a high MCAT is good at predicting performance on the USMLE step 1 and preclinical grades,³ but as someone who is really smart once told me "we are not trying to make step 1 passers".

Sade and colleagues asked this same question a few years ago. Their specific question was to identify the specific characteristics that are important qualities of a superior physician. They also asked which of these qualities are hardest to teach in the medical curriculum. They based their work on a study by Price, et al⁴ who had previously generated a list of positive traits associated with a superior physician.  

Dr. Sade took this list of traits and showed them to the faculty of the College of Medicine at the University of South Carolina. The faculty were asked to rate the personal qualities on a scale of 1-10, where 1 is non-teachable and 10 was easily teachable. The survey was sent to all of the faculty at the college of medicine. They also asked a select group of experienced medical educators to take the survey. There was remarkable agreement between the faculty, greater than 80% inter-rater reliability. There was also a high correlation between the basic science faculty's ratings and the clinical faculty's ratings of the importance of characteristics (r=0.87, p<0.001) and the teachability of characteristics (r=0.93, p<0.001). 

The outcome of this survey was a list that ranked the characteristics from 1 to 87.  Each characteristic was given a rank for importance and for difficulty in teaching. The authors converted the rankings to a Z-score. (***Note: This was my favorite line in the manuscript...)  "The teachability Z-score was subtracted from the importance Z-score, and the combined Z-scores were multiplied by 10 and added to 50."  This gave a combined score that they called the NonTeachable-Importance Index (NTII). The NTII gives you a list of characteristics that are ranked from highest to lowest based on importance and the difficulty of teaching it to medical students.

That sounds like a good list of pre-matriculant variables to me.  If we can't easily teach it but it is important then obviously we should select students that have these characteristics before coming to medical school.

Using the NTII ranking, some of the characteristics are obvious: (1) is emotionally stable; (2) is a person of unquestionable integrity; (5) is unusually intelligent; and (6) has sustained genuine concern for patients during their illness.  Some are less obvious but seem really important: (9) is motivated primarily by idealism, compassion, and service; (14) is able and willing to learn from others; (17) is observant; and (18) is adaptable. The list goes on from 1 to 87. 

The characteristic that was ranked as the most difficult to teach was: is unusually intelligent. The next four were: (2) is naturally energetic and enthusiastic; (3) is imaginative and creative; (4) has a warm, friendly, outgoing personality; and (5) is motivated by sheer liking of people.

So why are we still choosing medical students based on the MCATs and GPAs? Maybe, we should be looking at these factors.

Next time, I am going to write about personality factors that influence medical student performance.

References

(1) Gough HG. Some Predictive Implications of Premedical Scientific Competence and Preferences. J Med Educ  1978; 53: 291-300.

(2) Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ  2002; 324: 952–7.

(3) Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med  2007; 82(1): 100-6.

(4) Price PB, et al. Measurement and predictors of physician performance: two decades of intermittently sustained research. Salt Lake City: Aaron Press, 1971; 121-149.

Why do we put so much import on the MCAT?

Every year thousands of potential medical students spend a lot of time and money to study for the MCAT. Students spend a lot of money, for example,  $1,749 to take the Kaplan Complete MCAT Preparation and $1,000 for the Princeton Review. Why?? They spend this money preparing themselves to sit for this life changing examination. The test itself is actually comparatively inexpensive-- $235 dollars. 

The worst part is that the MCAT is probably not measuring any of the most important variables for our matriculating medical students. I have been on the admissions committee at two different medical schools.  Both were State supported medical schools with a strong commitment to graduating students interested in primary care, rural practice, and practice in underserved areas. But at the Admissions Committee level there is a serious lack of understanding of the importance of pre-matriculant noncognitive data and the variables used to select students to enter the school. Many of my colleagues (physicians and basic scientists) over the years have held the belief to some extent or another that higher pre-admission scores lead to better medical students which leads to better doctors.

Nothing could be further from the truth. In fact, for most of the variables there is very little correlation. And even more worrisome for many of the important characteristics of being a physician, there is an inverse relationship.  At this point, many of you are thinking, "Delzell is completely off his rocker!". In fact, one of my basic science colleagues said as much last year.  Well, that may be true, but I do have evidence to back up these statements.

Let's go back. Way back, to 1978. Harrison Gough, PhD, a psychologist at the university of California-Berkley, published one of the most fascinating studies (1) that I have seen in the medical educational literature. It is also one of the best written papers that I have ever read, and as an editor for a major medical journal I get to read a lot of manuscripts. Dr. Gough collected data on medical students from the University of California-San Francisco from 1951 to 1977. Wow! That is like the Framingham study of medical students. A longitudinal study of a medical school and its medical students.  This study reports on 1,195 UCSF students from 1972 to 1977. Data collected included MCAT scores, undergraduate GPA, and a measure to assess scientific preferences.  He created a composite index score by adding the "measures of scientific talent" (MCAT Science subtest and Undergraduate Science GPA) to Science Preference.

Science Preference is a fascinating concept that was developed by Goldstein (2) and modified by Dr. Gough. Students rated the three subjects from college that they liked best and the three that they liked least. The average score for the three least liked subjects was subtracted from the average score for the three subjects liked best. This gives an overall score, which was termed the students' preferences for science.  

The next step was to measure correlations between these measures and performance during medical school. He looked at grades in all four years of school. As you would expect, there was a significant correlation between the composite index and GPA in year 1 (Pearson Product- Moment correlation .34; p < 0.01) and year 2 (Pearson Product- Moment correlation .21; p < 0.01). But, there was no significant correlation with GPA in years 3 and 4. More importantly medical school faculty ratings of clinical competence and general competence were not significantly correlated (Pearson Product- Moment correlation .01; p = ns).

Ok, so maybe MCAT and GPA are not great at predicting things after the first two years but those years are important. Right???  Someone very smart once told me, "they would be really important if we were trying to make Step 1 passers, but we are not trying to make Step 1 passers, we are trying to make doctors."

Fortunately, Dr. Gough didn't stop there, there was another part to his study. He selected 70 students for an intensive study of personality at the UC- Berkley Institute of Personality Assessment and Research. The students were evaluated by 10-15 trained assessment staff members. Students were observed closely for an entire day and the staff members described them using a 300 item Adjective check list. These descriptors were then correlated with the four previously evaluated science predictors for each student.  

There was no statistical correlation between MCAT scores and personality descriptors. But, Science GPA was correlated with "painstaking"(r =.26) and "silent"( .26). There were negative correlations with adjectives such as "poised" (r = - .32), "self-controlled" (- .30), and "interests wide" (- .27).  When compared to the composite index there were also several significant correlations, such as "awkward" (r = .27), "cautious" ( .23), and "conservative" ( .27).  The composite index was negatively related to several descriptors, such as "stable" (r = - .28) and "relaxed" (- .28).  

Each of the students was also judged by all of the observers. The reliability of their judgment is striking. The inter-rater reliability was greater than .80.  The students with higher composite index were rated lower in their ability to communicate (r = - .28), breadth of interests (r= - .35), self-acceptance (r = - .26), and verbal fluency (r= -.29). All of these correlations were significant. By the way, the MCAT by itself also had a negative correlation with every measure, with r values between - .11 and - .20.  Dr Gough's conclusion was that scientifically oriented students were "less adept in interpersonal skills, less articulate, narrower in interests, and less adaptable than their fellows.". 

Wow! That is amazing. Why didn't anybody tell about this when I joined the admissions committee? For my next blog, I am going to write about some important characteristics of future physicians and how we can better select students that will have those characteristics.

References

(1) Gough HG. Some Predictive Implications of Premedical Scientific Competence and Preferences. J Med Educ  1978; 53: 291-300.

(2) Goldschmid ML.  Prediction of College Major by Personality Tests. J Counseling Psychol  1967; 14: 302-308.

Dr Jerry Kruse's Seussian rhyme

The following Seussian rhyme was imagined and written by Dr Jerry Kruse, Professor and Chair, Department of Family & Community Medicine, Southern Illinois University School of Medicine.
It was given during the final plenary session at the 2011 STFM Conference on Medical Student Education.

The Saga of Michael Klein

Or…..

 Ein Kleiner Schnitt

Or…..

The Triumph of Reason Over Power, Finally!

By Dr. Kreuss*

In that faraway land to the North, in Quebec,

Lived a doctor whose practice was very low tech.

A family doc, accoucher Michael Klein,

Who didn’t like forceps or women supine

Or ‘lectronic monitors, stirrups or sections

Or enemas, shaving or IV injections.

He hated electrodes and IUPC’s

And treatment of labor as if a disease.

And one of the worst – epidural blockade –

A stab in the back to start the cascade

Of catheters, tubing, Nubain and pit

And Sulfate of Mag so she won’t have a fit.

Blood pressure cuffs and punctures of veins,

Cesarean Sections and Tucker-McLains

Retained placenta, post partum metritis

I’ll bet you a buck she’ll come down with mastitis.

“I don’t like these women to all be strapped down.

Stand up and walk!”  he cried with a frown.

Michael knew in his heart, way deep down inside

That obstetrical knowledge was not well applied. 

“Technology’s great, for those who are ill,

But for those who are healthy it’s really no thrill

To be strapped down and poked, and scared stiff as a board.

This just isn’t right!”, his fervent voice roared.

One thing more than others, did gnaw at his heart,

Made his blood boil, and stung like a dart

He just couldn’t stand it, to see a long slice,

A cut, an epis - what a terrible vice,

Disruption of skin for no reason at all,

A snip with the scissors that starts very small

But rips and extends as the baby comes through

Tears into the sphincter and up the wazoo.

A third, then a fourth, oh my what a mess

“They must like to sew, is my only guess.”

So Michael jumped up, and he raised his right hand

And opened his mouth, and he took a firm stand

“I’ll study this problem,” he said with a shout

“And when I am finished there won’t be a doubt

That these cuts are no good…the whole world will see….

This idea’s a good one, they’ll have to agree.

I’ll start up a randomized, single blind study

And I’ll work with Michel who’s my very good buddy

And we’ll put ole’ McGill right here on the map.

This study of perineal trauma’s a snap.

“We’ll put in a grant, we’ll get recognition.

We send all this stuff to a good statistician.

Our alpha will be less than point zero five

And beta point two will let us derive

The number of women we’ll need.  It’s a slew.

We’ll enroll ‘bout a thousand five hundred and two.”

He worked and he toiled, he felt quite convicted—

The results were exactly the ones he predicted.

“Midline epis, when routinely done

For women in labor, is not very fun.

Our EMG’s show that sliced muscles get weak,

They heal up quite slowly and let urine leak.

And that isn’t all that comes out that should not.

She’ll find our real quick when she sits on the pot.

“The relative risk for a fourth degree rip

Is greater than twenty, with each little snip.

And all of us know, if the rectum is torn

That a permanent hole will often be born

That connects the vagina and rectal mucosa

And where it comes out then will make you nervosa.

So my warning to you, who practice OB

Is to use the epis quite conservatively.

Don’t be in a rush, and don’t interfere

Throw down the scissors, they’re not needed here.”

Now Michael was proud, and really excited

With his results he was very delighted.

He wanted to spread the good news he had found

He wanted to broadcast this stuff all around,

But a funny thing happened.  He couldn’t believe it

No one would listen, they just couldn’t perceive it.

The doctors in charge of the medical journal

The ones who are experts in issues maternal

Did not want to hear about data that’s new.

It was hard to convince that conservative crew.

“No one will believe it,” the editors cried

And they wrote down “REJECTED” with feelings of pride

And one after another they all did the same.

They suppressed this great knowledge – oh my what a shame!

But this story’s not over, he didn’t back down

He battled the towers who sport cap and gown.

Then once, then again and a third time, it’s true

His papers were granted another review.

And though eight years late and quite overdue

All three ended up in the publishing queue.

And now the world knew, both up in the North

And in states to the South, the new message went forth.

The paradigm shifted, the good word was this:

“The epis is archaic, and you’ll be remiss

To ignore this great knowledge that newly exists

And to squander this chance to cause perineal bliss!”

The rēsearch of Michael had ended the reign

Of procedure and practice we all thought mundane,

Of cuts and incisions most surely inane

And the scissors were thrown to the floor with disdain.

In just over a decade, the rate of epis,

Of pelvic dysfunction and fourth gaping degrees

Had tumbled to levels that went far below

The figures observed just a few years ago.

From sixty percent of all getting cleaved

To just nine in a hundred a cut to receive.

“Never give up!”  Michael’s voice still rings clear.

The moral today is to be of good cheer

To persistently fight, in the face of all the odds.

To battle ideas of conventional gods

To fight for new facts, and new evidence find

To give power to reason and sight to the blind.  (THE END).

*Dr. Kreuss                 Jerry Kruse, MD, MSPH

                                    Professor & Chair

                                    Department of Family & Community Medicine

                                    Southern Illinois University School of Medicine

                                    Springfield, Quincy, Carbondale and Decatur, Illinois

Bibliography:

1.      Klein MC, Gauthier RC, Jorgensen SH, Robbins JM, Kaczorowski J, Johnson B, et al.  Does episiotomy prevent perineal trauma and pelvic floor relaxation?  Online J Curr Clin Trials, Doc 10, July 1, 1992.

2.      KIein MC, Gauthier RJ, Robbins JM, Kaczorowski J, Jorgensen SH, Franco ED, et al.  Relationship of episiotomy to perineal trauma and morbidity, sexual dysfunction, and pelvic floor relaxation.  Am J Obstet Gynecol 1994; 171:591-8.

3.      Klein MC, Kaczorowski J, Robbins JM, Gauthier RJ, Jorgensen SH, Joshi AK.  Physician beliefs and behavior within a randomized controlled trial of episiotomy; consequences for women under their care.  Can Med Assoc J 1995; 153:769-79

4.      Huston P.  The pursuit of objectivity [editorial].  Can Med Assoc J 1995; 153:735.

5.      Schultz KF.  Unbiased research and the human spirit: the challenges of randomized controlled trials [editorial].  Can Med Assoc J 1995; 153:783-6.

6.      Klein MC.  Studying episiotomy:  When beliefs conflict with science.  J Fam Pract 1995; 41:483-8.

7.      Frankman EA, Wang L, Bunker CH, Lowder JL.  Episiotomy in the United States:  has anything changed?.  Am J Obstet Gynecol 2009; 200: 573.e1-573.e7

8.      Hyer R.  ACOG 2009:  Steep decline in episiotomy rates credited to research, peer pressure.  ACOG 57^th Annual Clinical Meeting, Medscape Medical News.  http://www.medscape.com/viewarticle/702541

The rhyme contained in this blog is the intellectual property of Dr. Kruse and cannot be copied without his express consent.  Thanks. JED