Inaugural medical education lecture honors the late Richard C. Vari, PhD

The Virginia Tech Carilion School of Medicine hosted the inaugural Richard C. Vari, PhD Endowed Lecture in October 2022. The event honored the school’s founding curriculum dean who became the senior dean for academic affairs. The beloved educator passed away from complications of ALS earlier this year.

The annual lecture features an internationally renowned innovator in medical education annually as part of TEACH Education Day. This year, Louis Pangaro, a professor of medicine at the Uniformed Services University School of Medicine in Bethesda, Md., presented “What does Physiology Teach us about Assessment and Evaluation?”

Skip to content

• Pangaro Lecture
• Questions and Answers

Introductions

Louis. M. Pangaro, MD, MACP
What does physiology teach us about assessment and evaluation?

Video Transcript

>> Thank you. Thank you very much for inviting me to give the inaugural Vari lecture. I met Dr. Vari, when I was active with IAMSE as part of the Alliance for Clinical Education about 12 years ago. And hearing what his friends and family have said about him and what I've read about him, I think we are gifted to be in the presence of someone whose effect will just have ripple waves through generations. For people who never met him, his family, or never knew anything about Hungarian meatballs. That I think we're all privileged to be part of that. 

I'm what passes as an expert for medical education. But when I take care of patients with a thyroid problem, I have a molecular understanding of how thyroid hormone interacts with the nucleus. When I take care of patients with diabetes mellitus, I can look at studies with thousands of patients randomized to different courses of treatment. This expertise is not available yet to you and me in medical education. We need to be very humble about this. And I will speak with great dramatic force today about my ideas, but I do not confuse the vehemence with which I speak with any kind of certainty. And I will try to pay tribute to the work that Dr. Vari and Carilion has done and Virginia Tech because this is an extraordinary school. Some of the things that has happened here probably could not happen in a school that's been around for 100 years, the innovation and the ability to create a new way of looking at the process of education. I will try to pay tribute to that, and at the same time try to extend some of those ideas into areas that may be a little bit controversial. I'm always glad if I can say something that is a little outside of the box. 

I'll speak as clearly as I can. I tend to have everything I say in the slides. I often talk to international audiences where English is not a first language. Then I'll always make my slides available so people don't have to take. So I apologize for some very old fashioned style of PowerPoint technology. And if I run out of time, we'll just jump to the end. So down, I think that's the title. 

There's something I want to emphasize in this tribute to Rick. This emphasis on adult learning in a patient-centered context. I will pick up two of those themes is, what does it mean to learn as an adult? What does it mean for a resident or a medical student to say, here's what I need to know. And then what is the idea of the patient-centered context have to do with a system in which they'll function.

In a sense, I'm after a unified theory of medical practice, competence, readiness, and evaluation in the clinical setting, and my theme is always to think of everything but not do too much. To embrace complexity and act with simplicity, I think is what the medical profession needs to be paid to do, need to focus on. It's kind of a judgment and expertise that is not simply a guidelines based or routinized. So that will be the underlying theme. 

I'd like to start off by saying, how do we define success for the curriculum for you and me as teachers? So I want you to imagine I am working with a beginning intern from my own medical school who sees a patient with thyrotoxicosis and overactive thyroid gland and we're about to put that patient on medicine. 

Simple Program Evaluation

I would be happy if this, my medical school graduate, could describe iodine metabolism and how methimazole or tell it interfered with them, I would be happy about that. But if this resident not knowing these basic mechanisms did not on their own without my coaching, fill the gap in their knowledge, then I would drop dead with embarrassment. This is my simple way of evaluation of saying what's the difference between what's essential and what's desirable? 

The job of faculty is to create independence, the capability to fill in the gap

And faculty, you and me, our job, the curriculum, is to create independence capability so that the graduate can fill this gap on their own. They have to have internalized some concept of what expertise looks like. And I'll be talking about that in this talk. And what is the role of faculty in fostering that. But basically at the end, the student, our graduate, has to be able to independently learn from experience without me being there to prod them. And if my graduate has a problem in physiology, how does methimazole interfere with the synthesis of thyroid hormone? And they don't say, God, I don't know that, I got to figure that out. And they don't have that itch. Then I have to drop dead with embarrassment. I failed. 

So this is a distinction between the essential and the desirable.

And the part of the importance of this to me is so much of what I have to do in my daily work with patients or students is desirable but not essential. And I worry about economy of action and all the things I, as chair of medicine, I had 1,000 faculty between Bethesda and Honolulu. All the things they are asked to do and the central role of faculty in creating this capability and respect for what their needs are is very important to my own thinking.

This is the disclaimer. Everything I say here does not represent the Department of Defense or the Uniform Services University. You heard me say this. And there's a couple of disclaimers. We don't need this. 

I need to tell you a little bit about some of the words that I'm going to use when I say learner or student. Sometimes I'll say student, but it could be a resident or it could be a fellow. Sometimes it'll just be a student.

And by readiness, I'm going to use the word readiness rather than competency. We use the word competency in very loose ways. So readiness, for a pre-clerkship student to go into the clerkships ready for a fourth-year students become an intern, ready for a resident for independent. That's what I say. What some people mean is competence. I'm capable of being independent.

I'm going to use the word assessment in it's etymologic sense sitting next to observe the student or the resident.

And I'm going to use evaluation to mean an interpretation of what I see.

I'm saying this because other communities use these words differently. So for instance, if I had a student who was a good reporter, they could get the facts, but not much of an interpreter, they couldn't explain why the patient had a had a fever. So they had basically the syndrome was failure to go from reporter to interpreter. That's what my observation is. That's the assessment. If this was a first-year medical student, that would be fine. If it was a PGY2 in medicine, that would not be fine. So the observation is the same. The assessment is the same. But the evaluation, the meaning is different depending on context.

I wanted to explain how I'm using the words because not everybody uses the same word. Overall, everything that we do from the beginning of med school to the end of GME is a movement from understanding into action. And this will come repeatedly in my thinking today because you'll hear a lot of repetition because my theme is the commonality of everything we do. Taking care of patients, taking care of students, taking care of each other.

So this movement from understanding into action.

So understanding how the student is doing is assessing and evaluating. And then we have actions seen on the lower right-hand corner here. Grading, in my lexicon, is not an educational activity. A few deans can scream if they wish. It's an administrative action, a classification. As you get credit, you are ready for advancement. This is a summative decision as opposed to feedback, which is an educational activity which any teacher can do based on their own observations. They've made an observation student is a reporter not moving interpreter on this patient today, placing it in context, this is okay for a beginning first-year student or a second-year student, I'll give you feedback based on that. You can give feedback based on information. But to make a summative decision, you get to graduate. This requires a much higher degree of certainty or knowledge. 

Question 1: What is a physician? 
What’s the role of science in practice?

So what is the end? What is a physician? And what is the role of science and practice? And what I'm going to use this etymology here, physics, physiology, physician, these three words derived from the Greek word for nature. 

So we claim to understand human nature whether at the bottom of this triangle, at the level of molecules in genes, at the middle of this triangle, at the level of the patient or in terms of the whole system. So we can say this is basic science, clinical science, or recently we are calling system science. But at each of these three levels, a physician makes a commitment to understand mechanisms. How things work. That gives us as part of our authority to violate practice guidelines. A knowledge of mechanism plus a sense of duty to the patient. So this commitment to mechanism and the first tribute to Rick, and he wrote papers on endocrine physiology and teaching about 20 years ago. I don't have time to go into all of them. But the idea is you understand how that drug works in this patient. So you're committed to mechanism, whether it is at the level of the gene, the patient, or the whole system. In our own curriculum, we called it molecules to military medicine, because we expect our students to be able to embrace all of those domains. These is the three domains as areas within knowledge which you must understand. So again, the emphasis, physician physiology, physics, it's a commitment to understanding nature. 

Do physicians use basic science?

This was controversial in the 90s because when you talk to doctors about how they decided it was pneumonia, what did they say? It looked like pneumonia. How did you decide it was nephrotic syndrome? It looked like nephrotic syndrome. So there was some thought that doctors don't do this. Then it was a nice series of articles by Patel and Hank Schmidt and others. I give you one reference there, is that the science is there below the surface, that he used the term encapsulation. That you understand the physiology, it's all encapsulated in a pathophysiologic mechanism.

So when we use the concept of fast thinking, this is Kahneman, the Nobel Prize winner. He was a psychologist but he won it for economics. Fast thinking is pattern recognition. The patient has polyuria-polydipsia, so it must be diabetes mellitus. So that's fast thinking, that's recognition. But what a physician needs is x-ray vision that you can visualize the glomerulus and a nephron, and you have this idea of an osmotic diuresis, that you can do the slow thinking to figure things out. This to me is the CNA QUONON a physician and physiology and if the faculty allows the student to graduate without a commitment to understanding the mechanism, whether it's at the level of the gene, the patient or the system, then we have to drop dead with embarrassment. 

So what is the role of basic science in practice?

Essentially we make a promise of duty and expertise and our commitment to understanding whether it's at the level at the bottom. I'm using the triangle as a metaphor, that that may be the wrong metaphor. It could be upside down, it could be sideways up, allow that triangular. I like the point, tiny little molecules or genes. And of course, these people won the Nobel Prize for their work on CRISPR and what's going on with a gene. That's at the bottom of the triangle. 

And then at the top we have this revolution of health systems science. Jed Gonzalo, who's here now did a presentation at IAMSE. Rick was the president of IAMSE and some of the papers I'll be citing, like Rowe here and others. Rick is on all of these papers I will be citing, but often he's the senior author, so he's not the first person on the list so you'll see other names. And health systems science is at, if you will, the top it's the most broad concept and is spoken of as one of the three pillars. 

So building health systems science education as a core domain, was one of the fundamental things was happening here at Virginia Tech, Carilion.

And this is a paper by Dr. Musick, that cited this, Rick is one of the authors, co-authors in this paper. I think Patty was a co-author on this paper too, if I recall, Dean Learman, I think you were on that one too. So building in systems thinking, and you've taken steps that other schools, I wish we had, working with Radford to build inter-professional education. And when we speak of things as mapping, we're seeing things as domains. Mapping is how the faculty organize the structure. But there's another question about integration, not what the faculty do, but how the students integrate it into their head. 

And suppose, instead of mapping with different domains, and I've shown you my triangle again. A lot of repetition because I think it's an important concept. Mapping is looking at the domains basic, clinical and science. 

Now suppose health systems science, which we're now introducing, were used as a lever to get students to think that everything is a system. Basic science and biochemistry. Remember gluconeogenesis, is part of a system. The body is trying to maintain the glucose because the brain needs glucose. Suppose the same terminology of systems was used not just at the biggest macro level, but actually at the micro level so that when the students learn biochemistry, it's the same jargon terminology as if it were health systems science. 

So what are these sciences of basic and health system have in common?
What is a system?

So here's a definition from Rechtin famous book on why eagles can't swim. A collection of different things which working together produce a result not achievable by things alone.

In other words, the system is trying to maximize one thing, to avoid something else and to protect something. When we're taking care of patients, we call this efficacy and safety of course. So it's true at the clinical level in the middle of the triangle. So what is the system trying to maximize? 

Physiology as homeostatis

This is physiology. It's Claude Bernard there, as homeostasis, the system is trying to maximize X. The system says, I would be happy if and the system is trying to avoid Y. Hypoglycemia, what happens if you get hypoglycemia? Your blood sugar is 20, you seize, you’re in the primal jungle, you get eaten by a saber-tooth tiger. So I will drop dead, not just with embarrassment, but literally. You see how the prioritization I would be happy, and I'm using the physiologic concept of homeostasis to support this, the maintenance of the internal milieu. There's a set point. The body has a set point for sugar, osmolality, blood pressure. The patient has a set point for what they're trying to achieve and so does the system. So the set point implies there's a constant maintenance. There's a constant dialogue in the system about what we're trying to maximize and what we're trying to minimize. 

So we're trying to maintain the internal milieu. We're trying to maintain cerebral glucose and avoid hypoglycemia because in the primal jungle, if your sugar gets too low, you get eaten. So the very notion of gluconeogenesis and homeostasis, which we teach in physiology is the same principle at the health system science level. What is this system trying to maximize? Health of populations, the experience of patients, and minimizing costs. That remember, we have a quadruple aim now, but that was the triple aim. What are we trying to maximize? 

So at each level, does your student or your finishing resident… Have they internalized the concept of what has to be internalized, developed, or maximize in the student in order to be able to function. And expertise then, for a graduating student and resident, maybe the capability to say, what do I need to maximize for this patient, and what do I need to avoid. It's a systems thought. Whether it's at the level I need to avoid. I want to maximize control of their blood sugar. I don't want it greater than 200, but damn, I got to avoid hypoglycemia because that's really bad. That's at the level of the basic science. What do I need to maximize for the patient? What are the patient's values? And what does the patient wish to avoid that's at the center of my triangle. And then what is the system trying to maximize and avoid? And expertise, bottom of the slide, is the capability to ask and answer these questions and duty is the commitment to find the answer. 

And here I'm saying what are the priorities in this science, so basic science, clinical science in the middle of the patient and on the right, the care system. Basic science we're trying to avoid, we're maximizing glucose and osmolality. The patient's priorities in the center, maybe my pain. I want you to take away my pain, my job, the cost to me, and what is the health care system trying to maximize and minimize? And can we, as faculty, be sure that the student knows that each three-level, what is to be maximized and what is to be avoided. Systems thinking at all three levels. 

Professionalism is a promise of expertise and a promise of duty

Here I'm dependent on the emphasis, Dr. Pellegrino, who never actually wrote what is on the slide, but I heard him say it many times to our students, that professionalism is a promise. It's a promise of expertise and a promise of duty. And so for the next few slides, I'm going to be talking about what does a student need to know as an example of expertise, and how does a student relate to patients in making decisions as an example of duty. I think almost everything we say about competencies, milestones, EPAs [inaudible] is a footnote to Pellegrino. Professionalism is a promise of expertise and a promise of duty and competence is being able to deliver it. 

Question 2: what does expertise look like?

What does expertise look like? The faculty has to judge what has the student internalized, and these are the two problems. Cognitive and then when you move from understanding into action to take care of patient, what is that duty look like? 

And here, broadly speaking, the beginning of med school to the end of residency, you're moving from understanding into action. From what I call reporter/interpreter into manager/educator. And on the left, understanding making a diagnosis going from the findings, the symptoms, the labs and whatever, to a diagnosis, is primarily cognitive and requires a lot of expertise. But once we move and we say to the patient with a thyroid problem, for instance, you have an overactive thyroid; we could talk about surgery, radioactive iodine or medications, the patient's values are now part of that conversation. Once we're talking about values, this is no longer simply cognitive, this is ethical. So the movement from understanding into action, into this field of action of shared decision-making with the patient is also an ethical concern. 

So on the first take the cognitive part of this, what is understanding? What is knowledge? Well, again, the overall movement is from understanding into action. Do you see something? You think about it. You do something. Observation, reflection, action. This is the rhythm that underlies all human activity.

PDSA cycles, all business, IPO, All of that stuff is basically another way of saying you see something, you think about it, and then you do something. So let's imagine you're a faculty member in the hospital when you have a resident, you observe that at the bottom of the slide here, you observe the resident and then you as a teacher give feedback. You have taken some information, you've decided what it means. And then if you see many observations over time, multiple teachers, then the competency committee decides whether or not this PGY1 becomes a PGY2. That decision that a PGY1 becomes a PGY2 is a movement from understanding into action.

Because advancement, a summative grade and advancement decision is an ethical decision and action to be taken by the committee.

Now I'm going to parse this a little bit more. Stay with my attempted graphics here. So we have a movement from data to information. I made an observation and decided that the resident was okay. To certainty, I've now not just had one observation of the resident  today with diabetes, but we've had five teachers observing the resident in 20 cases of different kinds, of heart failure, pneumonia, clots, renal problems, infections, whatever is appropriate in your discipline. So I now I'm moving from information to knowledge to certainty because of sampling, and now I'm ready to make a decision which requires wisdom.

So where does knowledge fit in? To go from data information means you have to have a knowledge of expectations or framework. To go from information to certainty, it's a sampling problem. And then to go from certainty to a decision requires input of values. So I'm saying the same thing multiple ways. You see something, you think about it, you do something. Data, information, knowledge or certainty and wisdom, understanding into action. 

So we're talking about now the cognitive issue of knowledge. This is one of Bloom's Taxonomies writer at the University of Chicago starting in the '50s. And we're looking at cognitive as we move from the bottom left of the pyramid. Remember something as true, being able to explain it, then apply it and justify it, and then evaluate it and make decisions. And ultimately others added this to Bloom about 20 years later: create, study extend. So this is in a sense of cognitive process. You don't just remember something, but you can apply it. You can explain it. And then you can ultimately do something with it. Same rhythm. I know something, I can apply something, and I can make a decision. 

Question 3: What is knowledge?

Well, knowledge is necessary to move from data to information. I gave you the example. If I observe someone in there, just a report or not much of an interpreter, I have an observation. If now I tell you in context, it's a first-year medical student, you say that's okay. It's a second-year resident. It's not okay. So our knowledge of the framework is what allowed us to interpret the data and move from data to information. And I'm going to be spending some time later in the talk about frameworks and how they affect what the teachers do. But knowledge is necessary to convert an observation into some information. And knowledge is also necessary to interpret a sampling of data through a blueprint into some decision. So knowledge is applied in this process. I think you know this, the reason why I'm showing it in this graphic form, is to set up this ancient idea that knowledge is justified true belief. And I'm going to play with this idea in my own way for the next minute. 

JTB: Justified True Belief

So JTB, knowledge as the bedrock of going from understanding to action. What the student or the resident tells you about how methimazole affects the physiology of thyroglobulin and iodine metabolism must be remembered as true. They have to report it accurately. I think we all accept that.

They have to justify it. Meaning the student or the resident can say I can apply it in this situation and I can interpret these results accurately. This is what happened to the patient's thyroid hormones as a result of this. Or they can explain it, justification is an explanation. It's what people when they write multiple choice tests call deeper level or advanced level or Level 2 questions. It's not just remembering, it's interpreting something.

And finally, and this is the point of this joke I'm making with JTB now, is belief. There is a commitment to acting, basing management on it. The student has enough confidence in their knowledge of the physiology to say, I'm going to make a decision affecting this patient's health based on this. I'm using this antibiotic, this anti-hypertensive. I'm going to do this operation on the belly, this is chololaparoscopic cholecystectomy. I have enough confidence in my knowledge of the anatomy but I'm going to base this patient's health on it. You see what I'm using?

I'm using the word belief here as an emotional commitment. Not just that they remember it correctly. It's true. Not just that they can explain it, they can justify it, but they God damn believe it. How many of us when we're asking a resident or a student a question, try to assess their confidence in it? If they come close, I find myself saying, that's great, and I want to move on because I'm in a hurry.

But the real question is, have they internalized this sense, this promise of expertise? Pellegrino's promise? That I got this. I understand the anatomy of physiology well enough. I'm going to bet this patient's life on it.

So I like this idea of JTB. So the teacher's role, does the student recall in the information questly that you asked them the facts? Do they apply it? But some trying to foster their confidence is the student's professional identity such that they say, I have a promise to this patient that I understand their anatomy. Before I walk into the room, I know the anatomy of the right upper quadrant. And I'm not going to walk into the room DLR for the lap coli until I have that. Building confidence and capability is more than just cognitive. Anyway, that's point I'm making. I'm using physiology as the example because of Rick, but it could be anatomy.

What do I need to know? 

So what do I need to know? Can the student or the resident internalize the idea of what is adequate knowledge? What does adequate knowledge look like? What do I need to know? So I'm looking for my, what do I need to know cards here, that I basically give to every resident student and endocrine fellow I work with and say within 24 hours of having a patient whose say thyroid, allow me the endocrine reference, you have to know how methimazole works, how good is it? And how bad is it?

I want you to internalize this idea that your promise of expertise is to say, how does it work? How good is? And how bad is it? And if you don't know that, then your promise of professional expertise is lacking.

Now, I don't say it in dire terms like I'm going to fail you or anything like that, but you see the concept that I'm after? That the student has to internalize an idea of what it looks like. And it's not sufficient that they know the answer to the question, but that they realize that they sought these answers, and that the faculty expects them to know how does it work? How good is it? And how bad is it? And there's other jargons for this.

You could call this self-regulated learning. Not just self-directed, self-regulated because they knew what they were looking for. They weren't reading Harrison as if it were fiction, they were reading it for the answers. You could call this metacognition. There's a lot of jargon for this, but do you see the concept that the faculty says the student has internalized a set point. The same way that the hypothalamus has a set point for glucose and osmolality, we've built into the student a set point for what adequate knowledge and a promise of expertise looks like. I think you guys have been doing this. I'm just giving my jargon for it. So what do I need to know? 

Everything is a movement from understanding into action

Reporter, interpreter, manager, educator, and curriculum then is an invitation. So if I have a clerkship medical student who reports the facts and says the patient has a cough and sputum, I say, what do you think is going on? Give me two or three possibilities. You're not a resident, you don't have to have the right answer. You're a student. Tell me what you're thinking. Give me three possible explanations and why. Curriculum is an invitation to advance from reporter to interpreter to manager. Ultimately, you go from understanding into action. 

Question 4: What is action?

So what is action? Moving into shared decision-making for being a manager. For a sub-intern, it would be suggesting a diagnostic and therapeutic plans. A resident should have an effective safe plan, I think we would all agree that criterion is a little higher. Applying knowledge to specifics. 

So how is understanding formed?

Understanding moving into action. I said before that once you move into action and we're talking about decisions that affect patient's life, this is now ethical. Values are now at play. And how do you build this in? What is the context of making a decision that affects the patient's life for the physician and for us? Can you form the correct understanding of the patient's world and what adequate knowledge of the patients world looks like in a classroom? Even in PBL, which Rick introduced, which I love. It's great. That's very, very interactive. Dr. Steinweg is teaching in that course. How do you build in that context? 

Well, Rick had the idea when he was building the new curriculum here and he wrote about this years before he got here. Actual student-patient contact under preceptor supervision with real patients, nursing homes, hospitals, as well as standardized patients. He was envisioning that the correct understanding cannot be formed in the classroom. The promise of expertise, the promise of duty is not made to me or you or to the multiple choice tests. The promise is made to patients. So unless there is work with actual real patients, we don't know, I shouldn't say we don’t know. I went to medical school 40 years ago, right. I turned out okay. I think some of you did too, even though we were in the classroom, but trying to think of the role of health systems science and seeing the big system essentially requires it. So at Virginia Tech Carilion begins with real patients in the first. This was extraordinary. Only a few schools are still even doing that now. So this is a real achievement. And as you go from 49 to double that students, it takes resources, it takes an investment in faculty. Facultoss means capability. The faculty are the people who create the capability in the students. 

So in the 20th century model, the Flexner report, Carnegie 1, was the 2 plus 2 in which you learn basic science, that's the bottom of my triangle, before you saw patients. I cited a paper that was from IAMSE Ed Finnerty. Rick got the Finnerty award from IAMSE for his many contributions. That's why I felt I needed to cite that, but that was the old model. And the question is, what do you do to vary around this? 

So this is my medical school, but others look like that. The pre-clerkship period in red is no longer two years, it's only a year-and-a-half. And so you start working with real patients, but is that as good as seeing real patients from the beginning of first-year? I don't know. We don't have these randomized trials. 

The pediatrician Bronfenbrenner, came up with the idea of the patient at the center of a huge series of concentric circles, but what we're after through health systems science, and here I'm making the transition from the cognitive to the ethical, is the patient's universe is not what happens with me in the clinic today. It's what happens when they go home and I ain't there. Now, sometimes with virtual care, we can actually follow up now. It's a new innovative idea of the medical student or the intern calls the patient and videos with them at home a week or so later to see what's happening, but being able to visualize the patient's universe and their values now becomes important. 

People have tried spiral curricula. Here, you're having early care. I wonder if we should abandon the Flexner notion of 2 plus 2 and go back to what they were doing in the 19th century. And you start by seeing patients under the tutelage of Chad Demotte. And you have science at the same time. The question I'm asking is, can you form the correct understanding about action being ethical in the context of the classroom? And I don't know the answer to that. 

Question 5. Evaluation of expertise and duty

Judgment about success in JTB and application?

Now I want to transition to the role of the faculty in evaluating. Evaluation, the etymology of the word evaluation, I've underlined it there, is value comes from the Latin valere to be strong, strength, capability. What is the strength we have in mind the purpose, is this resident that I'm working with today on the wards in the hospital, building capability? Are they making progress towards independence? Have they internalized this idea of what expertise and duty looks like? 

Question 6: What is the capability we must create?

So we, in creating capability in the resident or the student, not just simply have to make sure they learned how methimazole works or what's the anatomy, but they have internalized a concept of what adequate knowledge looks like. This is a standardized set point in the resident. I love the question on your face, Leah, it's good. A standard against which learners judge their current performance, what they just did or what they are in the process of doing. Have they internalized some standard of what expertise and duty look like against which they are comparing their own performance? Do they have a mental model of this? In the pre-clerkship period, it's JTB, justified true belief, are they building enough knowledge that they have confidence in? In the clinical years, it's moving from understanding into action. 

What is capability? What is training versus education?

When we talk about capability, I'm going to make the traditional distinction here between training and education to show you what I mean. Training means, you can train somebody to what's common, typical and predictable, the guidelines work, and you probably don't need physicians for this. And maybe one of the ways we break the cost curve of what 20% of the GDP in the economy now is on dollars and it's hurting the economy, it's hurting the very nature of democracy, maybe physicians are doing too much, maybe we need more other people, physician's assistant, advanced nurse practitioners, others who do things. Advanced nurse practitioners, of course, can deal with more complicated patients, but you see what I mean, non-physicians doing things. You can train a corpsman to put in an IV. You can train a corpsman, sorry it's a military person, a physician's assistant to say that's simple cystitis, give them this drug and we'll be okay.

Education means we're training, we're preparing for the unpredictable complex, the need to figure it out, so that's an educational process, that's the specific capability that we're trying to create. 

The traditional definition of competence Epstein and Hundert, this is 20 years old now, the habitual and judicious use of communication, knowledge, skills, reasoning, emotions, values, and reflection in daily practice for the individual. This is what I call, and this is the 1999 paper, an analytic framework, it breaks competence apart.

Competence Defined Syntheticially

As opposed to a more synthetic definition which says, is the ability to give to each situation all that the situation needs and not a lot more. Economy of action. Each patient needs a special combination of my head, my hand, and my gut, KSA, knowledge skills, and attitudes. And how do we build that capability? 

So capability is thinking reason, decision making.

And here, I want to emphasize as I have through the whole talk commonality of all the ideas. So observation, reflection, action, data, information, knowledge, wisdom, report, interpret, manage, and these are all the same idea it's just different jargon. 

The ACGME introduced milestones, and the purpose was to make the competencies understandable. Dr. Nazca wrote this in his 2007 essay, that the competencies weren't working, so we had 23 milestones, five stages each, 3-5 behavioral anchors per stage, so you end up with 23 different sheets that look like this. How many of you were Program Directors using this stuff? There you go. And this stuff is written by people like me, for people like me. And there are some reasons why we're doing this, the ACGME, God bless them, is trying to keep Congress out of our business, so they've adopted a very granular model, but I want to propose something that's a little simpler and that may not be legible to you across the top but basically, can we simplify this? This is observing, this is reporting, this is interpreting, this is managing, and this is educating. And if you look, I can do this for all the specialties. 

Here's one for Dr. Learman, this is obstetrics. This is patient care and obstetrics and it's the same thing, reporting, interpreting, managing, educating. Not because RIME… Not because I’m smart, but because that's the rhythm of all activity, you see something, you think about it, you do something. People ask me how I invented RIME, I did not invent RIME, I just coined some jargon. That rhythm, you can find in Francis Bacon, you can find it in the Rig Veda, you can find it in Nevis Centre, you can find it in Aristotle, you see something, you think about. It's the basis of, what we teach medical students from the beginning of med school. My colleague Paul Hemmer calls these RIME-stones. 

EPAs, the current trendiness in medical education is EPA's. I'm not showing about all Ten Cotter’s idea of judging the resident against some idea of do you trust them or not, I'm talking here about EPAs promulgated by the AAMC, these 13 list of things that we think all graduates should be able to do, that's what I mean by EPAs. And those are on the right here in blue, but it's not a framework, it is a list and what we've done. There's a publication on this about how a gathering a history, performing a physical documenting a clinical encounter that's up here, providing an oral presentation, collaborating, these are what we would call dimensions of being a reporter. Can you prioritize a differential diagnosis? Can you recognize a patient who's sick? These are interpreter functions. So we use the EPAs as a pre-digested performance dimensions and this is business jargon frame of reference training, performance dimension training. Of the 13 EPAs, eight or nine of them were true when I was a medical student 40 years ago. And incidentally, they never worked, that's the problem. People don't like lists, so you might want to think about using a framework. Don, I see you taking a picture. I'll make my slides available, Mariah, you can just distribute them later on, just don't tell the federal government I said so.

Modern Medical Curricula

So modern curriculum. Pedagogy has changed from a focused to more student-centered education, this case decision, this is Rick's paper. And what should be the content of these conversations? 

Simple evaluation of learner knowledge

The teacher should be happy if the student can describe iodine metabolism, but if not knowing these, then you have to be a little bit embarrassed.

So the resident has to be able to recognize the gap, has an adequate concept of knowledge, a search strategy, and a commitment. I'm asking faculty to determine whether or not in this student and certainly in this resident, they have internalized a set point against which they judge their own performance in the same way that the hypothalamus says, is the osmolality okay or not? That's my job, not just to make sure they know methimazole, but they've internalized what they need to know about methimazole. 

Finally, we're going to talk about the problem of students and evaluation. This is a typical pattern, I've adapted this from Liz Armstrong from the Harvard Macy. Emily Watson and her class, she rotates through medicine OB like this, and all these different hospitals with all these different teachers. And Emily has a problem getting all her teachers on the same page, do they all think alike?

Or is it a crap shoot? You get Don Steinweg for your attending, you lucked out, you get Pangaro, you're screwed. 

Professionalism is a promise.

And what is the faculty's expertise? What do we promise? If the learner cannot trust the faculty's evaluations as fair, which may affect them getting their residency of choice, what does professionalism mean?

This is an obsession of mine, the fairness. Do the students trust leadership? Do they trust the faculty? An evaluation, our values, in the word evaluation teach the student what professionalism is. And if they cannot trust the evaluation system then I think the concept of professionalism is very suspect. 

Evaluation as an example of professionalism

• So evaluation, we have to be fair to society. That the graduate is ready.
• Fair to students with transparency, feedback, and trust.
• And finally, this is the final point I'm making, we have to be fair to teachers that they have the time and the training, and they have protection.

And there are two kinds of protection. In many schools, now, I, the chair of medicine, or one of the dean, I would say if somebody wants to get promoted to associate professor, what do the students say about this faculty member? And if they had Pangaro and he happened to be a hawk or they thought I was a hawk and I was being fair and they write down some negative things, is the promotions committee can say, well, we can't promote Lou because he's too strict.

We have to be fair to teachers. But quite apart from that, there's the emotional issue of asking me to grade. We do not ask teachers to grade. Teachers, clinical faculty do not give the patient heart failure or cancer, they make a diagnosis. We do not give the student a grade. We classify them as acceptable or not. Or in my jargon, is it a reporter and interpreter or a manager, because faculty are very good at making diagnosis.

Feedback to learners

So I would be happy if the feedback I gave the student today helped her with her next medicine attending. But I would drop dead with embarrassment if the feedback I gave the student made them worse with the next attending because the advice I gave them DeMotte, disagreed with or Sweet disagreed with, so I hurt the student. So you have to make sure that the system guarantees that the feedback is useful and that when I give feedback, I'm not speaking for myself, I'm speaking for the department of medicine or surgery, or peds. 

Question 7: How does the physiology help?

What is the faculty's own set point? I'm back to set point, back to homeostasis and back to Rick. What is the construct in the head of the teachers? And is it aligned across all the teachers in medicine? And maybe even between medicine and peds and family medicine, maybe even surgery and psych. What is in the head, and do the teachers have receptors for my jargon? 

Re-imagining Faculty Development in Health Professions Education

So here I'm using the notion of, this is another article from your group, communities of practice. What is an educator? Look at the bottom of the slide. Education, ex-ducere means to lead out of dependence into independence. That's the ancient Roman word. Pedagogy, the Greek word means the same thing, ped-agogy, leading a child. Ex-ducere leading out of dependence into independence. Are you telling me I need to hurry? I'll speed up. Creating capability. 

So here's the image from the Stanford faculty development program faculty, learner, patient care. In a word, progressive independence is what we're after in an image. At the end of the process, the faculty need no longer be physically present. That's what we're doing. Now, I have to take care of the patient and the patient's well-being. But to the extent that I'm there for the resident, my job is to foster their independence. They are not my tool. I am there to help them become independent. So this notion of capability. 

Question 9: What is physiologic communication?

So physiologically, I'll go through this quickly. This is basically a spinal reflex arc. You feel some pain, afferent spinal reflex, efferent, signal received, integration, response, observation, reflection, action. Here's that the level of the cell surface receptor, signaling, cell surface receptors, internal recognition and cell signaling. So something changes. Everybody understands this in medicine.

It's a three-phase process. You see something, you think about it, reduce something, observe, reflect, act, report, interpret, manage. 

And so my final plea is the simplicity of fairness to faculty, that we do something that works for them. So every faculty member understands history and physical assessment and plan or SOAP. That's the rhythm of RIME, reporter, interpreter, manager. 

And it's kind of saying that we communicate through neurotransmitters. Body communicates rapidly through dopamine, which has a molecular weight of 150, not through immunoglobulin, which has a molecular weight of 150,000.

So here's the six ACGME competencies now reflected to you as something with a molecular weight of 150,000. 

This stuff is written by people like me, for people like me, it's not written for my thousand faculty. We must be fair to faculty if we want consistent evaluation.

So do we need six competencies, 13 EPA's or something like RIME.

So here again is this idea of the rhythm of RIME is on the left and the details the EPA's can be used to populate RIME. 

If you're in GME, PGY1, 2, and 3 in the bottom reporter interpreter matter on the y-axis and you expect people to get better, of course. But not only do you expect them to get better by the end of the process, at their last year, you expect them to be at the manager educator level up there on the upper left. Everything in the core. So an internal medicine for heart failure, pneumonia, pulmonary embolism for surgery, say appendectomy, colon, whatever it is, it ought to be in the core. For internal medicine, if this X is down here, is in the core. If that patient with the X down here on the lower left, if this was a GI bleed for a resident in internal medicine that'd be a red flag because it's in the core. But suppose it was pheochromocytoma or malignant fibrous histiocytoma. It's not in the core. So the faculty needs to know what's in the core and needs to know at what level of capability, manager, educator for anybody finishing the OTB. 

So here's the final slides. The lessons of physiology, unifying ideas expect understanding of mechanism, build capability, which means they have internalized a physiologic set point for what expertise and duty look like. And then the framework use mental models that the faculty already have receptors for, like the rhythm of RIME. 

Embrace complexity, act with simplicity, thank you Rick. 

Questions from the audience

>> [inaudible question posed by an audience member]

Dr. Pangaro>> Yeah. You heard I've done a lot of theater and I think you would say he's a very competent… I'm terrified of giving this talk. I regard experience as an opportunity to fail because psychiatrist is that okay. I think most of us do. Whether I'm seeing a patient or giving a talk. It's built into us, it's hard wired. So what I want for the students is that the faculty members saying, this is good, you nailed this, you understood this, keep doing that. When you're 20 years older, there will be new drugs, there will be new therapies. You will have to do the same thing. And I want you to feel excited that the craft you have chosen, you get to have fun dealing with human physiology, what an extraordinary gift and help people at the same time. At this point, I segue into I suppose what lawyers do but you see what I mean, it's thrilling, it's exciting and you get to help people. As I was saying earlier, to trace it to me, this is like playing tennis at Wimbledon, playing Mozart at Carnegie Hall. What we do is so complicated, so risky, and yet we get to help people. It ain’t perfect. But boy, I want the students to feel that thrill. So building that confidence and knowing that it always feels it's taking... When I was a student, there wasn't HIV, when most of you were students there wasn't COVID. So all of a sudden, yeah, you got to level up, but that's exciting. What an opportunity to keep learning and you get to help people at the same time. You get well-paid, you have fun, and you get to help people. So that confidence is that you can do this. But I appreciate your question because I still feel it and I know every student I talk to. So oh my God, what's Pangaro going to tell me now, and I said been there, I still feel that way. Yea. 

Whicker>> Comments, questions, challenges? 

[Audience member]>> There's a tension in medical education. When I hear you talk about understanding the how, why, what's going to be good, what's going to be bad with my treatment, and how important the knowledge of signaling of cellservice all the way to the nucleus, how important that is to answering the how and the why and how good and how bad, and yet we don't have enough doctors to go around and so what we're seeing is an evolution of a new training program where you would become a physician's assistant. In three years of college and two more years of training, I don't see this understanding. I understand we have a label of a doctor and other labels, but if understanding what happens so deep and so descriptive gets us a better care. How would we triage patients in the system that's moving from DOs and MDs to a lot of physician assistants, nurse practitioners.

Dr. Pangaro>> At the system level, it requires not just leadership from above, but preparedness from below. I don’t think we can protect democracy if the cost of the GDP as medical care goes from one in five dollars to one in four dollars. And therefore, there must be more efficient ways of doing this and therefore there must be physician's assistant, nurse practitioners and others. That's at the mezzo level below. At the local level, what I see in my diabetic patients is they graduated high school without any understanding of human physiology. Now, they don't have to know the insulin receptor, or the pancreas anatomy, but they do have to know that their body thinks it's 200,000 years ago and starvation is around the corner because the internal milieu is still designed for threats. The body doesn't know there's a Safeway or McDonald's down the street. It thinks starvation is down the street. So the internal milieu is designed against deprivation, starvation, dehydration, etc. Until my patients understand something about human physiology and in addition to expertise or knowledge, have a sense of duty to this fellow citizens, that you do wear a seat belt when you drive a car, you do wear a helmet, and you do get vaccinated, things like that, we can't solve that problem. The problem to me begins with a failure of the educational system in the country to prepare. So in this interim phase done, I think we need a combination of computerized programs with teaching for patients that I have a sore throat, before I go to the doc in the box walk-in clinic, I go to the Internet site, and I put in certain things and it interacts with me and says, well, you don't have much of a fever this and this, you can do this. If you develop this in the next 24 hours, then do this. So I'm not just seeing the problem as what the providers do, I see the problem as the preparedness of citizens for what we need to do. Let me go further with this. This notion of embracing complexity,  and acting with simplicity, economy of action is, I think at the heart of the profession. For 25 years I have struggled with the coders in my own hospital telling me I wasn’t coding high enough because on every patient I didn't automatically do a review of systems. I'm asking questions about the knees or the skin in a graves disease patient who might see in just two weeks ago. So they want me to do this and I say, no, I leave stuff out. I'm an expert. Economy of action is what I do. And they said then we don't get paid enough, you have to do more of this. So at the very top level, and again, I have to say this honestly, educational research, what I do is a weak force in our universe. The strong forces of the ACGME, the LCME who theoretically are trying to represent the government and Medicare. And then there's families and things like that. But we're a weak force in this. So I think that the system at the highest level needs the leadership that says, we can't do our job, we can't… 20% of the GDP but our health rates are worse than Sweden or France. We can't do this unless you change secondary education in the United States. This is a 50-year project. Otherwise, we don't have the resources to deal with climate change, international wars. So I believe in what you said, we do need other people and I'm not sure what they are, allied health professionals. Our medical school, we just have added. We now have a College of Allied Health Sciences. So we train, we call them Corpsman, physicians assistants, and other people in San Antonio to try to provide them for the military. I think that's really important. Where I was going with this though with complexity is I tried to get the American College of Physicians to adopt the notion that instead of doctors for adults, that we were expert clinicians and that we needed to be paid to embrace complexity but act with simplicity. And that they needed to support this because right now my coders are telling me we get paid for acting with complexity, documenting crap that's not necessary or doing other things. So I see that as… And I have to say, my generation, I'm about to exit for this. I failed. I was not eloquent or clear enough with the people I talked with, the ACP or the AAMC what I thought they needed to be doing. Of course, the AAMC is all about the deans and the Council of teaching hospitals. You know this one. When Medicare said medical students could not write in the chart, I represented the Alliance for Clinical Education. I went to the AAMC and said you've got to help us. You've got to tell them that students need to be writing…. You're doing an experiment in medical… and they wouldn't do it. The AAMC did not want to get involved in that question because they were interested in the flow of dollars, they didn't want to be fighting them. Eventually as you know, Medicare did listen and then they changed some of the rules on it. So you've got to think in terms of the systems at all level. This is why I think Jed Gonzalo’s work and others in health systems science is it's not just social determinants of disease, it's thinking in systems. And for some reason, systems thinking is not a natural act. There was a paper we use in our course from MIT's. Why is that everybody is so solipsistic? We grew up, I trust myself, I trust my family. If you're my neighbor, I might trust you, but I don't trust anybody else. So silos is how human beings grew up. We still see the problem, what's going on in Ukraine now. And since the idea is that nothing is a community, there's always tension fighting for something or other. So not being able to solve the problem of human nature in the short run, I think the problem is seeing things as a system. I don't know if that gets anywhere near to a reasonable answer. 

[Audience member]>> So I don't know how many people know Jeff Foxworthy, the comedian. He talks about ”yunta”, which is do you want [inaudible] in Southern. And it struck me that when we talking about physiology, which by the way, I love the String Theory of Everything, a slide there a minute ago. What happens physiologically when we've had a system that's been working at some lower level, like your thyroid or your adrenal gland, and all of a sudden you realize that what you have now end organ resistance, where the end organ no longer has ”yunta”. That organ maybe the student who somehow doesn't have ”yunta” or isn't motivated or maybe a system. I just throw that out there for just thinking on your part to see if you….

Dr. Pangaro>> I think you know, I don't have the answer, but I would love to have the answer. And I work really hard to build a team of younger people. And most of the team I built are now in their 40s or 50s, so they're not young enough to understand the current generation. Yes. Thank you. Sure. So what the expectations are of students and how we select students. Our school like many others, are trying to pick students who have a sense of social responsibility and different kinds of backgrounds so that there's a little bit more of “I want to do this.” I'm not just in it because I like doing brain surgery. It's not simply the thrill of doing it or the social status that you get. But I think it is the job of the faculty to say, we're trying to build capability in you so that you're able to do this independently. And we're going to describe to you what we think it's like, what the thrill is of understanding, articulating a promise of duty. And if it turns out you don't want to do that, then you can choose another profession. But this is the way the faculty see it. What's nice about CBME, Competency Based Medical Education, is that there is in fact a mandate to tell society that everybody we graduate is ready for internship. So I can't say to a student, well, I know you've been here for five years and you are now $250,000 in debt but so I'll let you graduate. Can't do that anymore. That's the purpose of CBMEs. For me CBME is not an intellectual idea at all, but it's a social idea of accountability, of answerability. The message is clear, the faculty must make this determination so therefore the students need to understand it. And what's nice about CBME is as long as you're willing to try, we'll help you. I have some students who graduate in six years. They take a little longer and sometimes they've had a family problem, something that is developed. They need more time to get there. But we do not get to negotiate with society about what readiness to be an internist. Or at the end of residency, we do not get to negotiate with society, say, well, we could graduate you as an internist, but you don't like diabetes but everything else you can do. We can't do that. So I think the faculty bring clarity to what our job is and passion. But if junta they don't want to do it or something, that's their individual choice. But I do feel that CBME brings a certain clarity to the faculty that we have to do this, we have to make these decisions and as reliable and valid as well. We can argue for a long time about how we make those decisions, but I think that's the answer. 

Whicker>> Thank you so much for the question. Thank you for the answer. I hope to see you all this evening. Some of you are welcome to hang back with burning questions that you have to ask. Feel free to do so as the rest of you enjoy the rest of the day.

Dr. Pangaro >> I didn't put my email address, but I'll make my slides available and will have an email. If somebody wants to email me in a week, a month; always glad to talk about this stuff.