University of Cincinnati
Cincinnati, OH
Program Director: Peter H. Lenz, MD
Type of Program: Pulmonary and Critical Care Medicine
Abstract Authors: Adam G. Cole, MD, and Peter H. Lenz MD
RATIONALE
The Accreditation Council for Graduate Medical Education (ACGME) has identified core competencies for the development of capable physicians that training programs must address. The Next Accreditation System (NAS) for internal medicine residencies and subspecialty fellowships requires trainees meet key milestones as they progress through training, and 24 subspecialty milestones have been identified by the ACGME and American Board of Internal Medicine (ABIM). Achievement of competency in these milestones necessitates assessment of the progression of a trainee to a clinician ready for unsupervised practice. In order to assess progression to competency, some form of evaluation must be coupled with each milestone. Our Program Evaluation Committee (PEC) reviewed our fellowship curriculum and evaluation system to identify any milestones that were underutilized. We identified 5 such milestones in 3 competency domains: medical knowledge (MK3 – scholarship), practice-based learning and improvement (PBLI1 – monitors practice with a goal for improvement, PBLI2 – learns and improves via performance audit and PBLI3 – learns and improves via feedback), and systems based practice (SBP2 – recognizes system error and advocates for system improvement).
METHODS
We designed evaluation forms for fellows’ scholarly presentations that were directly linked to the milestones described above. Each evaluation was based on observable practice activities (OPAs) and utilized the entrustable professional activities model described in the NAS milestone project. Evaluations were completed immediately after each presentation by faculty and fellows, collected by an administrative assistant and entered in our graduate medical education (GME) electronic management system (MedHub). We then compared the number of milestones assessed for fellows before and after the implementation of our evaluation.
RESULTS
The mean number of milestones assessed for each fellow increased after the implementation of our new evaluation system. Total for MK-3 increased from 47 to 135 (mean per fellow 5.88 to 16.88, p=0.028), SBP-2 increased from 54 to 77 (mean per fellow 6.75 to 9.63, p=0.19), PBLI-1 increased from 22 to 33 (mean per fellow 2.75 to 4.13, p=0.16), and PBLI-3 increased from 32 to 70 (mean per fellow 4.0 to 8.75, p=0.0071).
CONCLUSIONS
The implementation of an evaluation form linked to milestones increased the number of documented milestones achieved by our fellows. For two milestones (MK-3 and PBLI-3), this increase was statistically significant. By linking the milestones with OPAs, our evaluation emphasized directly observed behaviors to limit ambiguity and measured the degree to which entrustment was achieved.
