The Program in Medical Education (PME) engages in ongoing planning and continuous quality improvement processes that establish short and long-term programmatic goals, result in the achievement of measurable outcomes that are used to improve programmatic quality, and ensure effective monitoring of the medical education program’s compliance with accreditation standards. The PME’s Office of Medical Education (OME) is responsible for planning, implementing and monitoring the educational quality improvement process, linking information gleaned from student, course, program evaluation and assessment data with quality improvement and faculty development initiatives. Faculty and staff of the OME develop and maintain systems for tracking educational outcomes and feeding results to all relevant course and clerkship directors, advisors, and other offices in the PME and the School’s affiliated hospitals.
The OME issues course evaluations online for all courses and clerkships. Student responses are incentivized by access/ability to view grades (i.e., students cannot view their grades or evaluations until after they have completed the required course and faculty evaluations). The OME reviews course evaluation trend data for each phase of the curriculum with the appropriate curriculum governance subcommittees; course/clerkship directors and site directors receive detailed results, including narrative comments. Working with PME leadership and the Educational Policy and Curriculum Committee (EPCC), course and clerkship directors, and curriculum oversight subcommittees, OME provides data for reviews of required courses and clerkships and phases of the curriculum (preclerkship, PCE and post-PCE) and a review of the curriculum as a whole every four years.
In 2018, the PME launched a central education data warehouse that includes data from learning management and student information systems for analysis and reporting. Utilizing this data warehouse architecture, the OME, in partnership with PME leadership, identifies key metrics for data-driven and data-informed continuous quality improvement.
In addition to the course evaluation system and its feedback to course directors and curriculum governance subcommittees, the PME also utilizes a complementary system of student education representatives (Ed Reps), who are identified by students (under the faculty advisors for this program) to give feedback in real time for continuous improvement as courses are in progress. The Ed Reps meet with course directors regularly throughout each course to help the faculty understand concerns that may be arising around issues ranging from the quantity and quality of preparatory materials to whether one section or group seems to be having problems that can be addressed in real time. The Ed Reps also provide periodic holistic feedback on trends in the curriculum or patterns across years, usually by sending class surveys and sharing the results with the appropriate subcommittee.
Relevant Policies
Requirement for student evaluation of courses and teaching
Medical student evaluations of courses, clerkships, and instructors provide valuable feedback on the quality of the medical education program and inform continuous improvement. In support of this, students are required to complete evaluations in all required courses and clerkships in which they participate, and of the instructors who teach them. These evaluations are conducted through the HMS student information system, OASIS. Students are required to complete all course, clerkship and instructor evaluations by the end of the course or clerkship. Course and clerkship directors must provide in-class time for the completion of these evaluations. Students are not able to view their course grade until after they have completed these required evaluations.
Course and clerkship directors and instructors are not able to view aggregated student evaluations of their courses and teaching until after they have submitted student grades to the HMS Registrar. The PME Office of Medical Education (OME) retrieves students’ evaluations of courses and clerkships, including faculty teaching in a course or clerkship, only after grades have been submitted, and generally six-weeks after the end-date of the course or clerkship. These data are considered to be the official record of student evaluation of the course/clerkship and teaching in the course for that offering.
Sharing clerkship evaluation data and reports of mistreatment with teaching faculty, and clerkship and site directors
The following parameters inform how student feedback data are shared with course and clerkship site directors and with teaching faculty.
- Individual faculty cannot retrieve their evaluations until all students’ grades have been submitted and students have moved on to the next course or clerkship. Instructions for faculty to log into OASIS and retrieve their evaluations on a self-serve basis.
- Course/clerkship and site directors cannot access course/clerkship evaluations and evaluations of all faculty teaching in their course/clerkship/site until all students’ grades have been submitted to the HMS Registrar (grades are due within four weeks after the end-date of the course or clerkship.) Instructions for course/clerkship and site directors to access evaluations via OASIS.
- Reports of student mistreatment submitted via the course/clerkship evaluation form are embargoed in the system so that course and clerkship directors are not able to access them. Instead, these reports are sent directly to the Dean for Students and Dean for Medical Education, who respond to each report individually. Alternately, students may report mistreatment in real time via the online ALERT portal. Section 7.16 Student Mistreatment Policies and Procedures provides information on reporting mistreatment.
- Faculty who wish to obtain records of their teaching ratings and student feedback to include in their promotion dossiers can access these data via OASIS on a self-serve basis or request a report from OME by contacting ome@hms.harvard.edu.
The PME’s clerkship evaluation policy is designed to encourage student feedback on teaching and learning, a critical tool to inform continuous quality improvement in medical education. Results of clerkship and faculty teaching evaluations for a course/clerkship cannot be accessed by clerkship faculty until all students’ final grades for the course/clerkship have been submitted to the Registrar’s Office.
In certain courses where the student-to-faculty ratio is one-to-one (e.g., Practice of Medicine, Primary Care Clerkship), students are provided with the opportunity to provide confidential feedback about the faculty member directly to the course or clerkship site director, without the information being released to the faculty member.
Additionally, questions about student mistreatment are included in the end-of-clerkship evaluation survey. To safeguard student confidentiality, reports of student mistreatment are embargoed in the system so that clerkship site directors are not able to access them. Instead, these reports are sent directly to the Dean for Students, who develops a plan of action to respond to each report individually. This plan may include follow-up with the clerkship site director and/or clinical site where the mistreatment occurred, but in all cases the situation is handled in such a way as to protect the student or students involved.
Policy on Unprofessional Student Comments in Course/Clerkship Surveys
The PME regularly solicits student feedback on various aspects of the curriculum in order to improve the educational experience for all students. To that end, we ask that students provide feedback that is candid, constructive and actionable for all core courses/clerkships. This is done via an electronic course survey process that ensures student confidentiality, and all course surveys and accompanying comments are withheld from course directors until after the course grades have been submitted to the Registrar.
In certain rare cases, however, comments from students cross a clear line from constructive to offensive, inappropriate or unprofessional. Because student comments are ultimately made available to the individual faculty member as well as the chair of the instructor’s appointing department and a number of educational professionals at HMS, unprofessional comments have the potential to inflict real harm on an individual faculty member, which violates a key tenet of the medical profession. For this reason, the EPCC established a process that ensures the continued integrity and confidentiality of the course survey process while providing an outlet to address the rare instances of unprofessional or inappropriate feedback that occur within the system.
In order to achieve this goal, an ad hoc committee comprised of three students and two Academic Society Advisors will be established to review any comment on a course survey deemed to be unprofessional or concerning by a course director, instructor or official in the OME. To ensure that students on this ad hoc committee represent the will of the student body, student members of the committee will be either elected student representatives to the EPCC or other students who hold elective office, e.g., the Student Council Executive Committee. If a majority of the committee agrees that the comment is inappropriate or unprofessional, IT will be aked to override the system confidentiality protections and identify the responsible student. This information will be provided exclusively to the student’s Society Advisor, who will determine the remediation approach most helpful to the student. The comment will also be redacted from the course survey record.
Updated 04/28/23