“Can student evaluations be made more representative? Testing alternative strategies,” the second paper resulting from Hamilton’s study of course evaluations, was recently published in the international journal Studies in Higher Education. The paper was co-authored by Associate Professor of Government Erica De Bruin, Henry Platt Bristol Chair of Public Policy and Professor of Economics Ann Owen, and Irma M. and Robert D. Morris Professor of Economics Stephen Wu.
Though student evaluations are regularly used to assess the quality of faculty teaching, the co-authors said “low response rates and unrepresentative student samples can bias the conclusions drawn from them.”
To learn about why this happens, the group conducted a randomized experiment to test whether student evaluations can be made more representative of the broader student population through changes to the prompt or the timing of feedback solicitation.
Among their key findings was that certain groups of students, including Pell Grant recipients, students with low GPAs, and those later in their college careers, are less likely to complete teaching evaluations and to write more than three words in response to qualitative questions evaluating the instructor across all study conditions.
“Black, Hispanic, and multi-racial students are less likely than white students to complete traditional evaluations solicited at the end-of-semester,” they said, noting that “providing an alternate prompt, which asks students to articulate their own criteria for effective teaching, increases the likelihood that students write more than three words and increases the total words written, conditional on responding.”
Regarding the timing of evaluations on response rates, the study showed that when solicitation of feedback is delayed until the following semester, overall response rates decrease, “particularly for seniors, but to a lesser degree for Black, Hispanic, and multi-racial students than white students, resulting in a more racially representative sample overall.”
Offering a suggestion to make faculty evaluation more relevant they said, “Given the limitations of student evaluation instruments, one specific recommendation is for colleges and universities to reduce the emphasis on teaching evaluations and to evaluate teaching effectiveness in a more holistic manner by incorporating multiple forms of evidence.”
