Tinkering With The Traditional To Assess And Promote Quality Instruction: Learning From A New And Unimproved Instructor Evaluation Instrument

Abstract

For over 30 years survey instruments have been used in colleges of higher education to measure instructional effectiveness. Extensive research has been conducted to determine which items best capture this construct. This research study was triggered by a college of education’s enthusiastic but failed attempt to create a new and improved instructor survey based on this research. Researchers found that the new instrument was no better than its predecessor. Student halo ratings contaminated results, reliability was lower than expected, and the survey results indicated one single dimension – general teaching effectiveness. Two associated variables of considerable interest, course relevance and rigor/demand, were also contaminated by student halo rating. Based on these findings and the extensive literature on student surveys of teaching effectiveness, we argue that traditional surveys based on conventional items may be valid for evaluating global teaching effectiveness and other summative purposes but not for the formative, self-diagnostic, and reflective purposes anticipated. New ways of evaluating teaching in higher education are presented and discussed. The article shares insights into theory-based survey development and a plan for validation.