Improving Teaching Effectiveness Through The Application Of SPC Methodology

Abstract

One method used extensively to aid in determining instruction effectiveness is Student Evaluations of Instruction (SEI). This paper examines the use of statistical Process Control charts as a way to correctly measure teaching effectiveness. This field studying SEIs has produced a significant literature. It is not surprising that there is considerable controversy over the efficacy of such instruments in measuring instructor performance or student learning. Numerous factors that may influence the outcome of SET score have been identified. These include: class size, workload level of subject taught, and the nature of the subject. Clayson’s (1999) review found a profound impact due to personality variables, such as instructors’ age, and teaching experience. It is clear that SEI results can be influenced by many factors. Interestingly, this extensive literature provides no clear guidance on how to interpret SET results in order to make comparative evaluations of instructors’ performance. The research proposed in this paper suggests that variations of Statistical Process Control methods could be used in such evaluations. Only one prior paper (Marks and O’Connell, 2003) has suggested this approach. Our research examines six semesters of SET responses for all courses in our business school – a data base of nearly 30,000 responses. The paper examines what measure should be used as the standard by which an instructor’s performance should be evaluated. Specifically, it evaluates whether a school-wide, department-wide or course specific measure should be used as the standard. Statistical analyses are conducted to determine if there are significant differences on a standard measure across departments and across semesters. The goal is to develop a system that can accurately gauge faculty members whose evaluations are statistically superior or inferior. Such a system would enable department chairs and the administration to determine the relative effectiveness of faculty in teaching particular courses.