In lieu of an abstract, here is a brief excerpt of the content:

164 Chapter 12 Are Student Teaching Evaluations Holding Back Women and Minorities? The Perils of “Doing” Gender and Race in the Classroom Sylvia R. Lazos Teaching is important. Among the traditional three main responsibilities of the professoriate—teaching, scholarship, and service—teaching is probably the most important from the public perspective. The Association of American Colleges and Universities has recently challenged its members to focus more on student learning and develop better ways to measure it (National Association of American Colleges and Universities 2006), citing the well-reported statistics that the United States is slowly slipping behind other industrial countries in student performance in math, science, and writing.1 In addition, demographics have changed the student population and its educational needs. Increasingly a greater proportion of the student population is less ready for college. Legislators, faced with shrinking state budgets, have become more prone to scrutinize what highly paid, tenured faculty members do with their time and routinely insist that they do more or better teaching . In sum, the current political climate where universities are operating demands that administrators carefully monitor faculty teaching effectiveness. While there are many ways to evaluate teaching, universities have come to rely widely on student evaluations. According to a 1993 survey conducted by management expert Peter Seldin, 86 percent of universities used student evaluations of teaching in decisions about faculty retention, tenure, promotion, and merit pay 1 The National Assessment of Educational Progress (NAEP) reports on nationally representative samples of student work in reading, mathematics, science, writing, US history, civics, geography, and the arts. NAEP has recently stated that the achievement of US students in grades four, eight, and twelve has been slipping as compared to that of other industrialized countries. For details see http://nces.ed.gov/nationsreportcard. Are Student Teaching Evaluations Holding Back Women and Minorities? 165 (Seldin 1993). The use of student evaluations grew rapidly from 1970 to the 1980s. Since 1985, many institutions have used the SIR, the Student Instructional Report, which was developed by educational assessment expert John A. Centra at the Education Testing Service in Princeton, New Jersey, which also administers SATs. Although thoughtful commentators have persistently proposed additional methods of evaluating teaching (Arreola 2000; Seldin 2004; Braskamp and Ory 1994), no other method approaches the popularity of student evaluations (Seldin 1999). These evaluations have the advantage of providing a summary number that purports to assess teaching efficacy. This makes comparisons among colleagues easier. And the supposed objectiveness of numbers washes away any possible ambiguities and complexities. This is why many university administrators continue to be enamored with student evaluations (Seldin 1999). The professoriate has produced an avalanche of articles critiquing and defending student evaluations.2 The criticisms are startling. Methodological questions start with the most basic one: what do student evaluations actually measure? Professors Harry Tagamori and Laurence Bishop have concluded that the questions on student evaluations are so ambiguous that you can’t even determine what they are asking. Tagamori and Bishop examined a random sample of student evaluation forms and found that more than 90 percent contained questions or items “that were ambiguous , unclear, or vague; 76 percent contained subjectively stated items, and over 90 percent contained evaluation items that did not correlate with classroom teaching behavior” (74–75). Another statistician, Professor Valen Johnson, assembled a massive data set of student evaluations at Duke University and concluded that there was a significant statistical link between a professor’s goal of receiving positive student teaching evaluations and grade inflation (Johnson 2003). Indeed a student’s expectation of what grade he or she will get in a class is a strong predictor of how positive the instructor’s evaluations will be for the instructor (Marsh and Dunkin 1992). These studies make a point that upon reflection should be intuitively obvious. Evaluations may not be measuring teaching effectiveness as much as they are capturing students’ subjective reactions at the moment that they are being polled, and their opinions reflect their feelings and thoughts about a range of things: whether they like the professor, whether their expectations about the course were met or they felt unsettled (perhaps because the professor deviated from the syllabus); and how well they imagined they were performing in school and in the class. Even student gossip becomes part of the picture (Feldman 1989a). Psychometric expert Mark Shevlin, comments that “students are not trained in rating or psychometrics”; rather he concludes that the main basis of their “global evaluation” of competency is “lecturer charisma” (Shevlin 2000...

Share