The importance of teaching evaluations to the tenure and promotion of women faculty cannot be underestimated. Administrators routinely consider classroom teaching in hiring, tenure, promotion, and salary decisions and increasingly rely most heavily on quantitative student ratings. Scholars who have attempted to determine whether/how gender enters into students' evaluations of their teachers generally fall into two camps: those who find gender to have no (or very little) influence on evaluations, and those who find gender to affect evaluations significantly. Drawing on insights developed from sociological scholarship on gender and evaluation, we argue that the apparent inconsistency on the question of whether student evaluations are gendered is itself an artifact of the way that quantitative measures can mask underlying gender bias. We offer concrete strategies that faculty, researchers, and administrators can employ to improve the efficacy of the system of evaluation.