Closed Minds?: Politics and Ideology in American Universities

Appendix B

The population of interest was U.S. college and university faculty, defined as the primary faculty who come into contact with undergraduates in the course of teaching academic subjects. Unfortunately, there is no available sampling frame that we could simply adapt to our purposes. Commercially available lists are inadequate in their coverage. Further, universities often protect their lists of faculty (for example, many universities will not distribute copies of their phone book to outsiders), or, when available, such a list normally would not distinguish between names of those who are eligible within our criteria and those who are not. Most universities, however, do publicize details about their faculty, both as a source of prestige and to inform current and prospective students. Nearly all universities identify their faculty on various pages of the school’s website. The University of Virginia’s Center for Survey Research was asked to develop a sampling plan and to execute the scientific sampling methods required for the study.We gratefully acknowledge the assistance of its staff in implementing the survey. Sample Size In order to obtain 1,000 complete questionnaires at a conservatively projected 33 percent completion rate, a sample of at least 3,000 professors was needed. Although it would maximize random selection to select participants across every school, this would have been prohibitive in both time and money inasmuch as each school selected would have to be inventoried to select a Appendix B Sampling Methodology 224 participant. Thus we used a multistage sampling design with probability proportional to size sampling at the first stage and equal counts of professors selected from each school at the second stage. This design yielded an equal probability of selection for each member of the population. We decided that eighteen professors could be selected at each school. Thus, in order to yield at least 3,000 professors, 169 schools would be ideal. In summary, we chose 169 schools and selected eighteen professors from each school to yield 3,060 names for the study sample.Sampling was done in two or three stages,depending on the available information,with the first stage selecting institutions and the remaining stage(s) selecting individual professors. Selecting the Sample of Institutions A complete list of U.S. colleges and universities from which to sample was obtained from the American Association of University Professors Research Office, at no cost. The list included all four-year institutions as well as universities in the United States and reported the total number of full-time faculty split by function (instruction, research, and public service). The list also included each school’s class, as defined by the Carnegie Classification of Institutions of Higher Education. The original data were produced by the U.S. Department of Education’s Integrated Postsecondary Education Data System in the fall of 2004. This could potentially introduce a small bias into our sampling to the extent that our master list omits any colleges created since 2004. However, this number is likely to be small and could well be zero. Selecting the Schools To ensure that the sample was appropriately representative of different types of colleges and universities, the sample of schools was stratified. A separate sample was selected for doctoral, master’s, and baccalaureate institutions based on the Carnegie Classification. The number of schools drawn from each stratum was proportionate to its size. This would ensure that the number of institutions would generate that stratum’s share of faculty members. Appendix B: Sampling Methodology 225 Population Selected sample Carnegie type Schools Instructors Percent Schools Instructors Percent Doctoral 259 204,213 52.9 89 1,602 52.4 Master’s 604 127,758 33.1 56 1,008 32.9 Baccalaureate 592 53,796 13.9 24 432 14.1 Total 1,455 385,767 100.0 169 3,060 100 The list of institutions included the number of faculty devoted to instruction , which was used as the measure of the school’s size. A specially written program of commands in Statistical Package for the Social Sciences was used to select institutions proportional to this indicator of size, labeled FT_Instr (full-time instructor) in the data file. The sampling procedure was run separately for each list, representing the institutions from each of the three Carnegie types of institutions. Selecting the Sample of Professors As criteria for eligibility, participants had to be an instructor of an academic subject...