Response to the Review Symposium of Four Critical Years

30 RESPONSE TO THE REVIEW SYMPOSIUM OF FOUR CRITICAL YEARS by Alexander W. Astin University of California, Los Angeles and Higher Education Research Institute, Inc. My general reaction to the critiques of Four Critical Years by J. Bruce Francis, Gerald Gurin, Arthur W. Chickering, and G. Lester Anderson (Higher Education Review, Winter, 1978) is that they are thoughtful, well-reasoned, and constructive analyses of a very complex empirical study. They would make an excellent last chapter or appendix to the book. I shall first discuss several points about which the reviewers generally agreed, and then discuss specific comments of each reviewer. There was general agreement that most of the college effects reported in the book were small in magnitude, a point which I might have emphasized more in my discussion of policy implications. Gurin and Chickering feel that I should have been more cautious in jumping to policy conclusions from empiri­ cal findings of relatively small magnitude. Francis, on the other hand, feels that my method of analysis underestimated true college effects (by as­ signing the confounded input-environment variance to input). I have mixed feelings about this issue. On the one hand, the modest size of most of the multiple correlation coefficients and the substantial amount of measurement error that inevitably creeps into data of this type persuades me that Francis has an important point: The coefficients are "lower-bounds" estimates of college impacts which would surely be larger if we had had better measures of the various dependent variables. A related issue is the habit many of us have acquired of using the "percentage of variance accounted for" as an index of how important particular variables are. To some extent we have all been sold a bill of goods by the statisticians and measurement specialists concer­ ning this particular index of "importance." Two or three percent does not sound like much. The fact is that variables that account for what seems to be a trivial amount of variance can, at the extremes, make a substantial practical difference. For example, although college grades account for less than 5 percent of the variance in students' starting salaries, an A average is worth about $1,800 more in starting salary than a C average (Astin, 1977, p. 155). Similarly, in my earlier dropout study, living in a dormitory dur­ ing the freshmen year accounted for only about 2 percent of the variance in dropout rates, "but this figure translates into an absolute increase of about 13 percent in a student's chances of finishing college. There are at least two problems with multiple R's and "percentage of variance" figures. First, they are highly sensitive to differences in the way the independent and de­ pendent variables are distributed. Second, they are not expressed in units of the dependent variable. In short, it is much more meaningful to evaluate the magnitude of any particular environmental impact in terms of unit changes in the dependent variable, rather than talking about percentages of variance accounted for. A second point of agreement concerns the methodology and general concep­ tualization employed in the research: The reviewers were generally positive on these points and some went so far as to suggest that the approach might serve as a general model for future research. Since I have devoted a good deal of my professional time and energy to these conceptual and methodologi­ cal issues during the past 18 years, the reviewers' reactions are particular­ ly gratifying. Francis points out that the book should have included much more of the technical details of the analyses. I agree. Researchers these days are in a real bind on this matter, and I must admit I have no magical solutions to 31 suggest. On the one hand, the scholar and scientist in all of us feels strongly about presenting enough technical detail to permit professional col­ leagues to make reasonable judgments about the technical soundness of the study and meaning of the findings. The practical man in each of us, on the other hand, wants our work to be read and, hopefully, to be taken seriously by practitioners. Unfortunately, this conflict is usually resolved for us by the commercial publishers who reproduce...