Penn State University Press
Abstract

The purpose of this study was to assess the educational value of cumulative participation over four years in intensive co-curricular experiences at Bridgewater State University in Massachusetts. We compared responses to the National Survey for Student Engagement (NSSE) institutional-contribution-to-gains and satisfaction items for 103 senior co-curricular participants and a matched group of nonparticipants. Participants reported significantly greater gains than their nonparticipant peers in nine areas of personal development, with moderate-effect sizes on developing a personal code of values/ethics and contributing to the welfare of your community, and smaller-effect sizes for the seven remaining areas. Among co-curricular participants, the number of years of co-curricular experience was significantly and positively correlated with gains in understanding people of other racial/ethnic backgrounds, deepened sense of spirituality, and agreement that if they had it to do over they would choose the same institution. This study provides a model for assessing the value of out-of-class experiences, while addressing common challenges with such assessments.

Introduction

Scholars have known for several decades that participation in co-curricular experiences contributes to students' academic and personal development.1 Positive outcomes such as improved persistence and enhanced cognitive skills are associated with out-of-class involvement, and particular co -curricular experiences, such as community service or athletic participation, have been linked to specific outcomes for students. But scholarly findings about the general value of co-curricular activities may not provide sufficient evidence to support decisions about individual programs on a particular campus, either for accountability or for program improvement, so national research must be supplemented with local evidence that demonstrates the educational value associated with participation in the institution's own constellation of co-curricular experiences. This study provides a model for institution-specific assessment, and it addresses three challenges that commonly face student affairs leaders and assessment professionals in determining the local educational value of out-of-class experiences.

One of the challenges to assessing the educational value of co-curricular experiences is the tendency to use student self-reports about the value of the experiences themselves—for example, asking student government senators to report what they learned from their role in student government. Such assessments are a valuable starting point, but the responses can be biased. Students who enjoyed elective office might tend to inflate the educational value of that experience if asked to report about it directly, for example.

A second challenge to assessing the educational value of co-curricular experiences is the difficulty of determining whether they have lasting value. It is easiest to get students to participate in surveys and focus groups either during or immediately after their co-curricular participation—for example, surveying participants in an alternative spring break service trip shortly after their return to campus. Assessments such as these cannot show whether the value of the experience persists over time.

A third challenge to assessing the educational value of co-curricular experiences is the need to account for potential confounding variables such as academic major or gender. For example, it can be tricky to determine the skill gains associated with a student's service as a health and wellness [End Page 31] peer educator as opposed to the similar skill gains that might come from courses in the student's major.

Purpose

The purpose of this study was to assess the educational value of participation in intensive co-curricular experiences sponsored by Bridgewater State University's Division of Student Affairs. Examples of these intensive co-curricular experiences include a year-long community service program, university-sponsored athletic teams, and paraprofessional positions such as resident assistants and orientation leaders. These co-curricular experiences are intensive for students, demanding a greater investment of time and energy than most other student activities. They are also resource-intensive for the institution, requiring a commitment of space and specialized facilities as well as staff time for training, coaching, and supervision. The bottom-line question for the division and for the university is "Is it worth it?" Do students' experiences in these programs justify the investments that the university and the students themselves must make? National literature suggests that these programs may have real value for students, but an institution-specific study was needed to confirm those national findings in the context of Bridgewater State University.

This study centers on an analysis of National Survey of Student Engagement (NSSE) responses by Bridgewater State University seniors, comparing students who had engaged in intensive co-curricular experiences at any time during the preceding four years to a matched group of seniors who did not participate. It is designed to overcome the three challenges to assessing the educational value of out-of-class experiences, discussed above. This study avoids the first challenge—asking students to describe their co-curricular experiences directly—by using the NSSE institutional-contribution-to-gains questions as the primary dependent variables. These questions ask students to estimate the extent to which the institution as a whole has contributed to their gains on sixteen outcomes, without reference to out-of-class activities or any other particular experiences that may have contributed to these gains. Our study solves the second challenge—determining whether gains persist over time—by considering only the responses of senior students, who may have participated in co-curricular activities at any point in their college career. Therefore, it only measures gains that are strong enough to persist to the senior year. We [End Page 32] address the third challenge—potential confounding variables—by using a matched comparison group. Co-curricular participants were matched one-for-one to nonparticipants on six potentially confounding variables simultaneously. While not all rival explanations for student outcomes could be controlled in this analysis, the method used here provides a foundation for reasonable claims that certain overall gains that students experience during their college careers are, in fact, related to participation in certain co-curricular experiences.

Literature Review

The NSSE Institute identifies participation in co-curricular activities as one of the "educationally enriching experiences" that contribute to student engagement.2 This statement is well founded in a rich history of scholarship about students' out-of-class experiences, going back several decades. For the purposes of this article, only a brief overview of that scholarship is possible.

General Outcomes of Out-of-Class Experiences

Considering out-of-class experiences generally, it has long been known that student involvement is associated with positive academic and personal development outcomes.3 Pascarella and Terenzini's meta-analysis of research on cognitive skills and intellectual growth during college found evidence that "suggests that co-curricular activities such as involvement in clubs and organizations may foster critical thinking." Further, they found that "role-taking opportunities that accompany extracurricular-peer involvement in college have positive impacts on growth in principled reasoning." They conclude that both "academic and social involvement in whatever form . . . exert statistically significant and positive net influences on student persistence and degree completion," and that "extracurricular and social involvement during college . . . has a net positive effect on student self-reports concerning the development of career-related skills."4

Outcomes from Particular Types of Experiences

Beyond the findings that out-of-class experiences and co-curricular involvement in general yield positive outcomes, researchers have also found that [End Page 33] different types of co-curricular experiences are related to different academic and personal outcomes. Examples considered in this brief review include athletes, members of fraternities and sororities, and community service participants because these make up a large proportion of the co-curricular participation considered in this study.

Scholars have examined athletic participation as a predictor of different academic and personal outcomes, and the findings have been mixed. On the positive side, researchers have found that participation in intercollegiate athletics is positively correlated with self-rated physical health, leadership, and satisfaction with student life.5 Male intercollegiate athletes have a "small but significantly greater likelihood of finishing a bachelor's degree" than their peers,6 and research has demonstrated that "participation in intercollegiate athletics has a positive and significant effect on persistence and graduation." African American male athletes "may gain more in both academic and social self-concept than their white counterparts."7 An examination of national NSSE data comparing student athletes vs. non athletes found that athletes were as likely as their peers to participate in effective educational practices.8 On the other hand, many of the research findings about the effect of athletics are negative. Pascarella and Terenzini's meta-analysis found negative effects of athletic participation—particularly for men in football and basketball—on verbal, quantitative, and subject matter competence as well as on growth in critical thinking.9

Members of Greek-letter social fraternities and sororities have also been studied extensively. There is a body of scholarship that shows widely varying outcomes for student learning and personal development among Greek organization members. For example, living in Greek housing may be linked to lower educational outcomes.10 Numerous studies have also found a positive relationship between Greek membership and increased alcohol consumption, which is itself linked to negative outcomes, including "death, sexual assault, bodily injury, academic failure, and academic underperformance." 11 Greek membership has also been found to be associated with less openness to racial diversity, lower grades, lower intellectual development in the first year of college, and increased rates of academic dishonesty.12 On the positive side, Greek membership has been found to provide a form of peer interaction that fosters learning.13 An examination of NSSE data from 42,182 students compared levels of engagement between Greek and independent students. The researchers found that "after controls, Greek members appeared to be equally and sometimes more engaged in academically challenging tasks, active learning, student-faculty interaction, [End Page 34] community service, diversity, satisfaction, and on learning and personal development gains."14

Pascarella and Terenzini describe the impact of Greek membership as "complex," citing studies that demonstrate that "fraternity membership would appear to inhibit growth in general knowledge acquisition and critical thinking for men during the first year of college," that "Greek affiliation, in general, may inhibit growth in principled moral reasoning and increase the likelihood of both academic dishonesty and binge drinking during college," and that "fraternities and sororities have a net and negative influence on members' racial-ethnic attitudes and openness to diverse ideas and people." On the other hand, they found evidence that Greek membership has a positive impact on career-related skills, interpersonal skills, community orientation, and commitment to civic engagement.15

Outcomes from participation in service-learning and other community service activities have been studied as well, and the findings have typically been positive. For example, volunteering has been found to be positively associated with development of skills in leadership and public speaking, commitment to promoting racial understanding, and the likelihood of attaining a bachelor's degree.16 Participation in community service significantly increases students' leadership and political influence and their belief that people can make a difference.17 Pascarella and Terenzini's meta-analysis found that community service has "statistically significant, positive effects on civic and community-oriented or 'other-oriented' attitudes and values"; that "service involvement positively influences such college outcomes as the importance students place on social justice, sense of civic responsibility, and importance of service to the community"; and that "participation in voluntary or other service activities during college has a net positive impact on the development of career-related competencies."18

Use of NSSE Data for Program Improvement and Accountability

The National Survey of Student Engagement is an annual survey of first-year and senior students at four-year colleges and universities about the nature and quality of their undergraduate experience. It was "conceived in 1998 as a new approach to gathering information about collegiate quality and piloted in 1999 with funding from the Pew Charitable Trusts."19 NSSE provides benchmark data for institutions on five scales that assess effective educational practice: Level of Academic Challenge, Active and Collaborative [End Page 35] Learning, Student-Faculty Interaction, Enriching Educational Experiences, and Supportive Campus Environment.20

Institutions of higher education increasingly use the results of the National Survey of Student Engagement as a measure of the quality of the educational experience they provide. Colleges and universities use NSSE data for both program improvement and accountability to internal and external constituencies. About 2.3 million students at nearly 1500 colleges and universities participated in this national study from 2000 through 2010. Many of these institutions have made NSSE "a key component of their approach to assessing quality."21

One way institutions may use NSSE results is to combine their NSSE data with internal institutional data to investigate specific local questions. Kuh has challenged institutions to use this approach to "develop rich, campus-specific profiles of the undergraduate experience."22 He has called as well for institutional NSSE studies that "control for student precollege characteristics," since certain groups of students may have collegiate experiences that differ markedly from those of their peers.23

For student affairs professionals, it is helpful to know that the programs and experiences they sponsor have been shown nationally to yield positive outcomes, and it is worrisome when national studies show mixed or negative results about the types of experiences that a student affairs division offers. In this sense, national data provide a context for understanding local programs. However, for the purposes of program improvement and accountability, data and analysis must be institution-specific. The present study provides one model for achieving this goal, using NSSE data that are already collected at many institutions.

Methods

Institution

This study was conducted at Bridgewater State University (BSU), a public institution with about 11,000 students, located in southeastern Massachusetts. Founded by American public education pioneer Horace Mann in 1840, BSU is the oldest permanently sited institution for teacher preparation in the United States, but it has grown beyond that mission to offer degree programs in five academic colleges—business, education [End Page 36] and allied studies, graduate studies, humanities and social sciences, and science and mathematics. BSU is the largest of the nine Massachusetts state universities and the fourth largest of the twenty-nine public college and university campuses in Massachusetts.24 In 2009-10 BSU awarded 2,085 degrees ( bachelor's, master's, and certificates of advanced graduate study). BSU draws 95% of its undergraduate students and 90% of its graduate students from Massachusetts. The student body is predominately female (62%), undergraduate (83%), and white (86%); and about half of the entering first-year students are first-generation college students.

Instrument

The National Survey of Student Engagement was created "to assess the extent to which students are engaged in empirically derived good educational practices and what they gain from their college experience."25 It is administered online or in hard copy in the spring academic term every year to random samples of first-year and senior students at participating institutions. The survey comprises ninety-nine items in various formats. In sixty-six survey items, respondents report the frequency with which they engage in various behaviors; indicate whether they have participated in, or plan to participate in, certain educationally engaging opportunities; describe the quality of their relationships on campus; estimate how many hours each week they spend on various out-of-class activities (such as employment, commuting, or studying); and describe aspects of the campus culture (e.g., emphasis on providing academic support). Then, nineteen questions ask students to estimate the various types of gains they have achieved from their overall experience at the institution and to rate their overall satisfaction with the college experience. The final page of the survey contains fourteen demographic items.26

Reliability

The Center for Postsecondary Research, which sponsors NSSE, conducts extensive research on the instrument's psychometric properties following each year of administration.27 For the institutional-contribution-to-gains items and overall satisfaction items that are the focus of this study, internal consistency reliability studies have yielded the Cronbach's alpha values for college seniors displayed in table 1.28 [End Page 37]

Table 1. NSSE Cronbach's alpha values for senior students on dependent variable items
Click for larger view
View full resolution
Table 1.

NSSE Cronbach's alpha values for senior students on dependent variable items

Student-level test-retest analysis has also been reported on the NSSE benchmark scales. For senior students, these analyses yielded Pearson's r values ranging from .658 (Supporting Campus Environment) to .760 (Active and Collaborative Learning).29 Institution-level analysis found correlations for senior students ranging from .806 (Supportive Campus Environment) to .939 (Enriching Educational Experiences).30

Validity

Response-process validity (consistent interpretation of items across various student groups) has been assessed using cognitive interviews and focus groups. These studies found that the NSSE instrument was clearly worded and easy to complete and that the survey "generally performs well for students from different racial and ethnic backgrounds."31 Content validity (whether the survey covers all aspects of the construct) is assessed by [End Page 38] panels of experts. Predictive validity (whether NSSE scales correlate well with outcome measures) has been assessed for each benchmark scale vs. measures of academic achievement (specifically, credit hours earned and persistence) using logistic regression. Each of the benchmark scales was found to be significantly and positively predictive of one or both of the outcome variables.32

Participation in the National Study

Sample

In fall semester, Bridgewater State University provided the NSSE Institute a roster of all first-year and senior students, from which NSSE drew the sample for the survey administration. The eligible sample for the survey (students who were enrolled as either first-year or senior students in both the fall and spring semesters) was 2,960 students. The definition of eligibility for participation is determined by NSSE.

Usable responses were received from 1,206 students (first-year = 560, senior = 646), for an overall response rate of 40.7%. Respondents were 71% female, 81% white, and 89% enrolled full-time. Senior respondents—the focus of the present study—were 71% female, 78% white, and 80% enrolled full-time.

Data Collection

NSSE may be administered in hard copy, via the Web, or in mixed format. BSU chose Web administration. BSU's Office of Institutional Research and Assessment provided the NSSE Institute the email addresses of all selected students as well as the text of customized invitation and follow-up messages from BSU's president. The invitation to participate in the survey went to the selected students in mid-February, with a follow-up message a week later and a reminder message two weeks after launch. Students received the final reminder message six weeks after the launch date. The timing of the survey launch avoided conflict with either the opening of spring semester or the spring break in early March. NSSE staff handled all aspects of the data collection.

Data Analysis

The NSSE Institute provided first-round data analysis, comparing responses from first-year and senior students at BSU to those from students at three different groups of institutions: all NSSE respondents (714 institutions), [End Page 39] those from institutions in the same Carnegie Classification as the study institution (172 institutions), and those participating NSSE institutions that are also among BSU's official peers (7 institutions). NSSE's analysis uses weighted means to adjust the results for differential response rates across various demographic groups (by gender, part-time/full-time status, and so on).

The Institutional Study

Selected Co-Curricular Experiences

BSU's Division of Student Affairs identified the following student groups as having intensive co-curricular experiences—that is, those requiring intensive investment of time and other resources from both students and the university:

  • College-sponsored athletic teams;

  • Peer mentors;

  • Event planning and support staff;

  • Children's Center teaching assistants;

  • Community Service participants (Jumpstart or Alternative Spring Break only);

  • Health and wellness peer educators;

  • Student programming committee officers;

  • Orientation leaders;

  • Leaders of student media groups (the newspaper, yearbook, and radio station);

  • Members of social fraternities and sororities;

  • Officers of the Residence Hall Association;

  • Resident assistants; and

  • Student government officers and senators.

Research Questions

The following research questions guided the study:

  1. 1. Are there significant differences in the gains students achieve during college for students who have participated in intensive co-curricular experiences at any time during the past four years ("participant group") vs. their peers who have not participated ("comparison group")? [End Page 40]

  2. 2. Are there significant differences in overall satisfaction with the college experience between the participant and comparison groups?

  3. 3. Among participants, is there a significant relationship between the number of years of co-curricular participation and the strength of estimate of gains?

  4. 4. Among participants, is there a significant relationship between the number of years of co-curricular participation and overall satisfaction with the college experience?

Defining the Participant Group

The Division of Student Affairs provided rosters of participants in each of the designated intensive experiences for the four academic years preceding the survey. The Office of Institutional Research and Assessment merged these rosters with the data set of NSSE responses, identifying 103 senior students who completed the NSSE and who had at least one year of intensive co-curricular participation during their college careers. For each of these 103 students, the number of years of participation in each experience was recorded and then the total years of participation in all experiences was computed. Table 2 displays the range of participation in each of the co-curricular experiences for these students.

Some students participated in more than one type of co-curricular experience, and some participated in one or more experiences for multiple years. Therefore, some students had many more than four years of total participation. For example, a student who was an orientation leader for two years and a resident assistant for one year would have three years of total participation. Table 3 shows the number of students who fell under each category of cumulative co-curricular experience.

Creating the Comparison Group

To limit the number of potential confounding variables we created a matched comparison group. The 103 co-curricular participants, described above, were matched one-for-one to 103 nonparticipants on the following variables simultaneously: gender, race/ethnicity, first-generation status, first academic major, classification, and self-reported grades.

Once the participant and comparison groups were identified, the researchers then compared demographic data for these students to other senior respondents to determine the extent to which these students were typical of the whole. For the 206 students in this analysis, [End Page 41] the gender distribution was identical to the overall group of 646 senior respondents (71% female). The racial/ethnic distribution of the participant and comparison groups was similar to all senior respondents (participant and comparison group = 76% white; all senior respondents = 78% white). However, students in the participant and comparison groups were somewhat more likely to be enrolled full-time than the larger group of all senior respondents (87% vs. 81%).

Table 2. Number of student participants and years of cumulative participation in each intensive co-curricular experience by students in the participant group
Click for larger view
View full resolution
Table 2.

Number of student participants and years of cumulative participation in each intensive co-curricular experience by students in the participant group

[End Page 42]

Table 3. Number of years of cumulative co-curricular experiences for students in the participant group
Click for larger view
View full resolution
Table 3.

Number of years of cumulative co-curricular experiences for students in the participant group

Data Analysis

The dependent variables for this study were eighteen NSSE items: sixteen institutional-contribution-to-gain questions and two overall satisfaction items. Two analyses were conducted. First, we used a Mann-Whitney test to compare responses to the dependent variable questions for the co-curricular participants and the comparison group. Next, for co-curricular participants only, we used a Spearman rho analysis to measure the correlation between the number of years of total co-curricular participation and the students' responses to these same outcome questions. We used nonparametric comparisons for both analyses because each of the dependent variables is ordinal level.

Results

Table 4 displays the means and standard deviations for each of the dependent variables. Data appear here for the co-curricular participants, the comparison group, and BSU's senior respondents overall.

Table 5 displays the results of the Mann-Whitney test comparing the co-curricular participants to the comparison group on each of the dependent variables. As these data show, the students who had participated [End Page 43]

Table 4. Means and standard deviations for dependent variable questions
Click for larger view
View full resolution
Table 4.

Means and standard deviations for dependent variable questions

[End Page 44]

in intensive co-curricular experiences during the previous four years reported significantly greater gains from their education in the following nine areas, as compared to peers with no such participation:

  • Acquiring job- or work-related knowledge and skills;

  • Working effectively with others;

  • Voting in local, state, or national elections;

  • Understanding oneself;

  • Understanding people of other racial and ethnic backgrounds;

  • Solving complex real-world problems;

  • Developing a personal code of values and ethics;

  • Contributing to the welfare of your community; and

  • Developing a deepened sense of spirituality.

The analysis revealed moderate effect sizes (.30 or greater) for developing a personal code of values and ethics and contributing to the welfare of your community. There were small effect sizes (greater than .10 and less than .30) for the remaining items.

Table 6 shows the results of the Spearman rho analysis examining the relationship between number of years of intensive co-curricular experience and the strength of response to the dependent variables, considering only those who did participate in these co-curricular experiences. These data indicate a significant and positive relationship between the number [End Page 45]

Table 5. Mann-Whitney results for co-curricular participants vs. comparison group
Click for larger view
View full resolution
Table 5.

Mann-Whitney results for co-curricular participants vs. comparison group

[End Page 46]

[End Page 47]

[End Page 48]

of years of participation in intensive co-curricular experiences and the strength of three of the dependent variables:

  • Gains in understanding people of other racial and ethnic backgrounds;

  • Gains in developing a deepened sense of spirituality; and

  • Agreement that if they had it to do over, they would choose to go to the same institution again.

Table 6. Spearman rho results for number of years of cumulative co-curricular participation vs. educational gains and overall satisfaction with the educational experience
Click for larger view
View full resolution
Table 6.

Spearman rho results for number of years of cumulative co-curricular participation vs. educational gains and overall satisfaction with the educational experience

[End Page 49]

Discussion

The findings of this study demonstrate that participation in Bridgewater State University's intensive co-curricular experiences over the course of a student's college career is associated with important personal gains that can be valuable to students in their postcollege lives, in terms of employment or citizenship or individual maturity and competence. The previous scholarship suggested positive outcomes for participants in community service activities and mixed outcomes for athletes and fraternity and sorority members. It was not possible to analyze the data in this study by specific type of co-curricular experience because so many students were involved in multiple types of activities. Thus, we focused here on the cumulative effect of many types of involvement. However, athletics and Greek membership represented the two most common forms of participation in this study in terms of cumulative years of experience, so—in light of previous scholarship—one might expect at least some of the results of this analysis to have been negative. Yet these findings were uniformly positive. There were no dependent variables on which the comparison group had significantly higher scores than the participant group. Such local data do not supplant previous scholarship, even in decisionmaking at the institutional level, but they do provide an important counterpoint and context for national studies when decisions are made about the value of these programs at Bridgewater State University.

We also found that increased participation is significantly and positively correlated with greater benefits in some outcome areas, and it is not correlated negatively with any of the outcomes studied here. This is an important finding since some might question (indeed, some have questioned) whether additional co-curricular participation is advantageous to students—that is, whether becoming heavily involved in more than one co-curricular activity might compete with classroom responsibilities for the student's time and attention. For these students, increased co-curricular participation was associated with positive, not negative, outcomes.

Of note is the fact that co-curricular participants reported greater gains than their peers in understanding people of different racial and ethnic backgrounds, and that more co-curricular participation was associated with greater gains in this area. This finding is consistent with previous literature that associates peer interactions with experiences with diversity.33 On the other hand, it argues against previous findings that Greek membership is associated with decreased openness to diversity.34 Greek membership represented nearly one-fifth of the years of co-curricular experience in this study. [End Page 50]

This study examined the cumulative impact of participation in many types of intensive co-curricular experiences over the student's whole career at BSU. Some of the experiences occurred as long as four years before the students completed the survey. The findings suggest that the educational value of these experiences is not short-lived, but continues to work in students' lives as a part of their total engagement in collegiate education.

Overall, our findings suggest that when students participate in these intensive co-curricular experiences, they are investing their time and energy wisely. Similarly, when the university devotes staff time and facilities and fiscal resources to support intensive co-curricular experiences, those investments lead to better educational outcomes. Students who participate in such experiences are likely to derive benefits from their college education that their nonparticipating peers are less likely to receive.

Limitations

Not all types of co-curricular experiences were included in this study. In fact, not even all types of intensive co-curricular experiences were studied. For example, participation in student performance groups or undergraduate research might have been included, but were not because the purpose of this study was to assess the educational value of co-curricular experiences that are sponsored or supervised by the Division of Student Affairs at BSU. Student performance groups and undergraduate research do not fall under the purview of student affairs.

Not all potential confounding variables could be controlled in this analysis. For example, the effects of student employment, commuting, and SAT scores were not considered here. Important as these issues are, they could not be used in this analysis because cell sizes prevented matching the participant and comparison groups on additional variables simultaneously.

Value of the Study

Despite its limitations, this study is important in several different ways. First, it demonstrates the value of intensive co-curricular experiences at least for this particular institution. Our findings have already been cited in program review self-studies for the student affairs departments that sponsor these programs, and they will be used in BSU's upcoming regional accreditation [End Page 51] self-study. Our results may also be used to justify additional institutional investments, or to argue for inclusion of these co-curricular experiences in the university's diversity plan, or to recruit student participants.

At another level, this study is important because it serves as a model for other institutions that want to test the value of co-curricular (or other) experiences for their own students, while addressing three common challenges to this type of assessment. This analysis avoided asking students to rate directly the educational value of their co-curricular experiences. The dependent variables for this study were questions about the value of the overall college experience, not about the value of co-curricular experiences. That is, this is an unobtrusive measure of the value of co-curricular experiences, and therefore not liable to the sort of biased responses that would be possible in an overt question about the experiences themselves. Even though the dependent variables were about the larger college experience, we succeeded nonetheless in establishing a relationship between the co-curricular experiences and educational gains by using a matched-group comparison of participants and nonparticipants.

We also measured persistent rather than short-term gains, addressing a second common challenge for assessments of out-of-class experiences. This analysis considered the responses of senior students, and it included in the participant group those seniors who had taken part in intensive co-curricular experiences at any time in the preceding four years. Many of these seniors were not participants during their senior year, yet this study identified value for co-curricular experiences that occurred in prior years and showed cumulative value for students who participated in multiple intensive experiences over their time at the University.

Our study addressed the challenge of rival explanations for its findings by matching the participant and comparison groups one-for-one on multiple variables simultaneously. While not all potential confounding variables could be controlled, such an approach allowed us to consider the variable of participation, independent of the effects of gender, race/ethnicity, first-generation status, first academic major, classification, and self-reported grades.

NSSE is widely used, and many institutions already have these data for their own students. This study demonstrates one way that information about students' participation in various programs may be merged with NSSE data to examine the educational gains associated with their participation. If institutions choose to use the NSSE institutional-contribution-to-gains and satisfaction items for these analyses, as was done in this study, the [End Page 52] results may yield not only internal assessment of the programs' value, but also possible benchmark data for similar programs at peer institutions.

Cathryn Turrentine
Keene State College
Tony Esposito, Michael D. Young, and D. David Ostroth
Bridgewater State University
Cathryn Turrentine

Cathryn Turrentine is Director of Institutional Research at Keene State College; at the time of this study she was Acting Assistant Director of Institutional Research and Assessment at Bridgewater State University.

Tony Esposito

Tony Esposito is Assistant Vice President for Student Affairs at Bridgewater State University.

Michael D. Young

Michael D. Young is Associate Provost for Academic Planning and Administration at Bridgewater State University.

D. David Ostroth

D. David Ostroth is Vice President for Student Affairs at Bridgewater State University.

Notes

1. See for example Alexander W. Astin, Four Critical Years: Effects of College on Beliefs, Attitudes, and Knowledge (San Francisco: Jossey-Bass, 1977); Ernest T. Pascarella and Patrick T. Terenzini, How College Affects Students (San Francisco: Jossey-Bass, 1991); Ernest T. Pascarella and Patrick T. Terenzini, A Third Decade of Research, vol. 2 of How College Affects Students (San Francisco: Jossey-Bass, 2005); George D. Kuh et al., What Matters to Student Success: A Review of the Literature (Washington, DC: National Center for Education Statistics National Postsecondary Education Cooperative, 2006).

2. National Survey of Student Engagement, From Promise to Progress: How Colleges and Universities Are Using Student Engagement Results to Improve Collegiate Quality (Bloomington: Indiana University, Center for Postsecondary Research, 2002), 11.

3. Astin, Four Critical Years, 241.

4. Pascarella and Terenzini, Third Decade of Research, 208, 369, 440, 542.

5. Alexander W. Astin, What Matters in College? Four Critical Years Revisited (San Francisco: Jossey-Bass, 1993).

6. Pascarella and Terenzini, How College Affects Students, 393.

7. Pascarella and Terenzini, Third Decade of Research, 439, 267.

8. Paul D. Umbach et al., "Intercollegiate Athletics and Effective Athletic Practices: Winning Combination or Losing Effort?," paper presented at the 44th Annual Association for Institutional Research Forum, Boston, June 2004.

9. Pascarella and Terenzini, Third Decade of Research.

10. Gregory S. Blimling, "A Meta-Analysis of the Influence of College Residence Halls on Academic Performance," Journal of College Student Development 40 (1999): 551-61.

11. Stephen R. Porter and John Pryor, "The Effects of Heavy Episodic Alcohol Use on Student Engagement, Academic Performance, and Time Use," Journal of College Student Development 48 (2007): 456. For studies on the relationship between Greek membership and increased alcohol consumption see, for example, Barry D. Caudill et al., "High-Risk Drinking Among College Fraternity Members: A National Perspective," Journal of American College Health 55 (2006): 141-55, or Randy M. Page and Michelle O'Hegarty, "Type of Student Residence as a Risk Factor in College Students' Alcohol Consumption and Social Normative Perceptions Regarding Alcohol Use," Journal of Child and Adolescent Substance Abuse 15, no. 3 (2006): 15-31.

12. John C. Hayek et al., "Triumph or Tragedy: Comparing Student Engagement Levels of Members of Greek-Letter Organizations and Other Students," Journal of College Student Development 43 (2002): 643-64.

13. Alexander W. Astin, "What Matters in College," Liberal Education 79 (1993): 4-15. [End Page 53]

14. Hayek et al., "Triumph or Tragedy," 643.

15. Pascarella and Terenzini, Third Decade of Research, 616-17.

16. Astin, What Matters in College?, 391-92.

17. Dwight E. Giles Jr. and Janet Eyler, "The Impact of a College Community Service Laboratory on Students' Personal, Social, and Cognitive Outcomes," Journal of Adolescence 17 (1994): 327-39.

18. Pascarella and Terenzini, Third Decade of Research, 339, 369, 516.

19. National Survey of Student Engagement, Quick Facts, http://nsse.iub.edu/html/about.cfm (accessed January 17, 2011).

20. NSSE, Benchmarks of Effective Educational Practice, http://nsse.iub.edu/pdf/nsse_benchmarks.pdf (accessed January 17, 2011).

21. Kuh et al., What Matters to Student Success; NSSE, Quick Facts; National Survey of Student Engagement, From Promise to Progress, 3.

22. George D. Kuh, "What We're Learning about Student Engagement from NSSE," Change 35, no. 2 (2003): 31.

23. Kuh et al., What Matters to Student Success, 104.

24. Bridgewater State University, History of Bridgewater State University, http://www.bridgew.edu/depts/IR/AAG_F10.pdf (accessed January 17, 2011).

25. George D. Kuh, The National Survey of Student Engagement: Conceptual Framework and Overview of Psychometric Properties (Bloomington: Center for Postsecondary Research, Indiana University, 2001), http://nsse.iub.edu/html/psychometric_framework_2002.cfm (accessed November 9, 2010).

26. National Survey of Student Engagement, NSSE Survey Instrument, http://nsse.iub.edu/html/survey_instruments_2010.cfm (accessed November 9, 2010).

27. Kuh, National Survey of Student Engagement.

28. NSSE, National Survey of Student Engagement Measurement Scales, Component Items, and Intercorrelation Tables (NSSE 2009 Data), Reliability-Internal Consistency, http://nsse.iub.edu/pdf/psychometric_portfolio/Reliability_Intercorrela-tion_2009.pdf (accessed January 17, 2011). For this and other psychometric data, the values are reported for senior students because seniors are the focus of this study.

29. NSSE, Reliability—2009 Student-Level Test-Retest Analysis, http://nsse.iub.edu/pdf/psychometric_portfolio/Reliability_Test-RetestAnalysis_Student-Level_2009.pdf (accessed January 17, 2011).

30. NSSE, Reliability—2009 Institution-Level Temporal Stability, http://nsse.iub.edu/pdf/psychometric_portfolio/Reliability_TemporalStability_InstitutionLevel_2009.pdf (accessed January 17, 2011).

31. NSSE, Validity—Cognitive Interview and Focus Groups, http://nsse.iub.edu/pdf/psychometric_portfolio/Validity_CognitiveInterviews.pdf (accessed January 17, 2011).

32. NSSE, Validity—Predicting Retention and Degree Progress, http://nsse.iub.edu/pdf/psychometric_portfolio/Validity_RetentionAndDegreeProgress.pdf (accessed January 17, 2011).

33. Kuh et al., What Matters to Student Success.

34. Hayek et al., "Triumph or Tragedy." [End Page 54]

Share