Assessing Student Outcomes in General Education at Samford University

Introduction

When we began the Samford Problem-Based Learning (PBL) Initiative, we realized that assessment would play an important role. Consequently, we formed an assessment team that consisted of both faculty members and administrators. At the onset of the PBL Initiative, the assessment team met weekly and discussed best practices for evaluating the impact of PBL upon learning at Samford University. The assessment team, in conjunction with external experts, adopted existing measures and developed new assessment measures that we believed would inform us of the quality of experience and effectiveness of PBL on undergraduate and professional education.

First, we adopted the critical thinking module from the Collegiate Assessment of Academic Proficiency (CAAP) in order to gauge the reasoning skills among students who were exposed to PBL versus critical thinking skills of students who were not exposed to PBL. In addition, we developed three assessment tools here at Samford University: (1) The Student Attitudes and Activities Assessment (SAAA), (2) The Instructional Landscape Survey (ILS), and (3) The End of Course Evaluation (ECE). The SAAA and ECE were designed to determine the effectiveness of PBL at the university, college, and course levels. The ILS was developed to depict the current state of our university's instructional landscape; in other words, we wanted to determine the range of instructional activities Samford faculty members use in their classrooms. The SAAA and ECE are examples of indirect assessment measures that examine perceptions rather than directly address learning. The CAAP critical thinking module is an example of a direct assessment method designed to gauge the development of undergraduates' reasoning abilities. From these beginning assessments, we believe PBL is having a positive impact upon our undergraduates and we expect to design future assessment efforts to supplement our current efforts.

Assessment Results

Critical Thinking Module of the Collegiate Assessment of Academic Proficiency (CAAP)

The Collegiate Assessment of Academic Proficiency (CAAP) is a standardized instrument that consists of four passages that are representative of the kind of issues commonly found in a college curriculum. Each passage presents one or more general arguments and uses a variety of formats, including case studies, debates, dialogues, overlapping positions, statistical arguments, experimental results, or editorials. This module was designed to assess gains in critical thinking that occur among students during the first two years of college. The critical thinking module is used to measure a student's ability to clarify, analyze, evaluate, and extend arguments.

Fall 1998 and Spring 1999. The 32 item, 40 minute, multiple-choice critical thinking module was administered by the Office of Institutional Research to entering freshmen at the beginning of the fall 1998 semester and at the end of the spring 1999 semester. These entering freshmen were enrolled in yearlong general education course sequences (i.e., Cultural Perspectives and Communication Arts) that are required of all entering freshmen. We were able to get all of the students in Communication Arts and Cultural Perspectives to complete this assessment because their participation was required in order to receive a grade for the course. During this initial implementation, no significant differences in critical thinking test scores emerged between PBL and non-PBL students.

Fall 1999. The Office of Institutional Research also administered the critical thinking module to entering freshmen enrolled in PBL and non-PBL sections of Communication Arts during the fall 1999 semester. Students were required to participate in order to receive their grades at the end of the fall semester. There were no PBL sections of Communication Arts offered in the 2000 spring semester. We needed, therefore, to re-administer the CAAP at the end of the fall semester rather than waiting until the end of the 2000 spring semester. Again, no significant differences in critical thinking emerged between PBL and non-PBL students.

Student Attitudes and Activities Assessment (SAAA)

Another measure developed by the Samford PBL Initiative, with the assistance of our external assessor Dr. Elizabeth A...