"You CAN Play Mathematical Equations on the Violin!": Quantifying Artistic Learning Outcomes for Assessment Purposes

The french philosopher of science Jules Henri Poincaré (1854–1912) once said, "You cannot play mathematical equations on the violin." By that he meant simply that there was a decided difference between the sciences and the arts.

Yet in today's academy, internal effectiveness reviews and external accreditation agencies require us to do just that—"to play mathematical equations on the violin," so to speak—by insisting that film department administrators and faculty come up with measurable outcomes and results, as well as the means by which to evaluate artistic qualities such as talent; beauty; and "good" screenwriting, cinematography, directing, acting, editing, sound, and set design, and so on.

This article demonstrates how the Florida State University Film School complied with this seemingly impossible task and how the Bachelor of Fine Arts (BFA) and Master of Fine Arts (MFA) programs have actually benefited from embracing mandated statistical assessment opportunities.

To begin, I should note that I was not originally a proponent of such assessment strategies. The vast majority of professors in the arts disciplines are opposed to the "ed-speak" vocabulary and "one size fits all" measurements that are being imposed on universities by state legislatures, regional accreditation boards, and internal "quality enhancement" initiatives. Initially, I tended to think that the professional judgment of a qualified faculty member (usually by means of a letter grade) was enough assessment for my taste. True, that sort of evaluation could be somewhat subjective, but I felt that a certain degree of subjectivity was part of the process of measuring students' work in the arts. After all, "you cannot play mathematical equations on the violin."

In fact, when I first confronted a mandated assessment process—at Georgia State University in 1996—we left most of the evaluation work in the hands of the faculty. We said that a faculty committee would review student films, as well as written term papers, scripts, and exams. We proposed a "capstone" course (which seems to be what so many accreditation agencies are after) and flirted with the idea of an exit examination. For the most part, we left the process qualitative—and in the hands of professors—through the old-fashioned method of grading. We just collectivized the process a bit more by fashioning a faculty committee.

Then in 2004 I arrived at the FSU Film School, in the year that the Southeastern Association of Colleges and Schools (SACS) committee was performing its ten-year accreditation review. An elaborate computer portal was prepared that specifically required measurable (that is, quantifiable) outcomes. An evaluation instrument was developed that used a 1.0–10.0 Likert scale measurement for evaluating each of the key areas of filmmaking. A scale of this sort was originally the brainchild of Ken O'Donnell of Chapman University, but we wanted an assessment instrument that was specifically designed for the FSU Film School—one that was also planned and approved by the faculty.

Originally, there were over fifty items represented, all to be rated on a 1–10 scale. These items were developed at a faculty retreat in which the faculty members from each area—writing, cinematography, directing, sound, editing, and set design—all wanted to have six to eight questions each. (See Appendix A.) These fifty-odd items were eventually boiled down to thirty-five. But because the assessors of the students' films were going to be busy industry professionals, it was decided to consolidate some of the areas and eliminate others so that the number of times could be reduced down to a more manageable number: twenty-two. (See Appendix B.) We realized that each of our evaluators was going to be asked to view six MFA thesis films and six BFA theses, and we did not want to overburden them. Supplying them with DVDs of all the films and allowing them to fill out their ratings directly on the college's Web site, with a proper, secure password, made their work easier.

In the final FSU Film School document, the respondent is allowed to indicate how sure he or she...