Plagiarism, a Turnitin Trial, and an Experience of Cultural Disorientation
In October 2003, all faculty at Massey University, a research university in New Zealand, were invited to join a university-wide trial of Turnitin.com, a plagiarism detection system being considered for widespread use to combat a perceived “plagiarism epidemic.” The university framed the Turnitin trial as an investigation into issues of academic integrity and a step in strengthening academic misconduct procedures, with no reference to plagiarism as an issue of academic writing. This is not, perhaps, surprising since rhetoric and composition is an emerging discipline in New Zealand and is not yet fully established as part of the curriculum. As the only full-time faculty member employed at this time to teach academic writing at the university, I joined the trial, hoping to bring a different perspective on the issue.
The purpose of this essay is to consider the value of Turnitin primarily from the context of reflecting, as a writing teacher, on what the trial taught me about writing, about my role as a writing teacher, about students and learning, and on the gaps that exist in our understanding of and relationship with one another in the student-teacher relationship. To deepen my reflection, I have used a form of reflective practice established by Donald Schön and developed by the British school of action research (see, for example, Whitehead and McNiff). This reflective paradigm, as described by Richard Winter, Alyson Buck, and Paula Sobiechowska, “requires more than observation. It requires us to engage in a process of introspection leading to self-clarification” (186). This essay summarizes the process of observation and self-clarification I engaged in as part of the university trial.
The university trial involved ten faculty members across a range of disciplines. My part of the trial took place in a thirteen-week communication in science course that is compulsory for all freshman science students. The course is taught through a combination of lectures and tutorials and is fully internally assessed. Demographically, the class is unusually homogeneous for a New Zealand context, with an even gender split and very few ESL students (predominantly Asian, but with some Maori speakers); 90 percent of the class are recent school leavers.
I wanted to start with an understanding of my students' knowledge of what constituted plagiarism. My perception, based on teaching experience, was that students entered university unprepared to use academic sources in their writing. However, attitudes of others within the university challenged my perception. I was unprepared for the level of anger expressed by some colleagues joining the Turnitin trial. Many of these faculty, particularly those working in fields with large numbers of ESL students, animatedly discussed the “plagiarism epidemic” (although there are no studies on rates of plagiarism in New Zealand universities) and expressed pleasure, almost jubilation, that these graceless students would be found and punished. Such a perspective appeared to be supported, though reinterpreted, by an article written by a student in the student newspaper entitled “George Bush Cheats so Why Can't I?” In this piece, Jess Cameron suggests that the code of behavior established by political and business leaders in recent years means that “an unstable foundation of morals regarding cheating and plagiarism for ‘Gen X’ is set” (16). Claiming that plagiarism is a result of cynicism, and laughing at claims of “unintentional plagiarism,” she challenges so-called liberal attitudes and calls for a harder line on plagiarism, which she describes as devaluing her own educational achievement.
So were my perceptions wrong? To investigate whether students understood what plagiarism is, I conducted a survey based on Julio Soto and Elizabeth McGee's study, modified to meet a New Zealand context. Student responses were analyzed using the Statistical Analysis System. The results showed a discrepancy between students' initial confidence and subsequent ability to answer the specific questions. A majority of students (69 percent) rated their understanding of plagiarism as either good or very good. Their answers to the more specific questions, however, showed that their confidence was misplaced (table 1). Although most students could correctly answer simple questions about plagiarism (92 percent correctly identified that including copied text without a citation is plagiarism), they had difficulty with the more complex questions and showed particular confusion over paraphrasing and the distinctions between correct formatting for paraphrasing and quoting. Only 12 percent of students correctly identified that copied material needs to be formatted correctly as well as referenced with a citation. Of those students who felt their understanding of plagiarism was good, only 11 percent answered this question correctly, and only 19 percent of those who said their understanding of plagiarism was very good.
This first-year class, then, was insufficiently prepared to use secondary sources with confidence. They initially overestimated their skills and showed that, while they understood the broad terms, they had insufficient knowledge of the distinctions between paraphrasing and quoting, and of how to acknowledge sources. This confirmed my perception that New Zealand students do not understand the complexities of using secondary sources and what constitutes plagiarism and that education on these matters is imperative.
Turnitin.com is an international website that checks all submitted papers against its ever-growing body of previously submitted papers, as well as commercial databases of journals and periodicals. Once a paper is submitted, it belongs to Turnitin, as the company is popularly known. I used Turnitin on a single assignment, a report on an aspect of science and ethics. The report is designed to be “plagiarism-proof” in that it asks students to link the scientific issue to some specific course material on ethics, and also asks them to address the New Zealand context (which discourages students from uncritically modifying material from international web-sites). Turnitin was contextualized within a detailed educational package. In 2004, students attended a lecture on using secondary sources and American Psychological Association (APA) referencing conventions. They then worked in groups on integrating sources within their own text and using referencing conventions. Activities included interactive exercises designed to illustrate the differences between paraphrasing and quoting. The following week, a peer review exercise on the assignment included questions and discussion on each student's use of sources. Students also had access to paper-based resources on integrating sources and APA referencing. In 2005, we added another support mechanism: brief individual conferences with an adjunct teaching assistant for each student. Questions asked during the conference included, “Do you think you may have used any unacknowledged quotations in your work?” and “Do you understand the conventions of APA referencing—is there anything you would like to discuss about this?” Adjuncts wrote summaries of students' responses, and each student wrote a reply to their comments, explaining how they would change their assignment if needed.
Assignments for both cohorts were processed through Turnitin and the individual reports sorted into four categories: no plagiarism, minor plagiarism (defined as less than six sentences of consecutive or nonconsecutive copied material with no form of in-text citation, or quotations treated as paraphrases, i.e., quoted with an in-text citation), moderate plagiarism (six to eight sentences of consecutive or nonconsecutive copied material with no form of in-text citation), and major plagiarism (nine sentences or more of consecutive or nonconsecutive copied material with no form of in-text citation). Results are detailed in table 2.
Because we did not measure plagiarism rates prior to 2004, we cannot state whether student plagiarism rates decreased due to the use of Turnitin or the educational package—or, indeed, whether rates had changed at all. While other faculty within the trial reported that they believed there were decreased rates of plagiarism as a result of the Turnitin trial, there are no quantitative data to confirm this. However, in my own case, the results suggest that the individual conferences had a substantial impact on the rate of plagiarism detected, effectively halving it across all categories.
Two more specific findings also emerged from these results following an analysis of the conference review sheets in the second part of the project. Students who experience problems with the less severe forms of plagiarism may be exhibiting errors in the academic writing process, rather than misunderstanding how to use conventions. Every student in the 2005 cohort identified as having plagiarism problems had attended an individual conference prior to submission of the assignment. However, all but one had attended with an incomplete assignment draft and had been identified as not having completed in-line citations. Adjuncts discussed the issue with these students, who promised to insert in-line citations. These facts suggest an error not of understanding but of technique. Second, the conference appeared to have a substantial impact on the outcomes for ESL students. Four of the students classified as having major plagiarism problems in 2004 were ESL students, but none of the students in 2005 showing plagiarism problems of any kind were ESL students. Instead, ESL students were more likely to overuse quotations. It is interesting that the introduction of the conference, rather than the introduction of an educational strategy and detection device, proved to be the decisive factor in almost eliminating plagiarism in our (admitted small) sample of ESL students. We were unable to establish, through analysis of the assignments and conference sheets, why the conference was successful in addressing plagiarism among ESL students. Further research is needed.
Reflections of a Writing Teacher
After so many years of working with academic writing, I find it disconcerting that I still misunderstand how students process information and how they write. The study clearly showed that the students who used a “patch writing” technique (Howard) were engaged in a writing process that could lead to accidental plagiarism. Although the workshop included interactive exercises, these did not satisfactorily model the writing process or ensure that students integrated and acknowledged sources as part of their thinking and drafting process. This was a fundamental error in the pedagogy we employed. The lecture itself, by presenting the conventions of citation outside of process, may have misled some students. We also presented sources as something separate from the students' own thinking. And we failed to address adequately the complexities of voice in academic writing, of how to locate one's own voice within the context of academic sources; yes, we touched on this—but such a fundamental, complex, and shifting issue requires more than a thirty-minute exercise. Clearly, some revision of pedagogy is required—and, somehow, more teaching time.
A second key aspect of my reflection has been the impact of the conference on plagiarism. Building a relationship with a reliable mentor may dramatically affect a student's experience of her or his position within the academic community. Jonathan Hall suggests that plagiarism may be partly a result of the depersonalization of tertiary education:
The modern university is big, bustling and impersonal. Students often feel like teachers don't know their names or care about their problems. . . . The plagiarism crisis is not something that dropped out of the sky without our complicity or our participation. . . . If, as the traditional consensus has it, the plagiarist has become cynical, it may be partly because he or she believes we are cynical too. (13)
Individual contact with an adjunct who is available to discuss the student's assignment and personal confusions may be a key difference between the classes of 2004 and 2005. And perhaps it is not surprising that this has an impact on ESL students, who are doubly disenfranchised by language and culture. As a teacher I find this outcome heartening—it suggests a simple and easily implemented solution that we can explain clearly to colleagues and that can easily become part of a broader strategy on plagiarism.
But what of Turnitin.com? Researchers in the field and individual faculty who have used Turnitin tend to have very clear views one way or another: either it is “the best thing we have ever had to combat plagiarism” (personal communication) or it is a reprehensible tool that undermines students' rights, supports the commodification of education, and creates a spirit of distrust and fear (McKeever; Marsh). After my involvement in the trial, I initially held an ambivalent view. In both 2004 and 2005 we found a student who had copied the work of another student from a different tutorial group. Since their work was marked by different tutors, under ordinary circumstances they would not have been caught. As teachers we like to think we can spot plagiarism without additional tools, but this trial convinced me otherwise.
The deterrent effect of plagiarism detection systems remains anecdotal until empirical studies support this claim. Nevertheless, I do think there is evidence that Turnitin can be used as an educational tool. This point is well articulated by another faculty member involved with the trial:
I use Turnitin as an alerting system, but I don't rely on its judgement because I feel the nature of copying, and particularly the level of intent, need case-by-case discerning. During this particular trial, Turnitin helped me to identify one group whose work contained two chunks of material copied from a website without attribution. I don't think I would have picked it up without Turnitin, as it was integrated quite seamlessly. However, my feeling upon reading the context in which the material had been used (to provide background and company profile to a case study) was that it was not deliberate plagiarism but rather a misunderstanding about the best way to provide such background and profile.
I showed the students the Turnitin report, discussed correct referencing requirements with them, and had them explain to me why what they had done could be problematic. I never accused them of plagiarism. . . . However I made very clear to them that if they did this kind of thing again, they could risk being accused of plagiarism, and the penalties were severe. I then had them resubmit the work, and this time it was not only correctly attributed, but they had seen how to put the background they needed into their own words and make it work for their argument. . . . I was really pleased with this outcome. I had the feeling that the students would not copy again, (a) because they knew it could be easily detected (and I think there was a clear deterrent effect on the whole class simply because of the visibility of plagiarism issues due to our in-class discussions of Turnitin and the trial) and (b) because they had worked through the issue of making source materials work for them instead of work against them.
They were happy, and I was happy, and having an independent report like Turnitin helped clarify what was needed and enabled us to have an objective discussion without the students feeling that I, personally, was judging them or accusing them of anything. I think having Turnitin to refer to made the students much less defensive towards me than I have previously experienced when having to deal with plagiarism problems I have detected myself. (Personal communication, October 6, 2004)
This is an example of Turnitin used with subtlety and skill, and would, on its own, incline me toward encouraging the use of Turnitin.com. If Turnitin could be used by teachers committed to teaching academic writing skills, who could (and would) sensitively read the reports, and who understood the distinction between fraud and incorrect or inadequate use of sources, and if this tool was used in conjunction with an effective educational package that addressed process and voice and personal conferences with tutors, then we might make a strong case for Turnitin.
However, there are a lot of subordinate clauses here. We cannot assume that Turnitin will only be used by such instructors—given the level of anger and anxiety around plagiarism, there is plenty of evidence to suggest that it is more likely to be used by those concerned solely with detection and punishment. In such hands, Turnitin becomes a blunt instrument to accuse those struggling to grasp a complex intellectual skill of moral failure—with huge repercussions for those students.
Just as important, in considering Turnitin, are issues of trust and respect. Plagiarism has been characterized as a breakdown in the student-teacher relationship—a student, in plagiarizing, snaps the relationship of trust and respect between herself and her instructor (though the causes of this breach may, of course, be complex). One perspective on Turnitin is that its use constitutes a breaking of that trust relationship by the instructor. If we treat all students as potential cheats, how can they approach us with confidence? A focus on detection and punishment combined with a tool that suggests an instructor's fundamental lack of confidence cannot be conducive to effective learning. Can the detection capabilities of a system such as Turnitin compensate for such a breach in the educative relationship?
So what shifted my ambivalence? Cynthia Hoogland develops Donald Schön's idea that stories are fundamental to the reflective process. She suggests that “stories conjoin emotions and intellect. . . . they are what head-talk becomes when it is joined to the body, or what ideas are fused to lived experience” (216). So let me end with a conceit, a narrative of something I experienced at the Sweetland Writing Center's 2005 Originality, Imitation, and Plagiarism conference:
The trip to the Michigan conference was my first extended visit to the United States. I arrived in Ann Arbor in a cab from Detroit, tired after the long-haul trip, and suddenly aware of an anxious thought: people tip in the United States. I had never tipped in my life—what should I do? The only person I could ask was the cab driver. He said 20 percent was compulsory. Wasn't that rather a lot, I asked. No, ma'am, he said, quite normal. So, I paid the 20 percent and asked the clerk in the hotel lobby whether 20 percent was a compulsory tip. She was not impressed.
Later that day, I headed out for a meal. Before I left, I asked the hotel clerk to explain the rules of tipping. “It's simple,” she said. “You just tip 15 percent for good service in a place where you're served.” “Like a restaurant?” I said. Yes. “What about McDonald's?” No. OK, I thought, slightly more complex than it appears.
As it turned out, over the next few days, tipping became a matter of outlandish proportions for me. Sitting in a Starbucks café I would think, “Is this the equivalent of a restaurant or McDonald's?” Then there was the matter of what “good service” meant. Did it mean “normal, to-be-expected kind of service,” or did it mean “exceptionally helpful and charming kind of service”? Did you tip 15 percent for normal and 20 or more for exceptional? Or 10 percent for normal and 15 percent for exceptional? And what about shops?
The hotel clerk became a vital source of information, greeting me at the end of each day with the wry, amused smile of the native and, “Now, how was the situation today, Professor?”
Some days, finishing a meal at the end of a long day, I would feel so confused that I would shrug my shoulders and walk out without tipping. Other times I just tipped the coins in my wallet (how many?—didn't care) into the jar in a form of mindless overcompensation. Some days, maybe, I got it right.
It is probably not necessary for me to spell out the elements of this extended metaphor, but I will mention just a few. First, the rules in both instances appear to be simple but are surprisingly complex, and implementation of the principles requires extensive cultural experience and the exercise of judgment arising from that experience. Second, to those who work within the cultural context, the rules appear to be clear and the complexities of usage almost invisible. Only the novices flounder and therefore see the issue as significant.
But there is a poignant difference between the two situations. When I failed to come to terms with the complexities of the situation, when I walked away without tipping, there were no repercussions. Despite this, I found the situation confusing and confidence-sapping (I'm used to understanding the basic social rules of modern living—how come I couldn't do this simple thing?). Had I known that moral censure, personal and family shame, and failure to achieve lifelong dreams might ensue from a single error in grasping these culturally determined practices, then the stress would have been considerably higher. I wanted to be able to tip—it wasn't that I didn't care or didn't try. But I sometimes failed out of frustration at my inability to grasp the complexities. If we had added to the situation a device that would unerringly detect every error I made, I would probably have taken the safe route (McDonald's every day?) or tipped excessively at every possible moment. Anything else would have been too difficult.
Originality, Imitation, and Plagiarism: A Cross-Disciplinary Conference on Writing was a valuable professional experience for me. But nothing at the conference taught me so much about students' experiences of learning to use secondary sources in their writing as did the practical experience of learning to tip. I realize that the idea expressed by this conceit is neither original nor revolutionary; I know that a cornerstone of our pedagogy is the idea that learning academic writing is a form of enculturation (Howard; Price). Although I knew this concept intellectually, I had never felt it; I'd never experienced the dilemma from the inside. Through this episode I experienced the plagiarizing student's cultural disorientation.
A significant person in the story is the hotel clerk. At times she was mystified by my confusion but always willing to unravel for me what was second nature to her. Had she been harried and cynical (“Foreigners can never understand the most basic issues”), and judged with the power and inclination to punish my errors, then I would never have learned the basics of this cultural practice. In reversing the role I customarily play, I saw again the attributes of an effective teacher—patience, lightheartedness, interest, a willingness to explore new ideas or revisit old ones, a constant courteousness and respect—and a willingness to laugh at mistakes.
Massey University has made the decision to continue and extend the use of Turnitin.com, and a role has emerged for me in talking to faculty about relevant research and providing a context of educative support. Recently, three tertiary institutions in our city have come together to research a multistrategy approach to teaching information literacy and secondary source usage, and to develop learning tools that may be used widely across tertiary institutions. If we cannot prevent our institutions from using detection systems, then we must become involved in how they use them, and work to provide an educational context to mitigate their effects.
Plagiarism is a complex, multifaceted term that encompasses quite differently motivated behavior. This study has produced some clear results regarding the effect of conferences on plagiarism rates of all students and ESL students in particular. Although Turnitin has potential as an educational tool, we cannot, given the present climate of antagonism regarding plagiarism, have confidence in its being used in this way unless we become involved in how the institution approaches its use. A potentially punitive and insensitive detection tool is unlikely to encourage learning, and the power of detection does not compensate for the breaking of the pedagogical relationship between student and teacher. If Turnitin is here to stay, then as writing teachers we have a task ahead of us.
Thank you to Malcolm Rees, Bruce MacKay, and the participants in the Turnitin trial, particularly Elspeth Tilley and John Walker.
Cameron, Jess. “George Bush Cheats so Why Can't I?” Chaff 70, no. 18 (2004): 16.
Carroll, Jude. A Handbook for Deterring Plagiarism in Higher Education. Oxford: Oxford Centre for Staff and Learning Development, 2002.
Hall, Jonathan. “Plagiarism across the Curriculum: How Academic Communities Can Meet the Challenge of the Undocumented Writer.” Across the Disciplines: Academic Perspectives on Language, Learning, and Academic Writing, 2005. http://wac.colostate.edu/atd/articles/hall2005.cfm, consulted August 30, 2005.
Hoogland, Cynthia. Cuba Journal: Language and Writing. Windsor, ON: Black Moss Press, 2003.
Howard, Rebecca Moore. Standing in the Shadow of Giants: Plagiarists, Authors, Collaborators. Stamford, CT: Ablex, 1999.
Johnson, Andrew and Rosemary Clerehan. “A Rheme of One's Own: How ‘Original’ Do We Expect Students to Be?” Journal of University Teaching and Learning Practice 2, no. 3a (2005): 37–47.
Larkham, P. J., and S. Manns. “Plagiarism and Its Treatment in Higher Education.” Journal of Further and Higher Education 26, no. 4 (2002): 341–49.
Macdonald, Ranald, and Jude Carroll. “Plagiarism—a Complex Issue Requiring a Holistic Institutional Approach.” Assessment and Evaluation in Higher Education 31, no. 2 (2006): 233–45.
Marsh, Bill. “Turnitin.com and the Scriptural Enterprise of Plagiarism Detection.” Computers and Composition 21, no. 4 (2004): 427–38.
McKeever, Lucy. “Online Plagiarism Detection Services—Saviour or Scourge?” Assessment and Evaluation in Higher Education 31, no. 2 (2006): 155–66.
Price, Margaret. “Beyond ‘Gotcha!’: Situating Plagiarism in Policy and Pedagogy.” College Composition and Communication 54, no. 1 (2002): 88–114.
Schön, Donald. The Reflective Practitioner: How Professionals Think in Action. London: Arena, 1991.
Soto, Julio G., and Elizabeth McGee. “Plagiarism Avoidance.” Journal of College Science Teaching 33, no. 7 (2004): 42–48.
Whitehead, Jack, and Jean McNiff. Action Research: Living Theory. London: Sage, 2005.
Winter, Richard, Alyson Buck, and Paula Sobiechowska. Professional Experience and the Investigative Imagination: The Art of Reflective Writing. London: Routledge, 1999.