American Annals of the Deaf

Abstract: The study examined how literacy portfolios were used as tools in a college developmental English class in which deaf students assessed their reading comprehension as well as their writing processes and products. The students' reading and writing assignments involved reflective thinking and were grounded in authentic tasks. Immediate feedback was provided. The study was multidimensional, longitudinal, and ongoing. A variety of field research techniques were used to ascertain the uses and influences of portfolios in regard to students' reading, writing, and reflective thinking. The results support the idea that the use of literacy portfolios can positively influence students who are deaf when they assess their reading and writing abilities.

Literacy portfolios were used in a developmental English class for deaf students at Gallaudet University during 1 academic year. The students in this class focused on ways in which they could assess their own reading and writing abilities. During this self-assessment process, students learned about strengths and weaknesses related to their reading and writing abilities, which enabled them to become actively engaged learners. For this research study, current assessment practices were examined for the purpose of providing a theoretical foundation.

Current Assessment Practices

Several of the assessment strategies used in the class were based on current theory that focuses on assessing students' performance on tasks and the processes students go through while they perform those tasks. Current research suggests that students and instructors should examine several assessment criteria before they assess student performance.

Wolf, Bixby, Glenn, and Gardner (1991) focus on four criteria that are necessary ingredients of student assessment: First, assessment should be multidimensional and longitudinal. Students should be assessed on the basis of a variety of products, such as written essays, interviews, reading logs, papers written in reaction to reading selections, and research projects. Assessment should occur over a long period in order to document growth. Second, standardized measures can be complemented and supported by tasks students do in their classes. This gives students the opportunity to reflect on their own quality of work as well as to [End Page 31] discuss standards that are used to judge good work. Third, assessment should include feedback to students from their peers and instructors. Immediate feedback enables students to analyze what they have just completed. Fourth, assessment should help instructors engage in examining student work and encourage instructors to think about what they have taught and how they might change it.

Valencia, McGinley, and Pearson (1990) also established criteria to use when investigating assessment practices; in application, these extend the criteria established by Wolf et al. (1991). According to Valencia et al., the assessment process should include a time for students and teachers to collaborate about their work; also, assessment "must be grounded in knowledge" (p. 127). Teachers should be knowledgeable about what they are assessing so that they can prompt and guide their students, as well as ask them questions that promote thinking and learning. Valencia et al. further explain that assessment should be continuous. It can occur in a variety of situations and can happen all the time. Ongoing assessment need not be formal; it can be done through routine teacher observations and daily activities.

Assessment should also be based on authentic tasks such as writing letters to public officials or the authors of books students have read, following directions when reading maps, or solving problems the students themselves have researched. Self-assessment is included in authentic assessment because students are analyzing what they have experienced and learned (Wiggins, 1989).

Self-reflection is an important component of assessment that helps students become more independent learners. Students engage in metacognition when they reflect on their own learning and become aware of the cognitive processes they use in their own reading and writing. Self-reflection enables them to think about their own knowledge and develop control over their own thinking processes (Baker & Brown, 1984; Flavell, 1981). Reflection played an important role in writing portfolios created by deaf students in college English classes at the National Technical Institute for the Deaf. Students in those classes became better at assessing and reflecting on their strengths and weaknesses as writers (Albertini, 1994).

For the present research study, I defined assessment as multidimensional, longitudinal, and ongoing. Students developed and maintained literacy portfolios, which enabled them to focus on the tasks they were working on for their class. Students received immediate feedback on their work from their peers and instructor. They were also asked to reflect on many of the tasks they had completed so that they could make decisions about how they could become better readers and writers.

The definition of assessment used in the present study is important to consider when teachers and students develop assessment activities. The criteria support the idea of having students create and maintain portfolios in their classes. Portfolios can enable students to collect a variety of their work over time, learn to self-reflect, collaborate with classmates and teachers, and view assessment as an ongoing process that can be grounded in authentic tasks they complete for their classes. Ongoing assessment activities that help students develop their portfolios should enable students and teachers to see the connection between instruction and assessment. Using ideas from current assessment practice, I defined literacy portfolios as purposeful collections of the products of students' reading and writing tasks that show the students' efforts, progress, and achievements during an academic year. In addition, the students included evidence of self-assessment and helped select their portfolio items. The literacy portfolios also provided a forum where the students could develop their ability to become more independent learners (Nickerson, 1996).

Participants

Nine students and their English instructor participated in the present study. Each of these students and the instructor signed informed consent forms so that I could conduct research in their class. The instructor, Ms. Smith (not her real name), had 11 years' experience teaching developmental English to deaf college students. The nine students ranged in age from 18 to 22 years and were prelingually, profoundly deaf. In being required to take developmental English before entering freshman-level English courses, these students were in the same situation as more than half of all students admitted to Gallaudet University.

Four of the students' performances related to the use of literacy portfolios was examined in depth by means of a case study approach. These four students--Gina, Maria, Matt, and Roger (not their real names)--possessed characteristics found in many college students who are deaf and therefore were considered to be representative of the students in this class. These students were similar to other students in developmental English courses at Gallaudet in regard to their reading and writing abilities; that is, they represented a range of abilities in reading and writing. Their motivational beliefs likewise varied. In addition, they reflected the population of students at this school based on several characteristics. Two were male and two were female; two were African American and two were White. Also, two of the four students had developed literacy portfolios in previous classes, while the other two had not.

One of the four students, Matt, was the best in the class, was intrinsically motivated to improve his abilities, participated frequently in class, and had above-average reading and writing abilities relative to his classmates. Two of the four, Gina and Roger, had average reading and writing abilities relative to the rest of the class. One of these two average students, Gina, was intrinsically motivated to improve her [End Page 32] reading and writing abilities and was an active participant in class discussions; the other, Roger, who had developed literacy portfolios in previous classes, was also intrinsically motivated, as he believed strongly that the portfolios had helped him improve his reading, writing, and self-assessment abilities. Maria, the fourth student whose work was analyzed in depth, was among those in the class at a lower level of reading and writing ability. This student was more extrinsically motivated (i.e., by grades and teacher praise) than the other three and seemed to do the assignments for her literacy portfolio because she felt she had to instead of out of a desire to improve her own abilities.

Types of Data

All of the nine students in the class agreed to develop and maintain literacy portfolios as a means of focusing on their reading and writing abilities. The students allowed me to make copies of the items in their literacy portfolios, which proved to be a valuable source of data. They also agreed that I could observe their English class and take field notes throughout the academic year to see firsthand what they were doing related to self-assessment. Some of these classroom observations were videotaped and transcribed at a later date. In addition, the students allowed me to ask questions about their attitudes toward reading and writing as well as their literacy portfolios during the three interviews I conducted with each student. I also asked the students to complete reading and writing surveys at three different times during the academic year.

The array of data sources used in the present study enabled me to determine how literacy portfolios helped these students assess their reading and writing abilities. The students' stories were documented to show what progress, if any, they made in relation to their literacy portfolios and self-assessment of their reading and writing abilities.

Throughout the process of developing each student's self-assessment story, I triangulated the data in order to discern and consider convergent and divergent information. So that I might track the development of self-assessment ability, I examined the items in the students' literacy portfolios. The portfolios were the richest source of data on each student; they were complemented by some of the data from the surveys, interviews, transcriptions of videotaped classes, and research field notes. Information that was divergent or showed contradictions with what the students reported on their surveys or during their interviews was examined with the goal of gaining a more thorough understanding of each student's progress.

For the present study, issues regarding validity were examined. Goetz and LeCompte (1984) suggest that validity is a potential strength of case studies because the researcher collects data over long periods and from many different sources. The present study included data that were collected during a single academic year. It also included data from sources such as interviews, surveys, field notes, transcripts of videotaped classes, and artifacts from the literacy portfolios.

Generalizability, or external validity, was also examined in the present study. Constant comparative analysis was used with the collected data (Glaser & Strauss, 1967). As the data were analyzed, descriptions of typical events and patterns emerged (Erickson, 1986). These patterns were then compared. Erickson explains that interpretive research should be generalizable within the study itself because of the richness of description and the interpretive analysis of the data, which focus on patterns that occur across both frequent and rare events. In addition, the ideas generated from the present study might be typical of other classrooms for deaf college students that use literacy portfolios to help them engage in self-assessment.

Reliability was also examined in the present study. Complete descriptions of the participants, as well as the materials and strategies used during data collection, are available, as well as descriptions of how the field notes were recorded and how the interviews and surveys were administered. Description of the data analysis and (retrospective accounts of how data were examined and synthesized (Goetz & LeCompte, 1984, p. 217) can also be obtained from me (Jane.Nickerson@Gallaudet.edu). These descriptions provide details that will allow the present study to be replicated.

Results

In this section I focus in particular on the ways the students in the present study perceived that the use of literacy portfolios could enable them to become better readers and writers.

The literacy portfolios the students created for the developmental college English class incorporated many of the same ideas that other students, teachers, and researchers had encountered while using portfolios (Au, 1994; Valencia & Place, 1994). The students in the present study were instrumental in developing ways to assess work they did for this class. The literacy portfolios helped them document their growth in reading and writing because they collected items for their portfolios that reflected what they did in class. Each literacy portfolio contained certain core components. These included sections for reading logs, reading assignments, surveys and interviews that focused on attitudes toward reading and writing, and writing assignments.

Reading Logs

Each of the students in the class was asked to keep three reading logs and assess them during the year. These reading logs helped students focus on two important aspects of the reading habit: the materials they read and the amount of time they spent reading. After keeping records for 2 weeks, the [End Page 33] students analyzed each of their reading logs. The first reading log had an impact on all of the students because they had never before kept track of how long they read and what they read. For Reading Log I, students on the average indicated that they read between 15 and 30 minutes each day, except for Roger, who reported reading 1 hour daily. Many of the students expressed surprise at how little they actually read each day. Many also indicated that they read only a limited variety of materials. All of the students read what their teachers assigned for class, and only about half of them indicated that they read for pleasure. All of the students but one explained that they would like to increase the amount of time they read. Most of the students understood the benefits of the reading logs and felt that it was important for them to keep track of how long they read and what they read.

Maria reflected on her reading habits after she finished completing and analyzing her first reading log. In an essay about her reading habits, she wrote:

I really love to read books and I'm in the habit of reading now. I had to force myself to read books but inside, I was saying, "I don't feel like reading a book." I know it's important for my future. So I read a lot. It helps me pull my reading level up. In the future I want to be more skilled in reading. Also, I want to read more difficult books so I can enjoy them like my mom does. [from Maria's essay titled "Past, Present, and Future Reading Habits," draft 3]

This passage suggests that Maria was thinking about her reading habits and how to improve them. She indicated that she enjoyed reading, which was a big first step for her.

When Matt was interviewed in April, he explained that he would continue to "keep track of what I read on my Reading Log" in the future, as he felt that the reading logs helped him focus on his reading ability. In his interview, Matt reflected on how the reading logs had helped him:

They helped me because I wrote down what I read and I saw how I improved. I wrote down how much time I read. I increased my reading time a little. I'm reading a wider variety of materials now too. I'm reading Junior Scholastic, newspapers, and The Chamber by John Grisham.

In contrast to the other students in the class, Matt explained that each of the reading logs helped him, whereas all of the other students expressed the idea that Reading Log I was the most important because they had never before kept track of what they read. Reading Log I had the most impact on all of the students, including Matt, because they were becoming more aware of their reading habits. Many of them were determined to make some changes in order to increase the amount of time they read and to read a wider variety of materials. If students developed better reading habits, they would also increase their background knowledge of topics they were reading about for class and pleasure, which is an area where many students who are deaf need improvement.

Reading Assignments

The second section of the literacy portfolios contained reading assignments, reaction papers, and letters. The reaction papers were written in response to various articles the students had read; they thought about their own reactions or other feelings and wrote them down. During this process, the students assessed their reading ability by asking themselves if they understood the article and assessed their writing ability after they wrote their reactions by making sure their ideas were clearly written and organized. One of the more emotional articles the students read was "What Children Need Most Is Adults Who Care," by Michael Ryan, which appeared in the popular Sunday supplement magazine Parade in 1994. In the article, Ryan described the lives of several children who had been shot and killed. After reading the article, Ms. Smith led a class discussion during which Matt expressed his concerns and other feelings about children being shot by other children. Matt reflected on his own life and discussed his experience of seeing someone being shot. Then he wrote about this experience and his own fear of being shot in his reaction paper:

[Children being shot] makes me feel bad because I'm a teenager. It's sad to hear that children die from gun violence. Also, it's scary for me because it happens to children every day, and I can be one of them. That is what worries me the most. I have a friend who shot his brother in front of me. I was only 13 years old at that time. I was shocked, and I couldn't do anything right after the boy was shot.

The Parade article had an impact on Matt because in it he read about several people's experiences that were similar to his. Matt was able to reflect on what he read and relate these ideas to his own experience. His reaction paper was one of the most powerful entries in his literacy portfolio because it was so clearly evident that it had helped him think about a terrible event in his life, express his feelings about what happened, and express his fears about being shot.

All of the students wrote emotional reactions to the Parade article (Ryan, 1994), and many of them expressed their feelings about gun control after they read this article along with several others presenting both sides of the gun control issue. This type of paper made each literacy portfolio personal because the students were reflecting on and writing about their own lives. The students were also motivated because many of them had personally experienced events like the one described in the Parade article. [End Page 34]

Roger wrote a reaction paper after reading "Success in a World of Silence," another Parade article (Anderson, 1988). The article portrayed John Yeh, a deaf man who had set up a multimillion-dollar computer software company. Reading about a deaf man who achieved success after many years of hard work was inspirational to the students. For instance, in his reaction paper, Roger wrote:

My opinion of this story is that it makes me feel good. All of this time, most people think deaf people cannot do anything but we can prove to them [that they are] wrong. John Yeh worked [hard to] prove that he can succeed and be able to take responsibility for his own actions. Look, he made it[,] so all of us, we can make it too. I [learned] from John Yeh [to] just keep fighting for what you really believe in, and you will aim high [in] life. Be more assertive, not passive.

Roger's reaction paper showed that he was thinking about how the experiences of another deaf person could help pave the way for success in his own future endeavors.

Students also placed letters in the reading assignments section of their portfolios. One was a letter to the editor of the Washington Post concerning several articles that had appeared in the Post about Heather Whitestone, the first deaf Miss America. The class discussed the articles and then collaborated to write a single letter to the editor to explain that while they were proud that Whitestone had been crowned Miss America, they were upset that she did not use sign language as her primary mode of communication and that she felt that sign language limits deaf people. The students worked cooperatively to write the letter and assess the ideas they wrote to make sure they were clearly stated. The students felt very proud of themselves when they completed the project, even though the letter was not published. Individually, the students wrote and sent letters to Ray Bradbury after they read several of his short stories, and wrote fictional letters to Madame D, one of the characters in the Edgar Allan Poe short story "The Purloined Letter."

Reading and Writing Attitudes

The third section of each student's portfolio contained a survey of his or her attitudes toward reading and writing. On the survey form, titled "Assessing Your Reading and Writing Abilities," students were asked to describe their strengths as readers and writers. (They provided further self-assessment on this subject later, in our interviews.) The surveys enabled the students to see how their attitudes toward reading and writing varied during the year. They also helped the students think about their attitudes toward reading and writing more generally, which many had not done before taking the developmental English class. I conducted interviews with each student three times: September and December (i.e., the beginning and end of the first semester), and April (i.e., the end of the second semester). When the students compared the answers on their various surveys, many of them realized that their attitudes had become more positive toward reading and writing, and they were elated about this.

The students also learned how to express their strengths and weaknesses related to their reading and writing abilities. Gina wrote:

My strengths in reading are that I do understand a lot. I tend to get a picture of what I read in my mind. I read a sentence and imagine what it means. My weakness in reading is that I don't bother to get definitions for hard words. Also, I don't understand some of what I read.

My strengths in writing are brainstorming and expressing my feelings, whether they are positive or negative. Also, I summarize well after I read. My weakness as a writer is writing an outline of an essay. My English grammar could make me confused as to whether I've written something correctly in English. Also, I'm aware of the organization of an essay such as the introduction, body, and conclusion too.

Gina was a student who judged herself harshly and wanted to improve fast; however, she was beginning to realize that reading and writing abilities do not improve quickly. This realization was important for students like Gina because they needed to understand that practice over time could help them improve their reading and writing abilities. Gina's answers on her surveys indicated that she was spending more time reading and writing at the end of the academic year than at the beginning.

The initial interviews, the ones I conducted in September, served as a baseline; they indicated that the students did not have much knowledge about self-assessment before the class started. The final interviews, in April, as the academic year neared its end, revealed many of the beliefs the students had about their reading and writing abilities, and provided insight into how well they were able to self-assess and to maintain their literacy portfolios. In the course of explaining how they had developed their portfolios, all of the students expressed the view that self-assessment had helped them become aware of their own learning.

In the interviews, I asked the students if they enjoyed developing their literacy portfolios. Matt responded that he enjoyed using a portfolio in his class

because I can look at my writing carefully and see the progress I've made. If my essay is lousy, then I can look through my portfolio to find other essays to compare it to and look at them. I can organize my papers. [End Page 35]

Maria was the only student who expressed dislike of the tasks of creating and maintaining a portfolio; however, she saw the benefit of having one. She explained this during her interview:

My portfolio was a lot of work. I used it during my senior year of high school and I was hoping college didn't have it. The assessment part helped my thinking. The assessment papers help me compare my work from fall to now. It's hard to find all of my papers and organize myself.

Maria felt that maintaining her portfolio was too much work for her, but she did complete all of the work for it. Maria understood why she was creating her literacy portfolio, and she knew that she might benefit from doing the work.

The interviews captured much of what the students thought about the process of developing their literacy portfolios. Except for Maria, all of the students enjoyed maintaining their portfolios. They realized that many of the items they included in their portfolios showed the progress they had made throughout the year. Most of them also developed more positive feelings about their reading and writing abilities as a result of maintaining the portfolios.

Writing Assignments

The last section in the students' literacy portfolios included writing assignments such as autobiographies and narrative and opinion essays, as well as content and organization assessment forms for some of the essays, a description of a character from a novel read for class, and an extra chapter for the novel written as a creative writing assignment. Students also placed preliminary drafts of these works in this section. The students were asked to assess many of their writing assignments for content, organization, and grammar. They wrote several drafts of each assignment, which enabled them to understand the writing process more clearly. Students were often able to see their own progress when they self-assessed. For instance, in one of her interviews, Gina said that she looked at her

first and last drafts often so I could look at my improvements. [When I evaluated] my essays, I focused on my background information in my introduction. Sometimes I would revise that. I used the content assessment form to help me narrow in on my topic. Draft 1 is very general and draft 2 is more specific. I looked at the form to see how I improved. The organization assessment form helped me think about transition words such as first, second, and in addition. I also made sure I had a topic sentence and a conclusion.

Gina's comments were similar to those of many other students in the class. In general, they did not enjoy having to assess their essays and then write several additional drafts, but the succession of drafts gave them vivid proof of their progress and helped them develop confidence as writers.

In summary, the four students whose comments are provided in the present study improved their reading and writing abilities as a result of developing their literacy portfolios and engaging in self-assessment activities. Further, I believe that every student in the class benefited from the use of literacy portfolios.

Discussion:
Benefits of Literacy Portfolios

The results of the present study indicate that the students in the developmental English class improved their reading and writing abilities in many areas. The assignments completed for the literacy portfolios positively influenced the students in several ways: They learned to assess their reading and writing abilities, they documented learning activities that occurred in class, and they became more reflective learners.

Students Assessed Their Reading and Writing Abilities

One major benefit of using literacy portfolios was that the students learned to assess their reading and writing abilities (Baker, 1993; Moss, Beck, Ebbs, Matson, Muchmore, Steele, Taylor, & Herter, 1992; Valencia & Place, 1994). The data on the four students described in the present study support the idea that the assignments they completed for their literacy portfolios enabled them to improve their reading and writing abilities after they learned to assess those abilities. As readers, the students strengthened their ability to integrate new information with their prior knowledge on various topics, they learned techniques to determine the meanings of new words by using contextual clues, and they learned strategies to help them comprehend text more fully than before. The students also became more enthusiastic and confident about their reading ability in general. As writers, they increased their knowledge of various topics, which enabled them to come up with more ideas for their written work, they developed greater confidence, and they learned to assess the content and organization of their writing. The use of literacy portfolios helped the students become engaged learners as they assessed their reading and writing abilities.

Students Documented Their Learning

The students documented some of their learning activities in their literacy portfolios. They became aware of two aspects of their reading habits when they documented what they read and how much time they spent reading for their reading logs, as other researchers have suggested occurs (Au, 1994; Valencia, 1990; Valencia & Place, 1994). As a direct result of documenting their own reading habits and then assessing them, many of the students became more motivated to read, and to read a wider variety of materials.

The students also documented many of their ideas related to self-assessment [End Page 36] when they assessed their reading and writing assignments. For reading, the students focused on comprehending challenging materials and then writing reaction papers about some of these texts. For writing, the students focused on writing summaries and essays which they often assessed using content and organization forms. These are checklists and questions designed to help students focus on ideas and organizational structures. They also documented their attitudes toward reading and writing when they completed the surveys and interviews. These activities enabled the students to document their learning situations and then reflect on their learning.

Students Reflected on Their Learning

The students who participated in the present study reflected on their own learning (Camp, 1990; Raines, 1996; Tierney, Carter, & Desai, 1991). During the year, many of them changed their attitudes toward reading and writing, and, as a result, they became motivated to read and write more. Many of the students started the class with negative feelings about their reading and writing abilities, which were expressed in their comments in class and during their interviews when they stated that they did not read much or write well. As a result of many of the assignments the students completed for their literacy portfolios, they began to develop more positive feelings about their abilities and about reading and writing in general. Motivation played a role in the fact that many of the students read more for school and pleasure. Their newly developed confidence as readers and writers helped them become more motivated to take risks when they read challenging material and subsequently wrote about topics raised in this material. The students also discussed topics in class, which enabled them to think of many perspectives on various issues and become better motivated to learn more about these topics.

One limitation of the present study is that it represents only four deaf students who created and maintained literacy portfolios for 1 academic year. It is possible that students in other English classes for the Deaf would not benefit as much as the ones in this study. It should be noted, however, that the students in this study were similar to many other deaf students who take developmental college English classes.

Summary

The present study proved that deaf students were positively influenced by the use of literacy portfolios, which enabled them to assess their reading and writing abilities. The use of literacy portfolios provided three benefits to the students in the developmental college English class that was the setting for the study: They learned to assess their reading and writing abilities in order to improve, they documented some of their learning activities, and they became more reflective learners. These benefits were valuable to the students because they helped them became more responsible and more independent learners.