• A Quasi-experimental Evaluation of an Early Literacy Program at the Regina Public Library / Évaluation quasi-expérimentale d’un programme d’alphabétisation des jeunes enfants à la bibliothèque publique de Regina
Abstract

Public libraries play an important role in the provision of free, interactive, community-based learning opportunities that build the capacity of parents/care-givers to support the development of their children’s early literacy skills. Despite public library practice of delivering early literacy programs, questions about program impact on how parents/caregivers support the development of early literacy for their children are largely under-researched in library and evaluation literature. This article presents the research design, analysis, and results of a quasi-experimental approach to evaluating the Regina Public Library’s Mainly Mother Goose program for children under 24 months and their parents/caregivers. Key lessons about the research approach are discussed and future research directions for early literacy programs in public libraries are identified.

Résumé

Les bibliothèques publiques jouent un rôle important dans la prestation de services d’apprentissage gratuits, interactifs et à base communautaire, renforçant les capacités des parents et tuteurs à soutenir le développement des premières compétences de leurs enfants en lecture-écriture. En dépit du fait que les bibliothèques publiques offrent ces programmes d’alphabétisation précoce, les questions concernant leur impact sur la façon dont les parents et tuteurs soutiennent l’alphabétisation précoce de leurs enfants demeurent pour une grande part sous-étudiées dans les recherches sur les bibliothèques et dans la littérature d’évaluation. Cet article présente la conception de la recherche, de l’analyse et les résultats d’une approche quasi-expérimentale de l’évaluation du programme Mother Goose, à la bibliothèque publique de Regina, destiné aux enfants de moins de 24 mois et à leurs parents et tuteurs. Les principaux enseignements concernant cette démarche de recherche sont discutés [End Page 103] et des directions de recherche pour les programmes d’alphabétisation des jeunes dans les bibliothèques publiques sont identifiées pour l’avenir.

Keywords

early literacy programs, public libraries, quasi-experimental evaluation, library programs for babies

Keywords

programmes d’alphabétisation précoce, bibliothèques publiques, évaluation quasi-expérimentale, programmes des bibliothèques pour les très jeunes enfants

Introduction

Since the early 1980s, Canadian public libraries have been offering programs designed for babies from birth to 24 months. The programs are filled with songs, rhymes, finger plays, bounces, and books. In the early days, the programs were primarily directed at babies and secondarily at parents. Over time, however, the programs have evolved and many of the programs for babies offered in public libraries are now targeting both the parent and the child. Today, for many public libraries, one of the aims of these parent-child interactive programs is to build parents’/caregivers’ confidence and competence to support their young children as they develop early literacy skills. That current focus of a parent/child interactive program is the framework for the programs for babies offered by the Regina Public Library (RPL).

RPL serves a population of almost 200,000 people. In addition to the Central library, there are eight branch locations across the city. In 2012, the library circulated 2,200,200 items and offered 4,252 programs for adults, teens, and children to a total audience of 99,000 people. Create Young Readers: Early Literacy is one of the four service responses identified in the library’s Service Plan. The library’s goal in meeting this service objective is to ensure that children from birth to age five and their parents/caregivers have resources, programs, and services to ensure that every child is ready to read. Mainly Mother Goose (MMG), an early literacy program for parents/caregivers and babies from birth to 24 months, is one of the programs offered at most RPL library locations to meet this goal. In 2012, the library offered 203 Mainly Mother Goose programs to a total audience of 5,668 children and parents.

In 2011, the RPL received funding from an anonymous donor to evaluate the impact of its MMG program. The overarching purpose of the evaluation was to understand how the MMG program may contribute to parents’/care-givers’ engagement in the development of their child’s early literacy skills.

This paper presents the findings of the evaluation of the RPL MMG program. The paper is divided into five sections. In the first section, a literature review on the evaluation of early literacy programs in public libraries is presented. A description of the program is provided in the second section, which outlines the aims, content, and mode of implementing the MMG program through the RPL system. In the third section, the methodological approach to evaluating the impact of the MMG program is presented, including the guiding questions, hypotheses, data collection, and analytical methods. The results are presented in the fourth section, which is followed by a discussion of the findings. In the last section, lessons learned are considered and future directions for [End Page 104] conceptualizing and evaluating early literacy programs offered by public libraries are identified.

Review of literature

While contemporary research has proven that early literacy programs can play a pivotal role in preparing children to succeed in school by contributing to school readiness, the impact of early literacy programs in public libraries is largely under-researched. Although children’s librarians claim that young children bene-fit from early literacy programs by allowing them to develop early literacy, communication, and social skills that are needed before they start school, there is a lack of evidence to support such beliefs. Evaluation tools, such as the one developed by Diamant-Cohen (2006), are available to help libraries evaluate their early literacy programs, but few libraries collect data to try to determine ways in which early literacy programs may affect families. Two recent studies confirmed the lack of research measuring the impact of early literacy programs.

MacLean (2008) ascertains that little has been written on the impact of public library pre-school storytime programs. In her review of the literature on public library pre-school storytime programs for children three to five years old, she found only five studies. Of the studies reviewed, McLean indicated that only one (Fehrenbach, Hurford, and Fehrenbach 1998) addressed the measurable impact of pre-school storytime programs. Two studies (Laughlin 2003; Martinez 2007) addressed the impact on the literacy behaviours of parents and librarians who had participated in an early literacy training program. The last two studies (Stone 1999; McGill 2003) identified the varying opinions that parents and teachers have on the impact library programs have on the development of early literacy skills in children. A 2009 study by the Canadian Language and Literacy Research Network also confirms that there is a lack of systematic research that evaluates the impact and effectiveness of early literacy programs.

This lack of research means that public libraries are limited in their capacity to use existing evidence or develop new evidence that enables them to ascertain whether early literacy programs, such as the Mainly Mother Goose program for infants and their parents/caregivers, contribute to learning outcomes for participants or whether the programs improve the quality of parental engagement in early literacy activities in the home. Some recent and forthcoming studies may provide the rationale for the library’s claims that early literacy programs help very young children take the first steps to literacy.

Four recent studies on early literacy programs conducted by R. Stewart (2010a, 2010b, 2011a, 2011b) for the Idaho Commission for Libraries provide valuable data on the impact of a library’s early literacy programs. Some of the studies are of particular interest as they address the long-term impact of some public libraries’ early literacy programs on children’s early literacy skills. The findings of the studies indicate that as a result of attending Every Child Ready to Read (ECRTR) family workshops, participants report changing their behaviour and that the positive behaviour changes continued long after participation in early literacy programs (Stewart 2010b, 2011a, 2011b). The fourth study (Stewart [End Page 105] 2010a) is a case study of four early literacy programs aimed at determining if children’s participation in early literacy programs influenced their Idaho Reading Indicator scores.

A Canadian study conducted by the Ontario Institute for Studies in Education (University of Toronto) provides some data on the value of pre-school literacy programs offered by public libraries. The Federation of Ontario Public Libraries commissioned an outcomes-measurement study of pre-school literacy programs in 10 public libraries to demonstrate the impact of pre-school programs. The study examines the impact of pre-school literacy programs on the six early literacy skills identified in the increasingly popular early literacy concept for libraries, Every Child Ready to Read @ Your Library (print awareness, letter knowledge, phonological awareness, vocabulary, narrative skills, and print motivation). The aim of the study is to provide evidence of the ways in which early literacy programs facilitate participating children’s early literacy development and school readiness, and how they influence family interactions supporting children’s early learning. Some of the preliminary findings indicate that a high percentage of behaviours observed in children demonstrate their readiness for school as well as their motivation to read. The results of the study provide much needed data on how well pre-schoolers who attend early literacy programs at the library are ready to learn to read when they start school.

The Federation of Ontario Public Libraries’ study and the one presented herein both aim to contribute to the development of an evidence base for advancing the dialogue about how to approach impact assessment studies of early literacy programs offered by public libraries.

Regina Public Library’s Mainly Mother Goose program

To see how Regina fits within the broader Canadian picture, it is useful to look at the findings from the national Understanding the Early Years (UEY) project,1 which indicates that some Regina children are falling behind in five areas of early childhood development when compared with the Canadian average. As a whole, children in Regina scored significantly lower than their Canadian peers in all five major areas of development (Table 1).

Table 1. Mean scores by major area (0–10 scale)
Click for larger view
View full resolution
Table 1.

Mean scores by major area (0–10 scale)

[End Page 106]

The analysis in above noted research bulletin by Understanding the Early Years (2009) also shows that “one third of Regina’s children are behind in at least one aspect of their development at kindergarten entry.”

The MMG program directly addresses early literacy issues in Regina by providing parents/caregivers access to opportunities where they can learn how to support their child(ren) in the development of early literacy skills. The parent/caregiver-child interactive program for babies from birth to 24 months is offered at most branch locations in the winter, spring, and fall. Each session, which runs for approximately 8 to 10 weeks, is half an hour in length. The parents/care-givers and their babies who participate in the program are from all neighbour-hood areas of Regina, ranging from financially secure to low-income families. Using a variety of developmentally appropriate activities such as songs, rhymes, finger play, bounces, and stories, the programmers demonstrate and model the early literacy components of the Every Child Ready to Read @ your Library program that have been identified as precursors to reading skills. Repetition is used in the weekly classes to provide parents/caregivers with time to learn the words and melodies of the songs and rhymes. Each MMG class includes these common elements but MMG programmers have pedagogical freedom to format each class to respond to participants’ unique learning styles and overall mood on the given day.

The MMG program aims to help parents/caregivers develop their children’s pre-literacy skills in a fun-filled and joyful environment. Parents/caregivers learn to play with their child and talk to them in a very interactive manner. They learn techniques to encourage their young children in literacy activities. The program sets in motion a lifelong love of learning, reading, and libraries for young children. The program also sets a strong foundation for early development and early learning.

Research questions and methodology

The MMG program presented some unique research-design challenges. First, the program was divided into sessions and each session divided into 8 to 10 classes across a three-month time period. Second, each session took place on a weekly basis and was delivered in 30-minute-class increments for a total of five hours of instruction per session. Third, the program was delivered in 13 different sites which included both library branches and outreach sites. In addition, there were multiple variables intersecting the lives of parents/caregivers in between their involvement in the program, including participation in other programs, visits with families and friends, and visits to health professionals and other daily errands, where learning that impacts early literacy development could occur.

All research questions relate to the participation of, and potential learning outcomes achieved by, parents/caregivers. The study did not examine changes in early literacy skill development of participating children. Four questions guided this study and two of these questions included a related null hypothesis that was assessed through statistical significance testing.2 [End Page 107]

Research questions with null hypotheses

Question 1: Do parents report an increased use of the following nine early literacy skill development activities after their participation in the MMG program?

  1. a. talking to child about what is going on around

  2. b. singing songs to child

  3. c. reading to child

  4. d. pointing out letters on objects such as toys or food boxes

  5. e. saying nursery rhymes to child

  6. f. helping child learn new words by talking and reading

  7. g. helping child see and feel different shapes

  8. h. h. listening and answering when child babbles or talks

  9. i. asking child questions

Null hypothesis 1: Parents/caregivers do not report an increased rate of using any of the nine early literacy skill development activities with their children after participating in the MMG program.

Question 2: Do parents/caregivers report increased number of library visits (excluding visits to library for the MMG program) after participating in the MMG program?

Null hypothesis 2: Parents/caregivers do not report an increased rate of visiting the library after participating in the MMG program.

Research questions without null hypotheses

Question 3: Do parents/caregivers report an increased sense of confidence and competence in using story-time materials and activities after participating in the MMG program?

Question 4: Do parents/caregivers use what they learned in the MMG program at home?

Scope of evaluation study

This study was concerned with the delivery and impact of 24 MMG programs, 12 in the winter and 12 in the spring of 2011. The following six branches delivered the MMG program in the winter and spring of 2011, with some branches delivering two MMG programs concurrently: Central Children, Connaught, George Bothwell, Regent Place, Sherewood Village, and Sunrise. The following outreach sites also participated in the study: Al Ritchie Family Wellness Centre, Balfour High School, Rainbow Youth Centre, and Regina Open Door Society.

Women were far more likely to participate in the MMG program evaluation than men. In the winter session, 96.3% of the parents/caregivers who completed questionnaires were women. Similarly in the spring session, 92.3% of the parents/caregivers who completed questionnaires were women. All follow-up interview parents/caregivers were women.

Though participating parents/caregivers were more likely to be women, they reported that many fathers were also actively engaged in early literacy activities with their children at home. Almost one third of parents/caregivers reported that [End Page 108] fathers were among those who read most to their children at home (30.6% reported by winter parents/caregivers, 28.1% reported by spring parents/caregivers). The gender of participating children was evenly split between males and females.

Data collection methods

A quasi-experimental design was employed for the study because this method is efficient in longitudinal research that involves longer time periods which can be followed up in different environments (Cook & Campbell 1979). Quasi-experimental design was also selected because it does not require a test control group, allows for research of changes in the same group over time, and can accommodate factors being studied over which one has little or no control.

A three-part data collection method was used to gather information about potential changes in parent/caregiver confidence and competence in supporting early literacy skill development in their children. Informed voluntary consent was obtained from all parents/caregivers through a consent form that was administered at the beginning of their participation in all data collection activities. First, a pre-program questionnaire collected baseline data from parents/caregivers before they started the first class in the session. Second, a post-program questionnaire collected information from parents/caregivers at the end of the session. Parents/caregivers were required to complete no less than 7 of 10 classes, or 70% of the session, to complete the post-program questionnaire. The third data collection method consisted of follow-up interviews with parents/caregivers four to six months after their participation in the MMG program. The sub-sample of parents/caregivers who participated in the follow-up interview process was randomly selected from the parents/caregivers that completed both the pre- and post-program questionnaires.

Of the 312 parents/caregivers who participated in the MMG program across the 13 sites during the 2011 Winter and 2011 Spring sessions, 119 parents/caregivers (80 winter, 39 spring) completed both the pre- and post-program questionnaires, which translates into a response rate of 38%. The outreach-site parents/caregivers constitute almost a quarter (24%) of all respondents to the pre- and post-program questionnaires. A total of 30 parents/caregivers participated in the follow-up interview process. Outreach site parents/caregivers accounted for a fifth (20%) of all interview participants.

Analytical methods

A variety of methods were required to analyse the diverse data sets related to the questions and hypotheses above.

For question 1, chi-square analysis was conducted to test the null hypothesis.3 Chi-square tests were run for the pre- and post-program results for the questions about the nine early literacy skill development activities. Winter and spring respondents, and branch-based and outreach-based participant results, were analysed separately. In addition, average response ratings were compared across different points in time (before program and immediately after program) and across different groups in the program (library and outreach). The average rating [End Page 109] of a question for the whole sample was calculated by assigning a numeric value to each frequency, adding up all of the responses based on these values, and then dividing by the total number of responses. The scale is from 0 to 5, 0 having the lowest frequency and 5 having the highest frequency along the following numeric values: 5 = many times a day; 4 = once a day; 3 = many times a week; 2 = once a week; 1 = once a month; 0 = rarely/never. The “before” and “after” average frequencies of how often all parents/caregivers did an activity were then compared and discussed.

For question 2, chi-square analysis was conducted to test the related null hypothesis. Chi-square tests were run for the questions about frequency of visits to the library in the pre- and post-program questionnaire results. Average response ratings for the whole sample were also developed, which were compared across different points in time (specifically, before the program and immediately after the program).

For question 3 and question 4, frequency counts and percentages were calculated for the sample of parents/caregivers who completed these questions on the post-program questionnaire and who had also completed the pre-program questionnaire. Frequency counts and percentages were also calculated for interview participant responses.

Results

This section is organized according to the four research questions noted above.

Parents’/caregivers’ responses to questions (pre- and post-questionnaire) about the use of nine early literacy skill development activities with their children

Average response ratings for both library branches and outreach sites are presented in this section. The results of the chi-square tests follow the review of the average ratings for all nine early literacy activities.

Talking to child about what is going on around them

On average, pre-program parents/caregivers rated the frequency with which they participated in this activity as 5.0, or many times a day. After participating in the MMG program, parents/caregivers provided an average frequency rating of 4.9, or barely below many times a day.

Neither outreach site parents/caregivers nor library branch site parents/care-givers experienced change in the frequency of this activity. In addition, average ratings were the same for outreach site parents/caregivers and library branch site parents/caregivers. The majority (90%) of parents/caregivers reported talking to their child about things going on around them many times a day both before and after participating in the MMG program.

Singing songs to child

On average, pre-program questionnaire parents/caregivers rated the frequency with which they participated in this activity as 4.5, or between once a day and [End Page 110] many times a day. After participating in the MMG program, parents/caregivers provided a slightly higher average frequency rating of 4.6.

Outreach site parents/caregivers and library branch site parents/caregivers saw similar change. Outreach site parents/caregivers were less likely than library branch site parents/caregivers to sing to their child many times a day both before and after participating in the MMG program. On average, outreach site parents/caregivers did this activity once a day, both before and after participating in the MMG program. The outreach site parents/caregivers had a 3.9 average rating before the program and 4.0 rating after the program.

Reading to child

On average, parents/caregivers reported that they read to their child once a day before participating in the MMG program (average rating of 4.1). After participating in the MMG program, the average rating increased from 4.1 to 4.4. This suggests that parents/caregivers were reading to their child more often after participating in the MMG program. The percentage of parents/caregivers who read many times a day increased for both outreach sites (+8.7 percentage points) and library branch sites (+11.5 percentage points) (figure 1).

Figure 1. Reported behavioural change in the frequency by which parents/caregivers read to children before and after participating in the MMG program (outreach and library)
Click for larger view
View full resolution
Figure 1.

Reported behavioural change in the frequency by which parents/caregivers read to children before and after participating in the MMG program (outreach and library)

Pointing out letters on objects such as toys or food boxes

Little change in the average rating of pointing out letters on objects was observed. Pre-program parents/caregivers had an average frequency rating of 3.8, [End Page 111] or almost once a day. Post-program parents/caregivers had a slightly higher average rating of 3.9.

Figure 2. Reported behavioural change in the frequency by which parents/caregivers point out letters on objects such as toys or food boxes before and after participating in the MMG program (outreach and library)
Click for larger view
View full resolution
Figure 2.

Reported behavioural change in the frequency by which parents/caregivers point out letters on objects such as toys or food boxes before and after participating in the MMG program (outreach and library)

While library branch parents’/caregivers’ ratings for pointing out letters dropped slightly after participating in the MMG program, outreach site parents’/caregivers’ showed an increase (+8.4 percentage points). Average ratings for outreach site parents/caregivers went from 3.5 (between many times a week and once a day) before participating in the MMG program to 4.2 (more than once a day) after participating in the MMG program (figure 2).

Saying nursery rhymes to child

On average, pre-program parents/caregivers rated the frequency with which they participated in this activity as 4.0, once a day. After participating in the MMG program, parents/caregivers had an average frequency rating of 4.1, just over once a day.

Outreach site parents/caregivers reported a greater increase in saying nursery rhymes than library branch site parents/caregivers after participating in the MMG program. On average, outreach parents/caregivers reported saying nursery rhymes many times a week before participating in the MMG program (3.1) but increased to almost once a day after participating in the MMG program (3.8). Close to half (42.9%) of outreach site parents/caregivers reported an increase in how often they said nursery rhymes to their child after participating in the [End Page 112] MMG program. Comparatively, just over a fifth (21.7%) of library branch site parents/caregivers increased use of nursery rhymes after participating in the MMG program, while the rest of respondents reported the same rates or a decrease in rates (figure 3).

Figure 3. Reported behavioural change in the frequency parents/caregivers say nursery rhymes to their child before and after participating in the MMG program (outreach and library)
Click for larger view
View full resolution
Figure 3.

Reported behavioural change in the frequency parents/caregivers say nursery rhymes to their child before and after participating in the MMG program (outreach and library)

Helping child learn new words by talking or reading

The average ratings for this activity remained constant at 4.6, or between once a day and many times a day, for both pre- and post-program frequency measures. A slight decrease in the frequency was observed in the outreach site parents/caregivers.

Little change was observed between outreach site parents/caregivers and library branch site parents/caregivers for this activity. As with the majority of early literacy activities, outreach site parents/caregivers reported doing this activity less frequently than library branch site parents/caregivers, both before and after participating in the MMG program. The majority of parents/caregivers reported doing this activity many times a day.

Helping child see and feel different shapes

On average, pre-program parents/caregivers rated the frequency with which they participated in this activity as 4.0, or once a day. After participating in the MMG program, parents/caregivers showed only a slight increase to an average rating of 4.2, or slightly more than once a day (figure 4). [End Page 113]

Figure 4. Reported behavioural change in the frequency by which parents/caregivers help their child see and feel different shapes before and after participating in the MMG program (out-reach and library)
Click for larger view
View full resolution
Figure 4.

Reported behavioural change in the frequency by which parents/caregivers help their child see and feel different shapes before and after participating in the MMG program (out-reach and library)

More change was seen in outreach site parents/caregivers than library branch site parents/caregivers. Before participating in the MMG program, almost 35 percent (34.8%) of outreach site parents/caregivers were helping their child see and feel different shapes many times a day. After participating in the MMG program, almost half (47.8%) of outreach site parents/caregivers were helping their child see and feel different shapes many times a day. This represents a 13% increase from before participating in the MMG program.

Listening and answering when child babbles or talks

Little change was reported by parents/caregivers for this activity. On average, pre-program parents/caregivers rated the frequency of this activity as 5.0, or many times a day. After participating in the MMG program, parents/caregivers provided an average frequency rating of 4.9, or just under many times a day. Outreach site and library branch site parents/caregivers reported little change in the frequency of this activity.

Asking child questions

On average, pre-program parents/caregivers rated the frequency that they used this activity as 4.9, or many times a day. After participating in the MMG program, parents/caregivers provided an average frequency rating of 4.9, or many [End Page 114] times a day—no change from before. Outreach site and library branch site parents/caregivers reported little change in the frequency of this activity. The majority of both groups did the activity many times a day with high frequency (between 87–95% on both pre- and post-MMG program questionnaires).

Table 2. Chi-square analysis of pre- and post-program results on nine early literacy skill development activities, winter 2011
Click for larger view
View full resolution
Table 2.

Chi-square analysis of pre- and post-program results on nine early literacy skill development activities, winter 2011

Table 3. Chi-square analysis of pre- and post-program results on nine early literacy skill development activities, spring 2011
Click for larger view
View full resolution
Table 3.

Chi-square analysis of pre- and post-program results on nine early literacy skill development activities, spring 2011

Chi-square test results related to frequency of early literacy activities

Chi-square tests on pre- and post-program data (spring and winter) related to the nine early literacy skill development activities did not meet the critical threshold (3.841) and could not be used to falsify null hypothesis 1 (tables 2 and 3). Therefore the MMG program did not significantly increase the frequency by which parents/caregivers engage their children in these nine early literacy skill development activities.

Parents’/caregivers’ responses to questions (pre- and post-questionnaires) about how often they visit the library

Before participating in the MMG program, less than half of the parents/care-givers visited the library twice a month or more (winter 42.5%, spring 38.5%). [End Page 115] After participating in a MMG session, the majority of parents/caregivers visited the library twice a month or more, excluding their visits to the library for the MMG program (winter 81.2%, spring 74.4%) (figure 5).

Figure 5. Reported behavioural change in the frequency by which parents/caregivers visited the library before and after participating in a MMG program (winter and spring)
Click for larger view
View full resolution
Figure 5.

Reported behavioural change in the frequency by which parents/caregivers visited the library before and after participating in a MMG program (winter and spring)

Table 4. Chi-square analysis of pre- and post-program results regarding library visits, winter and spring 2011
Click for larger view
View full resolution
Table 4.

Chi-square analysis of pre- and post-program results regarding library visits, winter and spring 2011

Across the whole sample of respondents who completed pre- and post-program questionnaires, two-thirds of parents/caregivers increased the frequency with which they visited the library after participating in the MMG program. Both outreach site parents/caregivers and library branch site parents/caregivers reported increases in the frequency of their library visits, despite the fact that outreach site parents/caregivers were not going to the library to participate in a MMG program.

Chi-square tests on pre- and post-program data related to the question about the frequency by which parents/caregivers visit the library passed the critical threshold (3.841) and therefore can be used to falsify the null hypothesis (table 4). The MMG program significantly increases the frequency by which parents/caregivers visit the library due to their participation in the MMG program. [End Page 116] Parents’/caregivers’ responses to questions (post-program questionnaires and follow-up interviews) about whether the MMG program increased their confidence and competence in using story-time materials and activities As outlined in table 5, the post-program questionnaire results demonstrate that the majority (74.6%) of parents/caregivers experienced a gain in confidence and competence in using story-time materials and activities due to their participation in the MMG program.

Table 5. Parents’/caregivers’ responses to questions (post-program questionnaire) about whether they gained confidence and competence in using story-time materials and activities
Click for larger view
View full resolution
Table 5.

Parents’/caregivers’ responses to questions (post-program questionnaire) about whether they gained confidence and competence in using story-time materials and activities

Participants’ responses to the follow-up interview question about gaining confidence due to the MMG program were even stronger. Most (86.7%) of the parents/caregivers reported that, four to six months after the program ended, they either agreed or strongly agreed with the statement that the program increased their confidence and competence to use story-time materials and activities. This percentage represents an increase of 12.1 percentage points when compared to the reported results from the post-program evaluation (74.6%).

Parents’/caregivers’ responses to questions (post-program questionnaires and follow-up interviews) about whether parents/caregivers use what they learned in the MMG program at home

The majority (88.1%) of parents/caregivers reported on their post-program questionnaire that they are using what they learned from the MMG program at home (table 6).

Similar to the higher percentage reported by interview participants above, participants’ responses to the follow-up interview question about whether they use what they learned in the MMG program at home were strong. Nearly all (96.7%) parents/caregivers reported that, four to six months after the program ended, they either agreed or strongly agreed with the statement that they are using what they learned from the MMG program at home This percentage represents an increase of 8.6 percentage points when compared to the reported results from the post-program evaluation (88.1%).

Discussion and future research/evaluation directions

This study did not falsify the null hypothesis that parents/caregivers do not exhibit an increased rate of using any of the nine early literacy skill development [End Page 117] activities with their children after participating in the MMG program. There are three factors that may have contributed to this finding.

Table 6. Parents’/caregivers’ responses to questions about whether they use what they learned in the MMG program at home
Click for larger view
View full resolution
Table 6.

Parents’/caregivers’ responses to questions about whether they use what they learned in the MMG program at home

First, the MMG program was attended by a cohort of parents/caregivers who reported, at the beginning of the program (pre-program questionnaire), high frequency rates of early literacy skill development activity use with their children. The majority of parents/caregivers (outreach and library branch) were already engaging their children at least once a day or many times a day (the two highest frequency options) in the nine early literacy skill development activities before participating in the MMG program. We have learned that when this type of participant cohort uses the scales of this study, little change is observed in the nine early literacy skill development activity areas. The overall lack of change in parent/caregiver frequency of use of the nine early literacy development activities is puzzling considering that some of the participants had limited knowledge of the key role that they play in the development of their child’s early literacy skills. Future research could explore the development of an approach to studying the different program impacts experienced by parents/caregivers who have and parents/caregivers who have not attended an Every Child Ready to Read @ Your Library workshop before participating in an MMG program. Future studies might also be well served by exploring the impact of the program on parents/caregivers who report low rates of engagement in the early literacy skill development of their children.

Second, the MMG program offers limited class time to parents/caregivers (30 minutes a week) and delivers the classes only 8 to 10 times over a three-month period. The high frequency ratings of pre-program early literacy skill development activities combined with the limited class time and longer time intervals between class offerings suggest that parent/caregiver engagement with the existing design of the MMG program can be expected to sustain, but not increase, the higher frequency ratings of early literacy engagement between parents/caregivers and their children. Future studies could be well served by exploring how the program could be adjusted or augmented to establish upward trends in the use of early skill development activities.

A third mitigating factor is the reliance on recall information from parents/caregivers. Parents/caregivers were asked to recall how often they participated in particular activities; however, clear time frames were not provided for respondents [End Page 118] (e.g., “in the last week, how many times did you . . .”). The data collected represent a point-in-time estimate made by parents/caregivers who attempted to recall activities over an ambiguous time frame. For example, parents/caregivers could variously have been thinking back on a particularly hectic week or on a specific day of quality time spent with their child. This introduced an unexplainable variable which was not factored into the research design. Future studies would be well served by specifying a time frame when positing point-in-time questions.

Given that this study focused on understanding changes in the frequency by which parents/caregivers engage their children in early literacy skill development activities as a result of their participation in the RPL MMG program, future studies could be expanded to understand whether MMG-type programs facilitate knowledge acquisition about specific early literacy activities. For example, assessing changes in how parents/caregivers approach the reading of a picture book with their child before and after their participation in the program would offer a simple measure of knowledge acquisition among participants that may be attributable to MMG-type programs.

With regard to the second null hypothesis, the study succeeded at falsifying the claim that parents/caregivers do not report increased rates of visiting the library after participating in the MMG program. The MMG program’s demonstrated capability to increase library visitation rates can be attributed to the way it introduces parents/caregivers to the library as a place for fun, music, dance, and learning. Such a definition of the library stands in stark contrast to conventional conceptions of the library as a place to be quiet and read. Perhaps this lively expression of the library attracts parents/caregivers to return to the library to see what else is offered.

In response to question 3, parents/caregivers (75% or more) noted an increased sense of personal confidence and competence in using story-time materials and activities as a result of participating in the MMG program. The role of the MMG programmers can be credited in large part with this result as they are the people who make a regular point of telling parents/caregivers that they can lead these early literacy development activities with their children at home and beyond. The class-time practice in a peer environment where everyone is trying out the activities may also have contributed to their sense of confidence and competence.

When asked about whether parents/caregivers are using what they learned from the program at home, 88.1% of respondents agreed or strongly agreed that this was the case. This suggests that while parents/caregivers, overall, sustained existing frequencies of early literacy skill development activities with their children, their new knowledge from the program is being integrated into their regular patterns of supporting their child in her/his early literacy skill development.

The study also observed that a greater percentage of parents/caregivers at outreach locations reported they more frequently engaged in all tested areas of their children’s early literacy skill development after participation in the program. This was not the case with all early literacy skill development areas for branch-based participants. This suggests that parents/caregivers accessing the program through outreach locations are experiencing a greater impact on their [End Page 119] engagement with their children compared to branch-based participants. It is unclear why this is the case and therefore future research on program impacts on outreach participants is needed to better understand this emerging finding.

Conclusion

The study’s findings suggest that the MMG program contributes to the maintenance of high frequency rates of early literacy skill development activities used by parents/caregivers and their children. No evidence was found to suggest that the MMG program significantly increased early literacy skill development activities by parents/caregivers. However, the study found that the MMG program significantly increased library visitation rates for parents/caregivers. In addition, the majority of participating parents/caregivers reported that the MMG program increased their sense of confidence and competence in using story-time materials and activities and, therefore, equipped them to use what they learned in the program at home.

As early literacy programming is an under-researched aspect of public libraries, its impact is still somewhat unclear. This study sheds light on some of the program impacts but primarily focused on questions of early literacy activity rate changes; such a focus leaves unanswered questions about the degree to which the MMG program, and other early literacy programs in public libraries, facilitate acquisition, retention, and application of early literacy knowledge and practice by parents/caregivers. Moreover, the study did not address questions about the impact of the program on the early literacy development of participating children. In order for public libraries to understand the types of learning outcomes that should be expected from early literacy programs, future research is necessary. This research must work toward the establishment of a limited set of specific measurable outcomes. Those outcomes should be integrated into the logic of evaluating early literacy programs, with a view to reporting impact to stake-holders and the community so the value that libraries bring to early childhood development and a child’s relations to her/his parent/caregiver is clearly stated and evidence based.

Scott Graham
SPARC BC (Social Planning and Research Council of British Columbia)
sgraham@sparc.bc.ca
André Gagnon
Regina Public Library
agagnon@reginalibrary.ca

Acknowledgment

The authors thank Lorraine Copas for her invaluable advice on the design of this study. We also thank Katie McCallum for her research assistance throughout the study.

Notes

1. Understanding the Early Years (UEY) is a national initiative that enables members of communities across Canada to better understand the needs of their young children and families so that they can determine the best programs and services to meet those needs. UEY’s overall purpose is to enable members of communities to work together to address the needs of young children from birth to age six by (1) raising family and community awareness of the importance of family and community factors that can influence young children’s development; and (2) strengthening their ability to use local data to help them make decisions to enhance children’s lives. For more on the UEY, see Human Resources and Skills Development Canada (2011). [End Page 120]

2. The null hypothesis expresses a default position regarding an observable phenomenon. A null hypothesis typically posits that there is no relationship between two measured phenomena or that a potential intervention has no effect. As such, the null hypothesis is a hypothesis which researchers try to disprove, reject, or falsify.

3. The chi-square (χ2) test is one of the most used tests of the non-parametric family of statistical tests. Chi-square can be used to test differences between two or more actual samples, which is how the test was applied to pre- and post-program data in this study.

References

Canadian Language and Literacy Research Network. 2009. National Strategy for Early Literacy: Report and Recommendations. http://docs.cllrnet.ca/NSEL/finalReport.pdf.
Cook, T. J., and D. T. Campbell. 1979. Quasi-experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally College Publishing.
Diamant-Cohen, B. 2006. Mother Goose on the Loose: A Handbook and CD-ROM Kit with Scripts, Rhymes, Songs, Flannel-Board Patterns and Activities for Promoting Early Childhood Development. New York: Neal-Schuman Publishers.
Fehrenbach, L. A., D. P. Hurford, and C. R. Fehrenbach. 1998. “Developing the Emergent Literacy of Preschool Children through a Library Outreach Program.” Journal of Youth Services in Libraries 12 (1): 40–45.
Human Resources and Skills Development Canada. 2011. Summative Evaluation of the Understanding the Early Years Initiative: Final Report. Departmental catalogue no. SP-1019-12-11E. http://www.hrsdc.gc.ca/eng/publications/evaluations/social_development/2011/sp_1019_12_11_eng.pdf.
Laughlin, S. 2003. Evaluation: Every Child Ready to Read @ Your Library Pilot Project. Chicago: American Library Association.
MacLean, J. 2008. Early Literacy Programming in Public Libraries: A Review of the Literature. Provincial and Territorial Public Libraries Council.
Martinez, G. 2007. “Partnering for Reading Readiness: A Case Study of Maryland Public Librarians.” Children and Libraries 5 (1): 32–39.
McGill, J. 2003. “Public Library – Nursery Relationships: An Investigation into Their Value and Impact.” MA thesis, University of Sheffield.
Stewart, R. 2010a. Idaho Commission for Libraries Evaluation of Read to Me Program: Final Report. http://libraries.idaho.gov/files/Case-Studies2009-2010.pdf.
Stewart, R. 2010b. Idaho Commission for Libraries Results from Follow-up Telephone Survey with Every Child Ready to Read Family Workshop Participants (Spring 2009) and First Book Participants (2008–2009): Interim Report. http://libraries.idaho.gov/files/Interim-Report-March-2010.pdf.
Stewart, R. 2011a. First Book 2008–2009, 2009–2010, 2010–2011 Analysis of Longitudinal Data: Brief Report. http://libraries.idaho.gov/files/RTM-FirstBook-Report-2010-2011_0.pdf .
Stewart, R. 2011b. Every Child Ready to Read Family Workshops Analysis of Longitudinal Data: Fall 2008 to Fall 2010: Brief Report. http://www.libraries.idaho.gov/files/ECRTR-Report-2011.pdf.
Stone, E. 1999. “The Impact of Public Library Use on the Educational Attainment of Primary School Children.” MA thesis, University of Sheffield.
Understanding the Early Years. 2009. UEY Regina Research Update: Measuring Early Childhood Development in Regina. http://www.reginakids.ca/rsu_docs/uey-regina_edi-results-sept200953738.pdf. [End Page 121]

Appendix 1. Mainly Mother Goose Program Evaluation – Pre-Program Survey

Preamble

This evaluation is being conducted as part of the Mainly Mother Goose program evaluation. The purpose of the evaluation is to determine the impact of the Mainly Mother Goose program on parent/caregiver engagement with their children regarding early literacy.

  1. 1a. Your responses to the following questions will be analyzed thematically and/or at the aggregate level and will not be attributed to you personally. Do you consent to having the information you provide on this survey be used in the final evaluation report?

    • Yes □

    • No □

    • Signature:

A. Basic Administrative Data

  1. 1b. Have you already completed this form in another Mainly Mother Goose program?

    • Yes □

    • No □

    • If you answered <underline>yes</underline>, please do not complete this form.

    • If you answered <underline>no</underline>, please complete this form.

B. About you and your child

  1. 2. What is your postal code?

  2. 3. What is your gender?

    • □ male

    • □ female

  3. 4. What is your child’s gender?

    • □ male

    • □ female

  4. 5. How old is your child?

    • □ 0-5 months

    • □ 5–8 months

    • □ 8–13 months

    • □ 13–15 months

    • □ 15–18 months

    • □ 18–24 months

    • □ Other: [End Page 122]

C. Early Literacy Engagement with your Child

Many times a day Once a day Many times a week Once a week Once a month Rarely/Never
6. Talking to my child about what is going on around us
7. Singing songs to my child
8. Reading to my child
9. Pointing out letters on objects such as toys or food boxes
10. Saying nursery rhymes to my child
11. Helping my child learn new words by talking and reading
12. Helping my child see and
feel different shapes
13. Listening and answering when my child babbles or talks
14. Asking my child questions

D. Your Involvement with the Mainly Mother Goose Program

  1. 15. How many times have you gone to a Mainly Mother Goose program with your child?

    • □ This is our first time

    • □ This is our second time

    • □ This is our third time

    • □ Other:

  2. 16. Have you gone to a Mainly Mother Goose program with any of your other children? <list>

    • □ Yes

    • □ No

  3. 17. Is the time of the Mainly Mother Goose program good for you? <list>

    • □ Yes

    • □ No

  4. 18. How could the timing of the Mainly Mother Goose program work better for you?

  5. 19. Is the location of the Mainly Mother Goose program good for you?

    • □Yes

    • □ No

  6. 20. How could the location of the Mainly Mother Goose program work better for you?

  7. 21. Why are you coming to the Mainly Mother Goose program? [End Page 123]

  8. 22. How did you find out about the Mainly Mother Goose program?<list>

    • □ Word of mouth

    • □ Library staff

    • □ TV

    • □ Radio

    • □ Newspaper

    • □ Website

    • □ @ the Library

    • □ Flyer/Poster

    • □ Other:

  9. 23. Before the Mainly Mother Goose program, how often did you visit the Library with your child?

    • □ Daily

    • □ 2 to 4 times a week

    • □ Once a week

    • □ Once a month

    • □ 2 or 4 times a month

    • □ 2 or 4 times a year

    • □ Never

    • □ Other:

  10. 24. What library branch(es) do you go to?

  11. 25. Why do you go to that branch or those branches?

  12. 26. Anything else you would like to tell us?

Thank you for completing this survey!

Many times a day Once a day Many times a week Once a week Once a month Rarely/Never
8. Talking to my child about what is going on around us
9. Singing songs to my child
10. Reading to my child
11. Pointing out letters on objects such as toys or food boxes
12. Saying nursery rhymes to my child
13. Helping my child learn new words by talking and reading
14. Helping my child see and feel different shapes
15. Listening and answering when my child babbles or talks
16. Asking my child questions

[End Page 124]

Appendix 2. Mainly Mother Goose Program Evaluation – Final Post-Program Survey

Preamble

This evaluation is being conducted as part of the Mainly Mother Goose program evaluation. The purpose of the evaluation is to determine the impact of the Mainly Mother Goose program on parent/caregiver engagement with their children regarding early literacy.

  1. 1. Your responses to the following questions will be analyzed thematically and/or at the aggregate level and will not be attributed to you personally. Do you consent to having the information you provide on this survey be used in the final evaluation report?

    • Yes □ No □

    • Parent/caregiver Name: _________ Signature: ___________

A. Basic Administrative Data

  1. 2. Did you complete the pre-program survey?

    • Yes □

    • No □

  2. 3. What is your postal code?

  3. 4. What is your gender?

    • □ male

    • □ female

  4. 5. What is your child’s gender?

    • □ male

    • □ female

  5. 6. How old is your child?

    • □ 0–5 months

    • □ 5–8 months

    • □ 8–13 months

    • □ 13–15 months

    • □ 15–18 months

    • □ 18–24 months

    • □ Other:

  6. 7. How many Mainly Mother Goose classes did you attend?

    • □ 1

    • □ 2

    • □ 3

    • □ 4

    • □ 5

    • □ 6

    • □ 7

    • □ 8

    • □ 9

    • □ 10 [End Page 125]

Part 1: Now that you have participated in the Mainly Mother Goose program, how often do you do the following activities with your child?

Many times a day Once a day Many times a week Once a week Once a month Rarely/Never
8. Talking to my child about what is going on around us
9. Singing songs to my child
10. Reading to my child
11. Pointing out letters on objects such as toys or food boxes
12. Saying nursery rhymes to my child
13. Helping my child learn new words by talking and reading
14. Helping my child see and feel different shapes
15. Listening and answering when my child babbles or talks
16. Asking my child questions

Part Two: Please indicate the degree to which you agree with the following statements.

Because I participated in the Mainly Mother Goose program:

Strongly Agree Agree Neutral Disagree Strongly Disagree
17. I use more rhymes with my child
18. I listen more to my child when he/she babbles or talks
19. I use more words and a greater variety of words with my child
20. I talk more with my child about things going on around us
21. I sing more songs with my child
22. I read more to my child
23. I point out letters on objects such as toys or food boxes more often
24. I ask my child more questions
25. I help my child see and feel different shapes more often
26. I made new connections with other parents/caregivers
27. I gained confidence and competence in using story time materials and activities
28. I am using what I learned at home
29. I am more interested in coming to the library

[End Page 126]

  1. 30. Who in your household reads to your child most often? (check all that apply)

    • □ Father

    • □ Mother

    • □ Grand Parent(s)

    • □ Caregiver

    • □ Other:

B. Your Involvement in the Mainly Mother Goose Program

  1. 31. What three activities did you learn from the mainly mother goose program that you continue to do with your child most often?

  2. 31. How would you rate your experience and your child’s experience in the Mainly Mother Goose Program?

    • □ Excellent □ Not very good

    • □ Very good □ Poor

    • □ Good □ Unsure/No answer

    • □ OK

  3. 32. My child liked the following three (3) activities the best:

    • 1.

    • 2.

    • 3.

  4. 33. I liked the following three (3) things the best:

    • 1.

    • 2.

    • 3.

  5. 34. What was the most important thing that you learned from the Mainly Mother Goose Program?

  6. 35. Now that you have participated in the Mainly Mother Goose program, do you have a better understanding of your role in the development of the early literacy skills of your child?

  7. Yes □

  8. No □

  9. Unsure/No answer □

  10. 36. Do you have any suggestions for improving the Mainly Mother Goose program?

  11. 37. Do you have any stories about your experience in the Mainly Mother Goose program that you would like to share?

  12. 38. Who in your household comes to the library with your child most often? (check all that apply)

    • □ Father

    • □ Mother

    • □ Grand Parent(s)

    • □ Caregiver

    • □ Other:

  13. 39. Do you plan to continue to come to other PL pre-school programs with your child?

    • □ Yes

    • □ No [End Page 127]

    • □ Likely

    • □ Unlikely

    • □ Don’t know

    • □ Unsure/No answer

  14. 40. Now that you have participated in the Mainly Mother Goose program, how often do you visit the Library with your child?

    • □ Daily

    • □ Once a week

    • □ 2 to 4 times a week

    • □ Once a month

    • □ 2 to 4 times a month

    • □ 2 to 4 times a year

    • □ Never

    • □ Other

    • 41. What would bring you to the library more often?

C. Follow-up and Final Comments

  1. 42. Would you be willing to participate in a follow-up interview in three (3) months to discuss your experience with the Mainly Mother Goose program?

    • □ Yes

    • □ No

    • □ I am unsure at this time but contact me later

  2. 43. If yes, please provide your current contact information as well as indicate that best time to reach you?

  3. Name:

  4. Phone (Home):

  5. Phone (Cell):

  6. Phone (Work):

  7. Email:

  8. Best Time to Reach Me: Morning Afternoon Evening

  9. 44. Anything else you would like to tell us?

Thank you for completing this survey! [End Page 128]

Appendix 3. Mainly Mother Goose Program – Interview Guide

Follow-up months after the end of the program

Preamble

You have been selected to participate in this interview because of your participation in the Mainly Mother Goose Program at the Public Library.

  • • The interview should last approximately 30 minutes but can be longer or shorter depending on the questions asked.

  • • • Responses will be reported in a way that protects your identity and privacy. Participation is entirely voluntary and you may end the interview at any time. You may also skip any questions you don’t wish to answer.

The results of this interview will only be used to inform the evaluation of the Mainly Mother Goose Program.

A. Basic Administrative Data

  1. 1. May the information you give in this interview be used for the purposes of the Mainly Mother Goose program evaluation?

    • Yes □

    • No □

      Parent/caregiver Name: Signature:

  2. 2. What is your postal code?

  3. 3. What is your gender?

    • □ male

    • □ female

  4. 4. What is your child’s gender?

    • □ male

    • □ female

  5. 5. How old is your child now?

    • □ 0–5 months

    • □ 5–8 months

    • □ 8–13 months

    • □ 13–15 months

    • □ 15–18 months

    • □ 18–24 months

    • □ Other:

Part One: Please indicate the degree to which you agree with the following statements.

Now that several months have passed since I participated in the Mainly Mother Goose program:

Strongly Agree Agree Neutral Disagree Strongly Disagree
6. I use more rhymes with my child
7. I listen more to my child when he/she babbles or talks
8. I use more words and a greater variety of words with my child
9. I talk more with my child about things going on around us
10. I sing more songs with my child
11. I read more to my child
12. I ask my child more questions
13. I point out letters on objects such as toys or food boxes more often
14. I help my child see and feel different shapes more often
15. I made new connections with other parents
16. I gained confidence and competence in using story time materials and activities
17. I am using what I learned at home
18. I am more interested in coming to the library

[End Page 129]

  1. 19. Who in your household reads to your child most often? (check all that apply)

    • □ Father

    • □ Mother

    • □ Grandparent(s)

    • Caregiver □

    • Other:

Part Two: Now that several months have passed since you participated in the Mainly Mother Goose program, how often do you do the following activities with your child?

Many times a day Once a day Many times a week Once a week Once a month Rarely/Never
20. Talking to my child about what is going on around us
21. Singing songs to my child
22. Reading to my child
23. Pointing out letters on objects such as toys or food boxes
24. Saying nursery rhymes to my child
25. Helping my child learn new words by talking and reading
26. Helping my child see and feel different shapes
27. Listening and answering when my child babbles or talks
28. Asking my child questions

[End Page 130]

C. Your Involvement in the Mainly Mother Goose Program

  1. 29. What activities did you learn from the Mainly Mother Goose Program that you continue to do with your child at home?

  2. 30. What types of supports/programs could the Public Library offer you that would help you develop early literacy skills with your child (i.e., list of resources, web-site, presentations, etc.)?

  3. 31. Do you have any stories about using activities with your child that you learned from the Mainly Mother Goose program?

  4. 32. After attending the Mainly Mother Goose program do you have a better understanding of how talking, singing, reading and listening to your child prepare him/her for learning to read? Why?

  5. 33. Now that several months have passed since you participated in the Mainly Mother Goose program, how often do you visit the Library with your child?

    • □ Daily

    • □ Once a week

    • □ Once a month

    • □ 2 to 4 times a week

    • □ 2 to 4 times a month

    • □ 2 to 4 times a year

    • □ Never

    • □ Other

  6. 34. What would bring you to the library more often?

  7. 35. Do you have any suggestions for improving the Mainly Mother Goose Program?

  8. 36. Is there anything else you would like to tell us?

Thank you for your participation! [End Page 131]

Additional Information

ISSN
1920-7239
Print ISSN
1195-096X
Pages
103-121
Launched on MUSE
2013-06-27
Open Access
Yes
Archive Status
Archived 2022
Back To Top

This website uses cookies to ensure you get the best experience on our website. Without cookies your experience may not be seamless.