Democratization and Diversion:The Effect of Missouri’s A+ Schools Program on Postsecondary Enrollment
Recent federal and state education policy has targeted community colleges as an affordable venue to increase postsecondary attainment. We examined a state program aimed at increasing community college enrollment, the Missouri A+ Schools Program, which provided eligible graduates from participating high schools the opportunity to earn a scholarship at a Missouri public two-year college. The Missouri A+ Schools Program aims to increase the democratization of education by providing greater access to attend postsecondary institutions but may simultaneously create a diversion away from four-year colleges. The staggered adoption of the Missouri A+ Schools Program across high schools allowed a quasi-experimental estimation of the effect of the program on postsecondary enrollment. The Missouri A+ Schools Program increased the overall college-going rate by 1.5 percentage points for graduates from A+ designated high schools. Furthermore, the A+ Schools Program increased two-year college-going rates by 5.3 percentage points, and decreased four-year college-going rates by 3.8 percentage points. Overall, the A+ Schools Program provided a democratizing effect by increasing overall postsecondary enrollment, while simultaneously creating a diversionary effect through increased two-year enrollment and a decline in four-year enrollment.
college access, college choice, financial aid, community college, quasi-experimental
Policymakers at all levels of government have emphasized the importance of community colleges and technical schools as essential entry points to postsecondary education for many high school graduates, especially those from low-income backgrounds. During his inaugural term in office, President Obama called for increasing the number of community college graduates by five million over ten years (Obama, 2009). In 2015, President Obama announced a proposal to provide free community education for students who earn good grades and persist through college (The White House, 2015).
At the state level, the Tennessee Promise program was established in 2014 to provide tuition waivers for students to attend community colleges. Additionally, this program pairs students with mentors to help them navigate the college admission process. Similarly, other state politicians, including Governor Jay Nixon of Missouri, have called for the expansion of financial assistance for high school students to attend colleges and technical schools (Nixon, 2011). At both the federal and state level, it has become apparent that some policy-makers view subsidizing community college as both a politically, and economically, feasible solution to increase postsecondary enrollment and drive economic development.
State policy makers have also created and expanded state-funded financial aid policies to encourage enrollment in postsecondary education. In general, researchers have found that various state-funded financial aid programs are successful at increasing postsecondary education enrollment (Dynarski & Scott-Clayton, 2013; Heller, 2002). With respect to community colleges, state-funded financial aid programs also had a positive impact on enrollment (Toutkoushian & Hillman, 2012). In spite of the growing evidence of improved college access through state-funded financial aid programs, few studies have examined the effects on enrollment patterns created by programs which only support community college access, such as Missouri’s A+ Schools Program1.
Created in 1993, the Missouri’s A+ Schools Program provides scholarship support to eligible graduates from designated high schools to attend any instate, public two-year college. The Missouri A+ Schools Program is a state policy which has two main purposes: (a) to create incentives for schools to restructure and improve their curriculum; and (b) to provide an opportunity for students, especially those who are not college bound, to attend two-year colleges. State-funded financial aid programs, like the A+ Schools Program, democratize education by promoting college access and affordability through the opportunity for students to earn a scholarship to attend college. [End Page 802]
As the democratization of higher education addresses issues of access and affordability, policy-makers and scholars often overlook the diversionary impact of these scholarship programs away from four-year colleges (Gonzalez & Hilmer, 2006; Hilmer, 1997; Rouse, 1995; Sandy, Gonzalez, & Hilmer, 2006). College-bound high school students participating in programs similar to the Missouri A+ Schools Program must weigh the perceived benefits of attending a subsidized two-year college with those of a potentially costlier four-year college. Thus, these policies create trade-offs between democratization of education and the diversion of enrollment from four-year to two-year colleges. While the democratization of education is seen as an overall good, critics caution of the possible “stunting” of educational attainment for students choosing to enroll in a community college instead of a four-year institution (Leigh & Gill, 2003). Interested in this trade-off, we examine whether the Missouri A+ Schools Program affected postsecondary enrollment of high school graduates across two- and four-year institutions.
The A+ Schools Program
Established by the Outstanding Schools Act of 1993 (Missouri Senate Bill 380), the A+ Schools Program was implemented to “improve education for non-college bound students” (Missouri Governor’s Office, 1993). The first goal, school improvement, was addressed by making school-level participation in the program contingent on a commitment to an outstanding education via curriculum review and revision, community involvement, and a focus on decreasing dropout rates. The second goal, increased student performance, was addressed through requirements that students participating in the program commit by their sophomore year, maintain satisfactory academic progress, and stay civically engaged. Upon meeting these requirements and successfully completing high school, graduates are eligible for a full tuition scholarship to attend an instate community college or technical school.
Missouri high schools voluntarily applied to the Missouri Department of Education and Secondary Education (DESE) to become an A+ School by demonstrating a commitment to meet the 11 program requirements. DESE designated new A+ schools on an annual basis. Of the 11 school requirements, five were curricular standards, two focused on student needs, three were requirements to involve the community, and one required the district to hire an A+ coordinator to oversee the program. The curricular requirements of the program require schools to commit to providing rigorous coursework in all academic subjects to better prepare students for postsecondary enrollment or employment. [End Page 803] Requirements focused on student needs involve schools providing early intervention services for at-risk students as well as mentoring services for students entering the labor force after graduation. Community requirements involved the creation of apprenticeships and internship programs, as well as schools developing school-community partnerships (e.g., local business, parents, vocational schools) to increase student success (Missouri General Assembly, 2009). Combined, these program requirements were designed to help ensure that schools provide a comprehensive approach to increasing student success.
It is important to reiterate that Missouri high schools choose to apply to the A+ program. Given the considerable amount of resources required, some Missouri high schools choose not to apply or participate. For instance, as mandated by statute, Missouri high schools must hire an A+ coordinator to oversee the program. Additionally, the application process can be costly, because the high schools are required to: (a) form an A+ committee; (b) review and revise their curriculum; (c) collect multiple years of data; (d) form partnerships in the community; and (e) complete the review process conducted by the Missouri Department of Higher Education. A school must notify the Missouri Department of Higher Education at least two years before submitting its application. During this initial process, the school will work closely with the state on implementation of the program. While the program components and requirements for schools are outlined, schools ultimately decide on the design and structure of the programming they will implement. This leads to potential differing effects across schools as some programming might be more effective than others.
While schools must demonstrate a commitment to student success, the A+ Schools Program has a similar expectation for students. To successfully complete the program, students must: (a) maintain a 2.5 cumulative GPA; (b) have an attendance rate of at least 95 percent from grades 9 to 12; (c) perform 50 hours of supervised tutoring or mentoring; (d) maintain a record of good citizenship by avoiding the unlawful use of drugs and alcohol; and (e) have attended an A+ designated high school for 3 consecutive years prior to graduation. Students meeting these requirements are then eligible to receive a full scholarship2 to attend any instate public community or technical college to pursue approved programs of study3 (Missouri Department of Elementary & Secondary Education, 2009). Students could continue to receive these benefits for up to five semesters as long as they complete a full-time course load each semester and maintain a 2.5 cumulative GPA (A+ Schools Program, 2010). In short, the A+ Schools program provides incentives to encourage students to complete the A+ curriculum in high [End Page 804] school and to attend community colleges or technical schools after high school graduation. It is also important to note that students are not automatically eligible for the A+ scholarship because they attend an A+ school, rather they must have also met the student standards.
Since its inception, the number of schools and students participating in the A+ Schools Program has steadily increased. Figure 1 summarizes high school participation in the A+ Schools Program and highlights the substantial growth in the number of A+ eligible graduates in recent years. The first 26 A+ high schools were designated in 1997, and 8.8% of graduates from these schools (433 students) were eligible to receive A+ tuition assistance in the first year. The number of A+ high schools steadily increased over time, but the trend accelerated between 2006 and 2010. In 2006, there were 11,031 A+ eligible high school graduates (31% of total graduates from A+ designated schools). By 2010, the last year of our sample, the growth in high schools becoming A+ designated substantially increased the number of A+ eligible graduates to 17,879, with 36% of graduates from A+ designated schools participating in the program.
[End Page 805]
Descriptively, it is clear that the number of schools and students benefitting from the A+ Schools Program increased over time. However, this was not necessarily indicative of the overall effectiveness of the A+ Schools Program. Given the interconnectedness of the A+ Schools Program’s two goals (i.e., school improvement and student performance), disentangling the effect of each is difficult and not the intent of our study. We instead focused on examining the average intent-to-treat (ITT) effect of the A+ Schools Program on postsecondary enrollment. The annual introduction of newly designated schools provided a quasi-experimental design opportunity in which we estimated the effect of the A+ Schools Program on postsecondary enrollment.
Review of Literature
Unlike many college enrollment programs that target one barrier of enrollment, particularly financial need, the A+ Schools Program belongs to a small group of state funded college access programs utilizing a multifaceted approach towards increasing college enrollment. The interplay of multiple components of the A+ Schools Program, which are targeted at multiple constructs influencing a high school student’s postsecondary trajectory, makes understanding the overall effectiveness of this program important and pertinent to advancing our knowledge of state college access policies.
Most college access programs seek to increase postsecondary enrollment by offering financial aid to encourage student enrollment. Perhaps the most widely studied of these programs is the Georgia Helping Outstanding Pupils Educationally (HOPE) program, which provides a financial subsidy to attend an instate institution of higher education. Since its implementation, the HOPE scholarship has been found to have a positive influence on postsecondary enrollment (Cornwell, Mustard, & Sridhar, 2006; Dynarski, 2004). Cornwell et al. (2006) found that the majority of increased enrollment due to HOPE occurred in Georgia four-year institutions. However, Singell, Waddell, and Curs (2006) found that Pell grant recipient enrollment increased relatively evenly across four- and two-year institutions, indicating that finances may not be the sole barrier to access four-year institutions for low-income students. Other large scale state-funded merit aid programs have been found to positively affect college enrollment, including programs in the District of Columbia (Abraham & Clark, 2006; Kane, 2007), Florida (Castleman & Long, 2013; Zhang, Hu, & Sensenig, 2013), Massachusetts (Cohodes & Goodman, 2014), and Tennessee (Bruce & Carruthers, 2014). [End Page 806]
A critique of programs like the Georgia HOPE and District of Columbia Tuition Assistance Program is the lack of an early commitment from students. This places at a disadvantage the students who do not become aware of the program until later in high school, when it is too late for them to meet all the program’s requirements (Perna, Rowan-Kenyon, Bell, Thomas, & Li, 2008). Addressing this flaw, Tierney and Hagedorn (2002) argue that an effective postsecondary enrollment program should place an emphasis on academics and parental involvement, include or build strong connections with postsecondary institutions, provide stable financial support, prepare students for multiple postsecondary options, and emphasize early intervention.
Few programs focused on postsecondary enrollment address all of the recommendations set forth by Tierney and Hagedorn (2002). For instance, the Kalamazoo Promise covers up to 100% of tuition at any Michigan public college or university for any student graduating from the Kalamazoo School District. Andrew, Desjardins, and Ranchhod (2010) found that the Kalamazoo Promise increased opportunities for lower income students to consider more selective and expensive institutions. Additionally, Bartik and Lachowska (2012) found that the Kalamazoo Promise increased the probability of students earning additional school credits each year, fewer days of suspension, and increased grade point averages for African American students.
The Oklahoma Promise Grant is a state program covering tuition for up to 6 years at an Oklahoma public postsecondary institution. To be eligible, students must register in the 9th grade, come from a family with a household income less than $50,000, meet certain academic benchmarks, refrain from drug use and criminal activity, and maintain an overall high school GPA of 2.5 or higher. Mendez, Mendoza, and Malcolm (2011) found that students with financial aid packages that included the Oklahoma Promise Grant had the highest retention rates and were more likely to persist to their second year. This is likely a result of the academic preparation required for eligibility which increases the likelihood that students graduate from high school better prepared academically (De La Rosa, 2006; Mendoza, Mendez, & Malcolm, 2009).
Indiana’s 21st Century Scholars Program covers tuition for eligible students to an Indiana public postsecondary institution or a portion of tuition at a private or independent college. The program is open to all students who qualify for the Federal Free and Reduced Lunch Program, take a pledge, maintain a 2.0 GPA, and do not commit a crime. St. John et al. (2004) found that students participating in the 21st Century Scholars Program, as identified by those signing the pledge in middle [End Page 807] school, were more likely to aspire to attend and more likely to enroll at a postsecondary institution following high school. Affirmed scholars, those students who completed the program requirements, were around five times more likely to enroll and persist in a postsecondary institution than those not completing the requirements (St. John et al., 2004). However, after accounting for self-selection in the program, Toutkoushian et al. (2013) found a significantly smaller impact of the 21st Century Scholars program on college aspiration and enrollment.
Through their emphasis on early intervention and the provision of a financial award to encourage postsecondary participation, the programs discussed in this section take a comprehensive approach to increasing postsecondary participation. Missouri’s A+ Schools Program takes a similar approach, however, we still know little about its impact on democratization and diversion of postsecondary education. This study addresses this question by examining the effect of the Missouri A+ Schools Program on two- and four-year college enrollment.
Following the student choice utility model described in DesJardins and Toutkoushian (2005), we define the utility a student receives based upon their choice of college as Utj = U(Fj, Fi, Si), where Utj represents the utility for student i when choosing college j. We define the inputs to the utility function as: Fj represents financial factors associated with attending institution j (e.g., tuition, fees, room and board, financial aid); Fi represents financial factors associated with student i (e.g., family income, wealth); and Si represents nonfinancial student characteristics related to the utility of the choice of a particular college (e.g., ability, parental influence, cultural and social capital, preference for a particular major). Within this choice framework, utility is maximized by choosing college j such that the utility of choosing college j is larger than the utility for all alternative college and non-college options (i.e., Uij.>Uik, for all k ≠ j).
We apply this college choice model, which predicts how the A+ program is likely to alter the college choice of students exposed to the A+ program, by simplifying the choice set to three potential choices for student i: the choice to attend a two-year college (Ui2); the choice to attend a four-year institution (Ui4) ; and the choice to not attend college at all (Ui0). We implicitly assume that the utility from attending a two-year college (Ui2) (and the parallel choice among four-year and the no college options) represents the utility obtained by choosing the two-year college within the choice set which maximized student i’s utility [End Page 808] across all two-year college options. We describe the conceptual predictions of the A+ program on college choice through the A+ program's two primary mechanisms, the direct effect of the scholarship component (which alters Fj) and the program participation component (which alters Si).
The Scholarship Component of the A+ Schools Program
The predicted effect of the A+ scholarship component is pretty straightforward. As described earlier, students who complete the A+ program are eligible to receive a tuition waiver at two-year institutions within the state of Missouri. Empirically, given the preponderance of evidence regarding the negative correlation between net-price and college enrollment, particularly for low-income students, it is safe to assume that the probability of choosing a particular college increases as the net-price of that college decreases (Deming & Dynarski, 2010; Dynarski & Scott-Clayton, 2013, Heller, 1997). Within our utility framework, we posit that the utility of attending a two-year college for student i is greater if they were A+ eligible as opposed to if they were not (i.e., ) due to the reduction in net-cost at two-year colleges. However, the utility for both the choice of a four-year institution ( ) and the choice to not attend college ( ) remains equal whether or not a student is A+ eligible as the A+ scholarship does not alter the net-cost of these choices.
Within this framework, there are three potential margins for the scholarship component of the A+ Schools program to change college choices. For students on the margin between choosing a two-year college or not attending college, the scholarship would increase the probability that a marginal student would enroll in a two-year college, and decrease the probability of choosing not to attend college. For students on the margin between a two-year and a four-year college, the model predicts an increase in the probability of attending a two-year college and a decrease in the probability of attending a four-year college. Finally, the model would predict that the A+ scholarship component would not change the probability of enrolling in a four-year college versus not enrolling in college, as the relative utilities were unaffected by the scholarship. Combined, we would expect the scholarship component of the A+ Schools program to increase two-year enrollments, cause a decrease in four-year enrollments, and cause a decrease in the number of students not attending college. [End Page 809]
The Program Participation Component of the A+ Schools Program
Hossler and Gallagher’s (1987) model of student college choice suggests that a student’s decision to pursue a postsecondary education consists of three stages: predisposition, search, and choice. The predisposition stage consists of the evolving phase in which students are still considering whether they wish to attend college. In the search stage, students are learning the elements of their postsecondary options as well as learning which attributes to consider for their goals. In the choice stage, students formulate what they have learned into choosing an institution. It is clear that the scholarship mechanism works through the choice stage. We posit that the program participation mechanisms of the A+ program is likely to impact the predisposition and search stages of the college choice process by influencing the student’s nonfinancial factors associated with the choice of college (Si) in our college choice utility model.
The A+ Schools Program contains multiple mechanisms likely to influence the nonfinancial factors influential in a student’s utility of college choice. Perhaps most influential is the A+ Schools Program’s focus on increasing overall school quality through an increase in academic rigor for students in non-college preparation academic tracks which is likely to lead to increased graduation and college entrance rates (Cabrera & La Nasa, 2001; Moore et al., 2010). Additionally, community partnerships and family engagement opportunities provide students with access to cultural capital to inform them of the opportunities and resources available after high school (Engberg & Wolniak, 2010; Perna & Titus, 2005). Furthermore, by making the A+ scholarship contingent on the successful completion of requirements encompassing their entire high school years, the A+ Schools Program encourages students to consider a postsecondary education early in high school, likely influencing their high school trajectory and setting the foundation for students to be prepared and competitively eligible for higher education (De La Rosa, 2006; Heller, 2006).
Within the framework of our utility model, we posit that participation in the A+ program affects a student's nonfinancial college choice factors. Therefore, being exposed to the A+ program increases the utility of choosing a two-year college (i.e., ) increases the utility of choosing a four-year college (i.e., ), while the utility of not going to college remains the same (i.e., ). Given this framework, we would hypothesize that students on the margin between [End Page 810] choosing a two-year college and not going to college would be more likely to choose a two-year college. Similarly, students on the margin of choosing a four-year college and not attending college would be more likely to choose a four-year college. However, for students on the margin between choosing a two-year and four-year college the prediction is ambiguous as utility increased for both choices but the model does not offer a prediction of relative strength. Combined, the program participation mechanisms of the A+ Schools program should increase overall college attendance, although the relative increase across two- and four-year colleges is ambiguous, and decrease the number of students who do not attend college.
By combining the predictions of our college choice utility model for both mechanisms of the A+ Schools program, we proposed two hypotheses regarding whether the A+ Schools Program had a democratization or diversionary effect on postsecondary enrollment. First, we predicted that the democratization effect of the A+ Schools program would be strongly positive and we expected to see an increase in overall post-secondary enrollment of students after the implementation of the A+ Schools Program. The scholarship mechanism increased the utility of the two-year option relative to the no college option, which predicted a shift of students from not attending college to attending a two-year college. Furthermore, the program participation effect of the A+ program would likely increase the probability of a student on the margin of choosing either the two- or four-year college option relative to the no college option. Combined, the effects offered an unambiguous prediction that the A+ Schools Program would increase college enrollment after implementation.
Second, our prediction regarding whether the A+ program had a diversionary effect was less straightforward. With respect to the scholarship effect, the prediction that students on the margin between two- and four-year colleges would become more likely to choose a two-year college was unambiguous due to the financial implications. However, the A+ program participation mechanism was less obvious as students on the margin between four-year and no college would be more likely to attend a four-year college but the model did not make an unambiguous prediction for the effect on those on the margins between two-year and four-year colleges. Thus, while it was clear that we could expect more students to enroll in two-year colleges based upon both mechanisms, the overall effect for four-year enrollment remained an empirical question. [End Page 811]
To examine the effect of the A+ Schools Program on college-going rates, we used school-level administrative data from 1992 to 2010 collected by the Missouri Department of Elementary and Secondary Education (DESE). DESE designated the first A+ Schools in 1997, so by including data from 1992, we accounted for postsecondary enrollment trends prior to the A+ Schools Program. We excluded schools in the data which were not eligible to become A+ schools, such as continuation high schools and correctional centers. In total, we examined data from 500 high schools in Missouri throughout this eighteen-year period.
Table 1 contains the annual college-going rates of A+ designated schools compared with non-A+ designated schools. Counts from this table show the increasing trend of students graduating from A+ designated schools making up larger shares of Missouri high school graduates. By 2003, more students graduated from A+ designated schools than non-A+ schools, even though there were fewer A+ designated schools statewide. Regarding postsecondary enrollment, we noted no descriptive difference between A+ and non-A+ schools for enrollment at four-year colleges. However, for two-year colleges, we noted an annual difference of approximately 10% more students from A+ designated schools enrolling at two-year colleges than students from non-A+ schools. Albeit descriptive, the differences in four-year and two-year rates between A+ and non-A+ designated schools support our hypothesis of the diversionary effect of the A+ Schools Program.
As part of a state accountability requirement, each school district in Missouri is required to follow-up after 180 days with their recent graduates to determine their post-high school education and career outcomes. School districts conduct follow-up studies to track graduates’ postsecondary activities, including if they were: (a) attending a four-year college; (b) attending a two-year college; (c) not attending college; or (d) had joined the military. These enrollment rates also account for any student attending an institution in another state. To measure changes in postsecondary enrollment based on a school’s A+ designation, we used the percentage of the high school’s graduates that enrolled in any post-secondary institution, whether two-year institutions, four-year institutions, or other postsecondary program. [End Page 812]
We used state administrative data to establish the year a high school became A+ designated. By reviewing annual enrollment, we determined the first year students graduated A+ eligible from a specific high school and flagged the school as A+ designated. This A+ designation dummy variable was set to 0 then switched to 1 in the year A+ eligible graduates were identified at the school. We used state administrative financial [End Page 813] grant data from the Missouri Department of Higher Education (MDHE) to cross reference our designation flags and verify the year a high school became A+ designated.
Figure 1 highlights the growth in the number of A+ eligible high school graduates by year. In 2010, 78% of all Missouri public high school students graduated from an A+ designated high school. However, it is important to acknowledge that not all students graduating from an A+ eligible high school are eligible for the A+ scholarship, and not all A+ eligible graduates use the tuition assistance. Of the students graduating from A+ designated schools in 2010, only 36% were eligible to receive an A+ scholarship. Also, in addition to the new schools being A+ designated, the growth of A+ eligible graduates was also a result of growth within schools occurring as the A+ Schools Program becomes engrained within the school and its students. Nonetheless as this figure demonstrates, the proportion of students with the opportunity to be eligible and utilize the A+ scholarship increased substantially since its inception.
We use data from the National Center for Education Statistics (NCES) to control for important variables related to postsecondary enrollment. Using the Common Core of Data, we collect school-level demographic data including the total number of students, percentage of free lunch recipients, and percent of minority student enrolled. We chose to use free lunch recipients instead of free and reduced lunch recipients because reduced lunch data was not available for all schools prior to 1996. We believe free lunch recipients to be a valid proxy for the low-income student composition at a school. Lastly, school-level fixed effects control for all time-invariant heterogeneity across schools.
Research Design and Methodology
The potential statistical biases associated with estimation of a treatment effect, such as the opportunity to participate in the A+ Schools Program, on student outcomes are well documented in the literature (e.g., Cellini, 2008; Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007). Essentially, estimation bias is likely to occur if something unobservable to the researcher is correlated with both the outcome of interest and the exposure to treatment. In the case of the A+ Schools Program, this is likely to occur as both a student’s opportunity to participate in the program, and their choice to participate in the program, are not randomly distributed and most likely correlated with both observed [End Page 814] and unobserved student characteristics (e.g., socioeconomic status, academic merit, and motivation).
For a researcher, the ideal research design to estimate the causal effects of a particular program would be a randomized control trial where participation in the program was randomly assigned either to schools or students within a school. By definition, random assignment would make the unobserved factors related to the outcomes of interest uncorrelated with the treatment status. Unfortunately, in general and more specifically in our case, large scale randomized control trials of state policy are often infeasible. The need-based grant experiment in Wisconsin is a rare example of a large-scale randomized financial aid study conducted at the state level (Goldrick-Rab, Harris, Kelchen, & Benson, 2012).
While a random control trial was unavailable, the staggered adoption of the A+ Schools Program across Missouri high schools allowed for a quasi-experimental research design. Specifically, the comparative interrupted time-series design estimated the effect of the A+ Schools Program as the deviation of the observed outcome from a predicted outcome (given the outcome’s trend in previous periods) and compares these deviations with a comparison group not affected by the policy change (Shadish, Cook, & Campbell, 2002).
In Missouri, as high schools adopted the A+ Schools Program across different years, we utilized the differential timing of treatment to minimize the threat of alternative unobserved factors which may have affected the outcome at the point of policy adoption (Dynarski, 2004; Shadish, Cook, & Campbell, 2002). This design allowed later adopting treatment high schools to serve as comparisons for earlier adopting high schools. Thus, the staggered introduction of the A+ Schools Program across high schools in Missouri allowed the treatment point to be separated from other time-invariant effects across the research period which minimized the history threat to internal validity.
Following the work of Dynarski (2004), we used a staggered introduction difference-in-difference estimation strategy to estimate the effects of the introduction of the A+ Schools Program on the postsecondary enrollment rates of students. Specifically, we estimated the following school-level model:
Where, Yjt represents a school-level measure of college-going for school j at time t (i.e., total, two-year, four-year). The indicator variable, APlusjt represents whether the particular school is designated as [End Page 815] A+ in year t. School-level fixed effects (δj) are included to control for time-invariant school-level heterogeneity. Year fixed effects (δt) are included to control for school-invariant changes in college-going rates in Missouri, such as changes in state-level policy. We also included a vector of time-varying school-level characteristics (i.e., free-lunch recipients, minority student enrollment) to help control for school-level changes likely to affect college-going rates (Sjt). Because of the potential serial correlation problem associated with difference-in-difference estimation, which can result in the underestimation of standard errors of the estimators, we follow the recommendations of Bertrand, Duflo, and Mullainathan (2004) and cluster our standard errors at the school level.
We examined the difference between college-going rates within high schools before and after A+ designation. By including school and year fixed effects and other high schools in the state, we established a quasi-experimental counterfactual scenario. That is, what would happen to trends in postsecondary enrollment at Missouri high schools in the absence of the adoption of the A+ Schools Program?
The primary assumption with difference-in-difference estimation is that that pretreatments trends are parallel for all observations in the sample, regardless of whether or not they received treatment (Shadish, Cook, & Campbell, 2002). This assumption is important for our analyses as it ensures that the counterfactual scenario (i.e., postsecondary enrollment trends for schools that are not A+ eligible) provides a valid means of comparison. As this assumption cannot be directly tested, we provide illustrations to help support the validity of this assumption. Figures 2 (two-year) and 3 (four-year) illustrate the average college-going rates by year for schools that became A+ schools in 1997 and those that never became an A+ school or those that did not become A+ before 20024. Figure 2 illustrates that the two-year enrollment was increasing prior to 1997 at similar rates for both schools that became A+ schools and those that did not. After adoption of the A+ Schools program, the two-year enrollment rate increased over the next 5 years relative to schools that did not adopt. In contrast, Figure 3 shows that four-year enrollment was decreasing for both the schools that adopted the A+ program in 1997 and those that did not. Overall, the figures indicate that for the sample of schools that adopted A+ in 1997, their college enrollment trends were similar to schools that did not adopt prior to 1997.
In addition to the visual argument, our study was limited to schools in one state and therefore exposed to the same state-level policies or shocks that can uniformly impact postsecondary enrollment. Additionally, the year fixed effects in our study captured the potential year-to-year changes in statewide influences on postsecondary enrollment in the [End Page 816]
[End Page 817]
state that were independent of school characteristics. Second, although within the same state, schools in our sample could differ in structure and have access to different resources. By using school fixed effects, we teased out the effects of these time invariant differences in schools from our estimation of the A+ Schools Program’s influence on postsecondary enrollment. Finally, the staggered adoption of the A+ program allowed schools who became A+ Schools at later time periods to be comparison schools for early adopters, which minimized the bias associated with the fact that the schools most likely to benefit from the A+ programs may have been the ones that were most likely to adopt (Shadish, Cook, & Campbell, 2002).
Overall Postsecondary Enrollment
Results from the estimation of
This result supports our argument that the A+ Schools Program increased postsecondary enrollment for high school graduates. Our analysis occurred at the school level and did not distinguish between whether students participated or did not participate in the A+ Schools Program. Thus, our results should be interpreted as an average intent to treat effect of the Missouri A+ Schools Program and not an average treatment effect when thinking about the program’s effects on individual students.
After A+ designation, students are exposed to the benefits of increased access to academic support and resources. In line with existing research on postsecondary education behaviors of high school students, our results support the arguments that access to increased capital (Engberg & Wolniak, 2010; Perna & Titus, 2005) and academic preparation (Cabrera et al; De La Rosa, 2006) have positive implications for postsecondary enrollment. Furthermore, we can conclude that the A+ Schools Program did indeed accomplish its goal of increasing number of Missouri high school graduates pursuing a postsecondary education. [End Page 818]
Similar to total postsecondary enrollment, we found a positive effect on enrollment at two-year colleges. In fact, the effect for two-year postsecondary enrollment was more than three times greater than that of total postsecondary enrollment. A+ designation for a school resulted in an increase in the two-year enrollment rate of 5.2 percentage points, or a 31.0% increase in the two-year college-going rate based on the two-year enrollment rate of 16.8% for non-A+ schools in 1997.
The results of the two-year enrollment model were not surprising given the direct financial incentive provided by the A+ Schools Program. While designed to increase postsecondary enrollment in general, our results suggest that the A+ scholarship is enough to encourage students to enroll at two-year institutions at higher rates by increasing [End Page 819] the perceived utility of a two-year college through the reduction of the net cost of attendance. This is indicative of students meeting the A+ requirements choosing to take advantage of their A+ scholarship and reflects the persuasive power of grants on influencing students’ postsecondary enrollment decisions (Kim, 2004).
We hypothesized that the A+ Schools Program would have an ambiguous effect on enrollment at four-year institutions. However, results from our analytical model indicate that A+ designation had a negative effect on four-year enrollments. We found that A+ designation resulted in a decrease in the four-year enrollment rate of 3.7 percentage points, or a 10.6% decline in the four-year college-going rate on a base year enrollment rate of 34.7% for non-A+ schools.
The four-year enrollment findings indicated the A+ Schools Program lead to decreased enrollment at four-year institutions and may be indicative of the purchasing power of the A+ scholarship, which lowered the relative cost of attending a two-year versus a four-year institution. The concerns of financing postsecondary education may be so intense that students eligible for admission to four-year institutions would rather attend a two-year institution for relatively low to no cost. Recent increases in the cost of attendance at public postsecondary institutions can further exacerbate the notion that a postsecondary education is expensive, resulting in more students opting for two-year institutions.
Dynamic Effects Specification
Coefficients from our base models provide estimates of the average intent-to-treat effect of A+ designation schools, but provide no indication of the dynamic effects the A+ Schools Program has on postsecondary enrollment. Since the benefits of the A+ Schools Program are abruptly introduced for students upon their school’s eligibility, we would expect an immediate effect in postsecondary enrollment, but would also expect this to intensify as the program becomes ingrained within the school’s culture and more students become eligible. Following Autor (2003), we ran the following model with lead and lagged year variables relative to the A+ designation to further examine the timing of effects:
Where, in addition to the parameters from
With regards to two-year college enrollment, there was an immediate positive effect that increased in magnitude with each subsequent year (Table 3). We found an average increase of 1.6 percentage points the first year of designation, which increased to 5.9 percentage points two years after designation, and dropped to 5.0 percentage points three years after. On the other hand, the four-year enrollment rates continuously decreased over time after A+ designation. Four-year college enrollment was unchanged in the year of designation, decreased by 4.4 in the year after designation, and decreased by 5.5 percentage points in years four years and beyond. We found no significant timing effects for overall college going rates.
While the abrupt introduction of the A+ Schools Program can affect college enrollment rates, results of our dynamic effect specifications show that the overall effects occur gradually as the components of the A+ Schools Program are engrained within a school. Additionally, this specification also supports the assumption of parallel trends as there was not a significant difference in college going rates prior to A+ adoption. We believe the lack of any significant effect on overall college enrollment in subsequent years after A+ designation to be indicative of the A+ scholarship’s impact. By design, the A+ Schools Program is intended to encourage two-year postsecondary enrollment, and in doing so, disrupts four-year enrollment rates. The absence of a long-term effect on overall enrollment is potentially neutralized by this relationship along with trends in schools that are never A+ designated.
To examine for differential effects of the A+ Schools Program on income and race, we ran two additional models where we interact the variables representing the percentage of underrepresented minority students and the percentage of free lunch students with A+ designation to explore heterogeneous effects of the A+ Schools Program (Table 4). We found no significant effect of free lunch recipient student composition for any of the three college-going rates.
For the race specification, we combined the Black, Latino, and American Indian students to create a measure of the percentage of underrep-resented minority students in a school. In this specification, we found an opposite effect from our main findings. We note no effect on the overall college-going rates, but do find that two-year rates decrease by 0.96 and four-year rates increase by .06 percentage points for each 1 percentage point increase in the underrepresented student population in a school. [End Page 821] One plausible explanation could be that the A+ Schools Program’s components provide underrepresented minority students with the support to seek out additional resources, such as scholarships, to minimize the net cost of a four-year college.
[End Page 822]
To test the validity and further examine our results, we conducted two robustness checks. First, we ran the model with only A+ designated high schools to verify that undesignated high schools were not biasing results. Secondly, to test for the possibility that our findings stem from factors not related to the A+ Schools Program, we examined postsecondary enrollment while randomly assigning treatment to schools. Each is discussed in further detail in this section.
[End Page 823]
Only A+ Designated Schools
In our prior analysis, we included schools that never became A+ designated as part of the comparison group. To check if this potentially confounded our findings, we limited our sample to only include high schools that were designated as A+ schools. Using this model specification (Table 4), we found nearly identical results as our previous models. Particularly, we found that overall college-going rates increased by 1.7 percentage points after A+ designation. Likewise, we found a similar substitution effect between two- and four-year college-going rates. Specifically, the models estimated a 5.1 percentage point increase in two-year college enrollment and a 3.4 percentage point decrease in four-year college enrollment.
Random Assignment of Treatment
As noted earlier, unobserved variables and trends introduce the potential for bias in nonrandomized control estimations. To test the possibility of our results being indicative of an unobserved statewide trend affecting postsecondary enrollment and not the A+ Schools Program, we randomly assigned treatment (i.e., A+ designation) to schools across time and ran the same estimation models. Designation of treatment was done in direct proportion to the number of schools that actually received designation for a given year (e.g., in 1998, 28 schools became A+ designated in our robustness and actual models). Results from these models showed no significant results for random A+ designation on any of our dependent variables (Table 4). The lack of a significant effects indicates that the results from our initial estimation model are indeed reflecting changes in postsecondary enrollment in response to the introduction and stabilization of the A+ Schools Program in high schools, and not merely changes related to external factors.
Limitations of the Findings
As with most empirical studies, ours was not without its limitations. Previous research documents heterogeneous effects in student responses to the availability of financial aid across race, gender, and socioeconomic status (see for example Cabrera & La Nasa, 2001). However, given the aggregate nature of our data, we were not able to directly examine how students from different groups respond to the A+ Schools Program. While we were able to control for some of these variables at the school level (e.g., percent of students receiving free lunch), we did not observe what percentage of students from these different groups were enrolling in postsecondary institutions upon graduation. Consequently, we were not able to infer which group of students is benefiting more, or less, from the A+ scholarship. [End Page 824]
As mentioned prior, each school district in Missouri is required to follow-up after 180 days with their recent graduates to determine their post-high school education and career outcomes as part of a state accountability requirement. While schools met this requirement, there is always potential that data were not collected from all students. Furthermore, because some of the data were self-reported by students, there was also the possibility that they misreported their post-high school status. Both of these scenarios had the potential of introducing bias into our estimates. Although we believe this concern to be minimal, it is still important to mention when considering the implications of our results.
Discussion and Conclusion
The A+ Schools Program’s multifaceted mechanisms provide a comprehensive and relatively unique approach towards improving postsecondary opportunities for Missouri high school students. However, the complexity of its design makes the A+ Schools Program’s overall effectiveness not fully understood as it does not fit within the structure of most traditional financial aid programs. Findings from our study echo those of the most similar state funded financial aid programs. Overall, we conclude that the A+ Schools Program had a positive effect on total postsecondary enrollment. Furthermore, and perhaps more noteworthy, we found very different effects when we disaggregated our data and looked at two- and four-year enrollment rates independently.
In her examination of two-year colleges on educational attainment, Rouse (1995) tested two patterns of enrollment related to two-year colleges, the democratization and diversion effects. The democratization effect posits that two-year colleges increase overall postsecondary enrollment by providing educational opportunities for individuals not prepared to attend a four-year institution. Thus, the diversion effect results when community colleges draw students that would have otherwise attended a four-year institution. Results from our analysis of the A+ Schools Program are indicative of both a democratization and diversion effect as the A+ designation of schools increased overall enrollment, mostly driven through increased enrollment in two-year colleges. Thus, the decrease in four-year enrollment and increase in two-year enrollment may be indicative of a diversion effect encouraging students to attend a two-year institution instead of a four-year institution.
Although the decrease in four-year enrollment may initially be deemed as a negative consequence of the A+ Schools Program, the democratization effect may have greater implications for the state as a whole. Increases in two-year enrollment outweighed the decreases in [End Page 825] four-year enrollment and may be indicative of the program broadening access by incentivizing and providing the financial means to students with no prior plans to pursue a postsecondary education. Hilmer (1998) found that the types of institutions students choose is influenced by their relative net-prices. Considering this relationship, the A+ Scholarship minimizes this concern, and consequently increases the perceived utility of a two-year college for students, especially low-income students whom may have not attended without the award. Additionally, by facilitating access to a two-year college, the A+ Schools Program provides a relatively low-risk opportunity to pursue a postsecondary education for students unsure of their capacity to do so (Hilmer, 1998; Leigh & Gill, 2004).
A potentially negative consequence of diverting students from a four-year to a two-year institution is the added challenge of transferring to a four-year institution for those students desiring to receive a bachelor’s degree. Studying bachelor attainment rates in Ohio, Long and Kurlaender (2009) found that students beginning at a two-year college were 14.5% less likely to complete a bachelor’s than those beginning at a four-year institution. Considering this relationship, the attractiveness of enrolling in a two-year college and using the A+ scholarship can potentially complicate plans for students interested in a bachelor’s degree who had the option to enroll in a four-year institution immediately after high school. However, the relief of the financial stress of paying for their education provided by the A+ scholarship, coupled with increased academic preparation via the academic requirements needed to be eligible for the award, can alleviate the challenge of transferring for those students looking to continue their education at a four-year institution.
While starting at a two-year college may lower overall educational attainment, the increase in students participating in postsecondary education, even if brief, translates into more students advancing their human capital (Rouse, 1998). This has long-term benefits for the state as it increases the overall educational attainment, and consequently the economic productivity of its citizens, via increased wages, decreased unemployment, and increased tax revenue (Sparks, 2011; Trostel, 2009).
The broadening of postsecondary access for high school graduates remains a priority. Therefore, it is imperative that we fully examine and understand the impact of current access programs to not only improve them, but also inform the creation and development of future programs and policies. While much is known about the implications of traditional college access and financial aid programs, little evidence exists on the impact of hybrid programs, like the A+ Schools Program. Our study contributes to this body of literature by addressing this limitation and [End Page 826] informing scholars, as well as policy makers, of the true effectiveness of programs such as the A+ Schools Program. By studying the A+ Schools Program’s impact on two- and four-year enrollment, we provide evidence to suggest that programs requiring students’ early commitment to succeed and remain focused in school do indeed have positive effects on postsecondary enrollment.
José Muñoz is a Doctoral Candidate in Educational Leadership and Policy Analysis at the University of Missouri; firstname.lastname@example.org.
James R. Harrington is an Assistant Professor of Public and Nonprofit Management at the University of Texas at Dallas.
Bradley R. Curs is Associate Professor in the Department of Educational Leadership and Policy Analysis and the Truman School of Public Affairs at the University of Missouri.
Mark Ehlert is Associate Research Professor in the Department of Economics at the University of Missouri.
1. The program is now referred to as the A+ Scholarship Program. However, given the period of our analysis (1992–2010) and our study’s focus, we felt it most appropriate to refer to the program by its original name.
2. The A+ Scholarship covers the unpaid balance of tuition and general fees after all non-loan federal financial aid (e.g., Pell grant) are applied to a student’s account. Each year, the maximum tuition amount eligible for reimbursement is based on the standard per credit hour tuition rate at Linn State Technical College.
3. To receive the A+ scholarship, eligible students must not be pursuing a degree in theology or divinity.
4. The staggered introduction of the program makes it difficult to illustrate pre- and post-trends for all A+ schools and a valid comparison group. To address this, we choose the schools that became A+ schools in the first year of the program and compare them with non-A+ schools in the subsequent 5-year period.