July 1, 2004
Education is a cumulative process. Yet while students' knowledge and skills are built up over time, educational researchers are rarely afforded the opportunity to examine the effects of interventions over multiple years. This study of the America's Choice school reform design is just such an opportunity. Using 11 years of student performance data from Rochester, NY -- which includes several years of data before America's Choice began working in the district -- we examine the effects of America's Choice on student learning gains from 1998 to 2003. Employing a sophisticated statistical method called Bayesian hierarchical growth curve analysis with crossed random effects, we compare the longitudinal gains in test performance of students attending America's Choice schools to those of students attending other Rochester schools. Our analytical method allows us to examine student test performance over time, account for the nested structure of students within schools, and account for the very real problem of within-district student mobility.Through these analyses, we sought to answer three central questions. First, is there evidence that America's Choice increases students' rates of learning and, if so, how big is the increase? Second, does America's Choice improve the performance of particularly lowachieving students? And third, does America's Choice make education more equitable for minority students?Overall, we found that students in America's Choice schools gained significantly more than did students in other Rochester schools in both reading and mathematics test performance. These differences are moderate in the early-elementary grades (grades 1-3) and stronger in later grades (grades 4-8). In the early-elementary grades, students in America's Choice schools averaged three weeks of additional learning per year in comparison to students in other district schools. In grades 4-8, students in America's Choice schools averaged slightly more than two months of additional learning per year in comparison to students in other district schools.