Improving Mathematics and Science Education: A Longitudinal Investigation of the Relationship Between Reform-Oriented Instruction and Student Achievement

by Vi-Nhuan Le, Brian M. Stecher, J.R. Lockwood

The term reform-oriented teaching describes a collection of instructional practices that are designed to engage students as active participants in their own learning and to enhance the development of complex cognitive skills and processes. This monograph presents the findings of a multiyear National Science Foundation (NSF)-funded study of the effectiveness of reform-oriented science and mathematics instruction. It builds on an earlier RAND study, called the Mosaic project, which found "a weak but positive relationship" between reform-oriented practices and student achievement. The present study, called Mosaic II, extends this earlier research in two important ways. First, it incorporates more-diverse indicators of student exposure to reform-oriented practices, including innovative, vignette-based measures. Second, it follows students for three years in order to measure the relationship after longer exposure to reform practices. Mosaic II was designed to answer two major research questions: * Is the use of reform-oriented instructional practices in mathematics and science associated with higher student achievement? * Is the relationship between reform-oriented practices and achievement sensitive to the aspects of achievement that are measured? The research was conducted in three districts that participated in the NSF Local Systemic Change program, although the study is not an evaluation of the implementation or impact of that specific program. We found nonsignificant or weak positive relationships between reform-oriented instruction in mathematics and science and student achievement measured using multiple-choice tests. The results also reinforce the message that measurement matters--i.e., the observed relationship between reform-oriented instruction and achievement may depend on how achievement is measured. It is common practice to use existing state or district tests as measures of program effectiveness, because it is often not feasible to administer additional tests. Our analysis indicates that this decision may influence findings. Results should be of interest to educators and policymakers concerned with improving mathematics and science education. Background details on and tabulated results of this study are included in supporting appendixes on a CD-ROM that is included with the printed version of this document.

116 pages, ebook

First published September 1, 2006

About the author

Vi-Nhuan is a principal research scientist in the Education & Child Development department. She has considerable experience researching issues related to mathematics and science reform, educational assessment, postsecondary access and persistence, and early childhood education. Her past projects have included examining the relationship between receipt of a preschool tuition credit and achievement outcomes at kindergarten and beyond, exploring the impact of merit pay on mathematics teachers’ attitudes and instructional practices, and studying the effects of participation in the College Bound readiness program on student test performance and college attendance.
Vi-Nhuan has particular expertise in quantitative analysis of large-scale databases and has published multiple studies that make use of district, state, and national datasets. She is well-versed with advanced statistical approaches, including meta-analysis, propensity weighting, and regression discontinuity designs. Her work can be found in American Educational Research Journal, Early Childhood Research Quarterly, Educational Evaluation and Policy Analysis, Developmental Psychology, The Journal of Applied Developmental Psychology, Journal of Educational Measurement, and Research in Higher Education. She has previously received funding from the Institute of Education Sciences, National Institute of Health, Walton Family Foundation, Ewing Marion Kauffman Foundation, and Hewlett Foundation. She holds a doctorate degree in psychometrics, a master’s degree in statistics, and a bachelor’s degree in psychology.