Value Added for Principals?

Graduate students of UCEA member institutions enter a wide variety of professions upon graduation. Some enter the professoriate, others the public school system, the policy world, and so on. The world of educational administration is vast, as is the circumference of our collective research. That being said, there are few of us who will not be interested in the evolution of principal evaluation.

The purpose of this column entry is to expose graduate students to modern research exploring value added measures of principal effectiveness. By now we are all familiar with value added modeling (VAM) for classroom teachers, but few are knowledgeable of the inner-workings of VAM for principals. The remainder of this column begins with a brief description of the extant research on principal effects and concludes with a request for reader feedback on which article or authors you’d like to see interviewed in this space.

Empirical approaches to quantifying principal contribution to student achievement are not entirely new. According to my literature review, the first attempt to quantify principal impact on student achievement was done by Eberts & Stone (1988). Using data from over 14,000 students across 300 school districts, the authors regressed 4^th grade mathematics achievement upon a selection of principal characteristics, controlling for prior achievement and a variety of other school and classroom factors. Amongst other conclusions, the authors found that principal involvement with school instruction (instructional leadership) was significantly associated with higher gains in student achievement.

Brewer (1993) sought to confirm the findings of Eberts & Stone with a data set of 2,070 high school students containing math and reading achievement data along with information about the schools and administrators to whom the students are tied. Brewer conducted similar regressions (although modeled to include additional factors) and concluded that principals influence student achievement through teacher selection and goal-setting (human resource leadership and vision). Interestingly, Brewer found that neither principal experience as a teacher or administrator is significantly related to student achievement.

The past few years have seen an explosion in research around principal contributions to student achievement. Most studies look at principals who move across multiple campuses and try to determine the effect those administrators had on student achievement during their stay. Some of these works have created value added estimates for individual administrators. Below are citations for each of the articles I’ve reviewed, along with a very brief description of their content and findings.

Branch, G. F., Hanushek, E., & Rivkin, S. G. (2010). Estimating Principal Effectiveness. Working Paper 32. National Center for Analysis of Longitudinal Data in Education Research. Retrieved from http://www.caldercenter.org/publications.cfm

In this CALDER center  working paper, Branch, Hanushek, & Rivkin use Texas data to explore principal contributions to school-wide achievement. Additional factors explored include the relationship between principal quality and student absenteeism. The authors find that variation in principal quality is greatest in low-income schools and that principal mobility between schools is most common for individuals with the highest and lowest value added measures.

Clark, D., Martorell, P., & Rockoff, J. (2009). School Principals and School Performance. Working Paper 38. National Center for Analysis of Longitudinal Data in Education Research. Retrieved from http://www.caldercenter.org/publications.cfm

In contrast to Brewer (1993) this CALDER publication finds that principal experience is positively correlated with student achievement. Using New York City administrative data, the authors use information about principals' educational background, training, and work experience in addition to a variety of school and student variables to calculate principal value added measures. The authors make the argument that the sorting of less-experienced administrators into already low-achieving schools can further expand educational inequity.

Coelli, M., & Green, D. (2009). Leadership Effects: School Principals and Student Outcomes. Preliminary paper. Retrieved from http://sites.google.com/site/mickcoelli/academics

This paper explored the relationship between principal effectiveness and the academic achievement and graduation rate of high school seniors in British Columbia. The authors conclude that principal effects are stronger with regards to academic achievement (as measured by a standardized ELA exam) than for graduation rates. One interesting finding from the article is that principal effects take time to show up in achievement data. In other words, it takes time for a principal to reorganize the school to match her vision.

Dhuey, E., & Smith, J. (2010). How Important Are School Principals in the Production of Student Achievement? Preliminary paper. Retrieved from http://homes.chass.utoronto.ca/~edhuey/index.php/home/page/research

This is another study using administrative data from British Columbia. Here the authors find significant, measurable principal effects on student achievement in both math and reading at the elementary level. The authors suggest that a one standard deviation increase in principal quality relates to a roughly 0.2 standard deviation increase in student performance.  In this study, the authors find no relationship between principal experience and principal value added.

Lipscomb, S., Teh, B., Gill, B., Chiang, H., & Owens, A. (2010). Teacher and Principal Value-Added: Research Findings and Implementation Practices. Mathematica Policy Research, Inc.!

This publication from Mathematica research explores the difficulties associated with assigning value added measures to principals. For an excellent review of various methdological approaches to VAM for both teachers and principals, look no further than this publication.

One piece of research recently brought to my attention is a working paper by Cannon, Figlio, and Sass (2011). Their works is particularly fascinating in that it explores the effect of principal policy choices on student achievement. Their data set contains three periods of survey responses over five years, which, “asked principals to identify a variety of policies and resource-use areas along a variety of lines, including policies to improve low-performing students, lengthening instructional time, reduced class size for subject, narrowing of the curriculum, scheduling systems, policies to improve low-performing teachers, teacher resources, teacher incentives, and teacher autonomy information” (p. 7). Tying survey response data to achievement data has allowed to the authors to match principal policy choices to changes in principal value added measures. The paper is very detailed in its consideration of assumptions and explanations of the statistical models employed. I highly suggest you check it out.

Educational researchers and UCEA have long been interested in the effects that principals have on student achievement. The vast majority of the works cited in this post represent approaches to understanding principal effects from outside schools of education; work primarily done by economists. Future research on principal effects will depend on scholars to have a firm foundation in the deep, established knowledge base constructed by the tireless efforts of Leithwood, Hallinger, Heck, and Murphy (to name only a few), but also a clear understanding of the methodology employed by researchers at the helm of modern educational policy development, whom increasingly are not coming from schools of education.

Finally, giving credit where credit is due, I would like to thank Dr. Li Feng of Texas State University in San Marcos for bringing the Cannon, Figlio, & Sass paper to my attention. It was the Cannon, Figlio, & Sass piece that brought to my attention the work of Clark, Martorell, & Rockoff (2009) as well as another article I have elected not to discuss in this space (Beteille, Kalogrides, & Loeb, 2009); therefore I wish to thank Sarah Cannon, David Figlio, & Tim Sass for their contributions to the above literature review.

References:

Beteille, T., Kalogrides, D., & Loeb, S. (2009). Effective Schools: Managing the recruitment, development, and retention of high-quality teachers. Working Paper 37. National Center for Analysis of Longitudinal Data in Education Research. Retrieved from http://www.caldercenter.org/publications.cfm

Branch, G. F., Hanushek, E., & Rivkin, S. G. (2010). Estimating Principal Effectiveness. Working Paper 32. National Center for Analysis of Longitudinal Data in Education Research. Retrieved from http://www.caldercenter.org/publications.cfm

Brewer, D. J. (1993). Principals and student outcomes: Evidence from U.S. high schools. Economics of Education Review, 12(4), 281-292. doi:10.1016/0272-7757(93)90062-L

Cannon, S., Figlio, D., & Sass, T. (2010). The Policy Choices of Effective Principals. preliminary paper. Retrieved from http://myweb.fsu.edu/tsass/Papers/

Clark, D., Martorell, P., & Rockoff, J. (2009). School Principals and School Performance. Working Paper 38. National Center for Analysis of Longitudinal Data in Education Research. Retrieved from http://www.caldercenter.org/publications.cfm

Coelli, M., & Green, D. (2009). Leadership Effects: School Principals and Student Outcomes. Preliminary paper. Retrieved from http://sites.google.com/site/mickcoelli/academics

Dhuey, E., & Smith, J. (2010). How Important Are School Principals in the Production of Student Achievement? Preliminary paper. Retrieved from http://homes.chass.utoronto.ca/~edhuey/index.php/home/page/research

Eberts, R. W., & Stone, J. A. (1988). Student achievement in public schools: Do principals make a difference? Economics of Education Review, 7(3), 291-299. doi:10.1016/0272-7757(88)90002-7

Lipscomb, S., Teh, B., Gill, B., Chiang, H., & Owens, A. (2010). Teacher and Principal Value-Added: Research Findings and Implementation Practices. Mathematica Policy Research, Inc.