Misconception: Teacher value-added estimates are not reliable enough to be used in high-stakes decisions.
Many studies on teacher estimates focus on single-year estimates, some of which are derived from simplistic value-added or growth models. However, PVAAS teacher value-added estimates are based on a robust statistical approach and report a multiple-year average whenever available. The approach provides very reliable teacher estimates, which educators can use for a variety of educational and policy decisions.
PVAAS in Theory
Many critics use the repeatability of teacher value-added estimates as a proxy for their reliability. However, "perfect" repeatability is not the goal as some year-to-year variation among individual teachers' estimates is to be expected. Cohorts of students change every year, and teachers might be more effective with one group than another. Also, some teachers might increase or decrease in their effectiveness over time. However, the presence of strong reliability indicates that teachers' value-added estimates are related to their consistent skills and are not generated primarily from a random component.
PVAAS reviewed value-added estimates over the past two decades and found that:
- Teachers with high value-added are likely to continue yielding high value-added. Teachers identified as having students who exceed expected growth after their first three years of teaching were extremely likely to have similar growth with their students three years into the future (about 95% were either average or above average in their students' growth).
- Teachers with lower value-added might improve over time. For the teachers identified as having students who do not meet expected growth based on three-year estimates, approximately half of them will continue to have students with similar growth three years later.
This has enormous implications in terms of the usefulness of the reporting provided by PVAAS such that educators and policymakers can rely on the teacher estimates to inform their decisions.
PVAAS in Practice
In 2019, less than 1% of teachers (.03% or nine teachers) moved from the highest growth designation (based on significant evidence of exceeding the growth standard) to the lowest growth designation (based on significant evidence of not meeting the growth standard) from 2017-18 to 2018-19 based on the teachers' composite value-added measures. In all individual subjects and grades, less than 1% move from the highest growth designation to the lowest growth designation.
In other words, in using a robust and reliable statistical approach, like PVAAS, for teacher estimates, Pennsylvania educators and policymakers can build insightful policies customized to the teachers in their schools, LEAs/districts, and state.
The 2013 Measures of Effective Teaching (MET) study raised the same question of whether value-added data, in conjunction with other metrics like observational studies, could be used for high-stakes decisions and it concluded:
The answer is yes, not because the measures are perfect (they are not), but because the combined measure is better on virtually every dimension than the measures in use now. There is no way to avoid the stakes attached to every hiring, retention, and pay decision. And deciding not to make a change is, after all, a decision. No measure is perfect, but better information should support better decisions.||
The MET study went on to encourage data practices to improve each measure, such as roster verification for student teacher linkages and multi-year averages of teacher estimates, both of which are used for PVAAS teacher value-added reporting.