Table of Contents

Misconception: PVAAS reporting is not reliable or valid since it is based only on the PA state assessments.

Educators might be concerned that value-added reporting relies on the use of standardized tests, which have limitations themselves. Perhaps they feel that the test does not correlate well with the curriculum or that there isn't sufficient stretch to measure progress of students with students with very low achievement histories or those with very high achievement histories. However, PVAAS estimates use a sophisticated modeling approach to address many of the concerns of using standardized tests, and PVAAS reviews the test scores annually to ensure that they are an appropriate use for PVAAS value-added reporting.

PVAAS in Theory

Student test scores are the basic ingredient of all PVAAS analyses. PVAAS is not involved in and has no control over test construction. Pennsylvania's assessment system performs a universal assessment of Pennsylvania standards, and the assessments are aligned to the appropriate standards that are sufficient for longitudinal modeling and prediction. Regardless, before using any tests in PVAAS modeling, rigorous data processing and analyses verify that the tests meet the following three criteria. :

  • They demonstrate sufficient stretch so that students with a history of lower achievement and those with a history of higher achievement can show growth.
  • They are aligned to state curriculum standards.
  • The scales are sufficiently reliable from year to year.

To date, Pennsylvania's state assessments have met these criteria. More specifically, PVAAS analyses verify that there are enough different scale scores at the top and bottom of the scales to differentiate student achievement. This processing also analyzes the percentage of students scoring at the top and bottom scores to ensure there are no ceilings or floors. After all analyses are completed and PVAAS growth measures are available, PVAAS verifies that LEAs/districts, schools, and teachers serving students with histories of higher achievement can show both high and low growth. This process is repeated every year.

Another common concern of educators is that they might be held accountable for how students did on a single test on a given day. PVAAS understands this concern and agrees that any single score represents only a snapshot of student performance at a particular point in time. However, the use of many test scores across subjects, grades, and years in PVAAS can provide a more complete picture of student learning and how students' achievement has changed over time.

PVAAS in Practice

Actual data might be the most readily apparent way to show that there is sufficient stretch in Pennsylvania's state assessments to measure the growth of students with histories of lower, middle, and higher achievement. The figure below plots the average achievement for each teacher's students in Pennsylvania against the growth index (the value-added growth measure divided by its standard error) for PSSA Mathematics in grade 5 in 2019. Each dot represents one teacher, and verified rosters were used where available. The figure demonstrates that teachers serving students with histories of higher achievementstudents with both higher and lower achievement can show both high and low growth, as measured by PVAAS. In the graph below, the actual correlation between the growth index and entering achievement is 0.15, which is weak. LEA/District and school value-added plots are similar to the teacher plot shown below.