PVAAS

Misconception: PVAAS is based on a "black box" methodology.

The PVAAS methodologies and algorithms are published and have been in the open literature for almost
20 years. For those interested in learning more about the statistical models used in EVAAS reporting, the following references are useful:

• On the SAS EVAAS Statistical Models specific to PVAAS: SAS Institute Inc. SAS® EVAAS Statistical Models and Business Rules of PVAAS Analyses (Available online at https://pvaas.sas.com/support/PVAAS-Technical-Documentation.pdf).

• On the Tennessee Value-Added Assessment System: Millman, Jason, ed. Grading Teachers, Grading Schools: Is Student Achievement a Valid Evaluation Measure? Thousand Oaks, CA: Corwin Press, 1997.

PVAAS in Theory

While PVAAS reporting benefits from a robust modeling approach, this statistical rigor is necessary to provide reliable estimates. More specifically, the PVAAS models attain their reliability by addressing critical issues related to working with student testing data, such as students with missing test scores and the inherent measurement error associated with any test score.

Regardless, the PVAAS modeling has been sufficiently understood such that value-added experts and researchers have replicated the models for their own analyses. In doing so, they have validated and reaffirmed the appropriateness of the PVAAS modeling, and many of the early concerns were later assuaged through subsequent research and understanding. The references below include recent studies by statisticians from the RAND Corporation, a non-profit research organization:

• On the choice of a complex value-added model: McCaffrey, Daniel F., and J.R. Lockwood. 2008. "Value-Added Models: Analytic Issues." Prepared for the National Research Council and the National Academy of Education, Board on Testing and Accountability Workshop on Value-Added Modeling, Nov. 13-14, 2008, Washington, DC.
• On the advantages of the longitudinal, mixed model approach: Lockwood, J.R. and Daniel F. McCaffrey. 2007. "Controlling for Individual Heterogeneity in Longitudinal Models, with Applications to Student Achievement." Electronic Journal of Statistics 1: 223-52.
• On the insufficiency of simple value-added models: McCaffrey, Daniel F., B. Han, and J.R. Lockwood. 2008. "From Data to Bonuses: A Case Study of the Issues Related to Awarding Teachers Pay on the Basis of the Students' Progress." Presented at Performance Incentives: Their Growing Impact on American K-12 Education, Feb. 28-29, 2008, National Center on Performance Incentives at Vanderbilt University.

PVAAS in Practice

PVAAS uses multiple statistical models based on the objectives of the analyses and the characteristics and availability of the assessment data used.

• The growth standard methodology (also known as the multivariate response model or MRM) used in value-added analyses is a multivariate, longitudinal, linear mixed model. In other words, it is conceptually a multivariate repeated-measures ANOVA model. The growth standard methodology is used when scores are scaled or transformed so that the difference between two scores is meaningful and when there are clear "before" and "after" assessments in which to form a reliable gain estimate. In Pennsylvania, this is used for PSSA Mathematics and ELA, grades 4-8.
• The predictive model (also known as the univariate response model or URM) used in value-added analyses is conceptually an analysis of covariance (ANCOVA) model. The predictive model is used when the test data do not meet the requirements for growth standard analyses as stated above. In Pennsylvania, this is used in subjects that are not tested in consecutive grades, such as PSSA Science and Keystones.